| Table of Contents |
|---|
Key Points
...
Message Brokers include: IBM MQ, ActiveMQ, RabbitMQ, Kafka etc
While ESB tools are a reliable way to connect applications, their technology is becoming more obsolete as the cloud dominates digital ecosystems and as companies experience a rate of growth that requires even faster adaptability across their systems.
ESB comeback?
unless you are running everything on a single cloud platform and have low needs for integration with customers, vendors and partners, an ESB ( or Event Bus ) may speed integration. If everything you need is on your cloud platform you are probably fine using just what AWS, Azure etc offer as commercial services.
Kafka as a Message Broker
...
ActiveMQ implements the functionality specified in the Java Message Service (JMS) API, which defines a standard for creating, sending, and receiving messages. ActiveMQ client applications—producers and consumers—written in Java can use the JMS API to send and receive messages. Additionally, both Classic and Artemis support non-JMS clients written in Node.js, Ruby, PHP, Python, and other languages, which can connect to the ActiveMQ broker via the AMQP, MQTT, and STOMP protocols.
Artemis easier to maintain, which is important if you're basing a commercial product on it. The smaller feature set means a smaller overall implementation, which fits well with developing microservices.
Implementing Apache ActiveMQ-style broker meshes with Apache Artemis
Artemis easier to maintain, which is important if you're basing a commercial product on it. The smaller feature set means a smaller overall implementation, which fits well with developing microservices.
This article describes subtleties that can lead to lost messages in an Artemis active-active mesh. That architecture consists of multiple message brokers interconnected in a mesh, each broker with its own message storage, where all are simultaneously accepting messages from publishers and distributing them to subscribers. ActiveMQ and Artemis use different policies for message distribution. I will explain the differences and show a few ways to make Artemis work more like ActiveMQ in an active-active scenario.
Artemis supports broker discovery for message meshes
The final vital piece of configuration assembles the various broker connectors into a mesh. Artemis provides various discovery mechanisms by which brokers can find one another in the network. However, if you're more familiar with ActiveMQ, you're probably used to specifying the mesh members explicitly.
Artemis provides a number of discovery mechanisms, allowing clients to determine the network topology without additional configuration. These don't work with all wire protocols (notably, there is no discovery mechanism for Advanced Message Queuing Protocol), and ActiveMQ users are probably familiar with configuring the client's connection targets explicitly. The usual mechanism is to list all the brokers in the mesh in the client's connection URI.
you should be able to connect consumers to all the nodes, produce messages to any node, and have them routed to the appropriate consumer. However, this mesh won't behave exactly like ActiveMQ, because Artemis mesh operation is not governed by client demand.
Message forwarding in Artemis is more flexible than Classic ActiveMQ
In ActiveMQ, network connectors are described as "demand forwarding." This means that messages are accepted on a particular broker and remain there until a particular client requests them. If there are no clients for a particular queue, messages remain on the original broker until that situation changes.
...
Written in Erlang, the RabbitMQ server is built on the Open Telecom Platform framework for clustering and failover. Client libraries to interface with the broker are available for all major programming languages. The source code is released under the Mozilla Public License.
Message Broker
A message broker (also known as an integration broker or interface engine[1]) is an intermediary computer program module that translates a message from the formal messaging protocol of the sender to the formal messaging protocol of the receiver. Central reasons for using a message-based communications protocol include its ability to store (buffer), route, or transform messages while conveying them from senders to receivers.
The following represent other examples of actions that might be handled by the broker:[2][3]
- Route messages to one or more destinations
- Transform messages to an alternative representation
- Perform message aggregation, decomposing messages into multiple messages and sending them to their destination, then recomposing the responses into one message to return to the user
- Interact with an external repository to augment a message or store it
- Invoke web services to retrieve data
- Respond to events or errors
- Provide content and topic-based message routing using the publish–subscribe pattern
The Advanced Message Queuing Protocol (AMQP)
...
Streaming Text Oriented Message Protocol (STOMP)
Simple (or Streaming) Text Oriented Message Protocol (STOMP), formerly known as TTMP, is a simple text-based protocol, designed for working with message-oriented middleware (MOM). It provides an interoperable wire format that allows STOMP clients to talk with any message broker supporting the protocol.
- RabbitMQ (message broker, has support for STOMP)
MQTT is a lightweight, publish-subscribe network protocol that transports messages between devices. The protocol usually runs over TCP/IP, however, any network protocol that provides ordered, lossless, bi-directional connections can support MQTT.[1] It is designed for connections with remote locations where resource constraints exist or the network bandwidth is limited. The protocol is an open OASIS standard and an ISO recommendation (ISO/IEC 20922).
Compare Kafka and Rabbit MQ
...
https://www.npmjs.com/package/mitt
// listen to an event emitter.on('foo', e => console.log('foo', e) ) // listen to all events emitter.on('*', (type, e) => console.log(type, e) ) // fire an event emitter.emit('foo', { a: 'b' })
Simple Java Events - use Property Beans for objects in the SAME process - see groovy event example
...
Microsoft Azure Event Grid is a fully managed event routing service provided by Microsoft Azure. It provides a highly scalable and highly available event infrastructure for event-driven architectures and serverless applications. Event Grid supports a variety of event sources and can route events to multiple event handlers in real-time
it charges based on the number of events processed and the number of operations performed
may not provide the level of customization and flexibility required for some scenarios.
can limit interoperability with other platforms and technologies that use different event processing solutions.
Google Cloud Pub/Sub
https://cloud.google.com/pubsub
Google Cloud Pub/Sub is a fully managed messaging service provided by Google Cloud, that allows applications to publish and subscribe to messages in a scalable and reliable manner. Pub/Sub supports a variety of programming languages and platforms and provides powerful features for handling events, such as dead-letter topics, push and pull delivery, and message filtering.
it charges based on the number of messages processed and the amount of data stored.
can limit interoperability with other platforms and technologies
Amazon Simple Notification Service (SNS)
...
Amazon Simple Notification Service (SNS) is a fully managed messaging service provided by Amazon Web Services (AWS), that enables applications to send and receive notifications. It provides a simple and scalable messaging infrastructure for event-driven applications. SNS supports a variety of event sources and can route events to multiple event handlers, including email, SMS, and other AWS services.
it charges based on the number of messages processed and the amount of data stored.
can limit interoperability with other platforms and technologies
Apache EventMesh
https://github.com/apache/eventmesh
Event Mesh on Low-code App Development Solutions swtc page
Apache EventMesh is a new generation serverless event middleware for building distributed event-driven applications.
Apache EventMesh itself has four primary components:
- eventmesh-runtime : an middleware to transmit events between event producers and consumers, support cloud native apps and microservices.
- eventmesh-sdk-java : currently supports HTTP and TCP protocols.
- eventmesh-connector-api : an api layer based on OpenMessaging api and SPI pluggin, which can be implemented by popular EventStores such as IMDG, Messaging Engine and OSS etc.
- eventmesh-connector-rocketmq : an implementation of eventmesh-connector-api, pub event to or sub event from RocketMQ as EventStore.
Apache EventMesh provides standard protocol such as cloud events; interface such as http, tcp etc.; and pluggable storage engine with the Apache RocketMQ as the default back-end storage. Compared to other mesh-based applications in the market, our platform supports fancy features like extreme low latency and stability, and cloud-native architecture
need to address complexity brought in by heavy clients based on the pull model; event-based streaming; and, the openness of event metadata and mixed media storage
Apache EventMesh has a vast amount of features to help users achieve their goals. Let us share with you some of the key features EventMesh has to offer:
...
Event Mesh - open-source cloud-native eventing infrastructure that decouples the application and backend middleware layer,
http://eventmesh.incubator.apache.org/
Features
- Communication Protocol: EventMesh could communicate with clients with TCP, HTTP, or gRPC.
- CloudEvents: EventMesh supports the CloudEvents specification as the format of the events. CloudEvents is a specification for describing event data in common formats to provide interoperability across services, platforms, and systems.
- Schema Registry: EventMesh implements a schema registry that receives and stores schemas from clients and provides an interface for other clients to retrieve schemas.
- Observability: EventMesh exposed a range of metrics, such as the average latency of the HTTP protocol and the number of delivered messages. The metrics could be collected and analyzed with Prometheus or OpenTelemetry.
- Event Workflow Orchestration: EventMesh Workflow could receive an event and decide which command to trigger next based on the workflow definitions and the current workflow state. The workflow definition could be written with the Serverless Workflow DSL.
...
Apache EventMesh (Incubating) is a dynamic event-driven application multi-runtime used to decouple the application and backend middleware layer, which supports a wide range of use cases that encompass complex multi-cloud, widely distributed topologies using diverse technology stacks.
EventMesh roadmap
https://eventmesh.apache.org/docs/roadmap/
...
Apache EventMesh Documentation
Apache EventMesh (Incubating) has a vast amount of features to help users achieve their goals. Let us share with you some of the key features EventMesh has to offer:
- Built around the CloudEvents specification.
- Rapidly extensible language sdk around gRPC protocols.
- Rapidly extensible middleware by connectors such as Apache RocketMQ, Apache Kafka, Apache Pulsar, RabbitMQ, Redis, Pravega, and RDMS(in progress) using JDBC.
- Rapidly extensible controller such as Consul, Nacos, ETCD and Zookeeper.
- Guaranteed at-least-once delivery.
- Deliver events between multiple EventMesh deployments.
- Event schema management by catalog service.
- Powerful event orchestration by Serverless workflow engine.
- Powerful event filtering and transformation.
- Rapid, seamless scalability.
- Easy Function develop and framework integration.
EventMesh at Webank
https://thestack.technology/apache-event-mesh/
...
Node-RED best pracices video
https://www.youtube.com/watch?v=V0SmNcIYCtQ
Solace Message Service and Event Hub with EDA - Event Driven Architecture ( compare to EventMesh, Kafka,
...
CouchDb
...
)
Solace master data management with event mesh pub sub
...
- What events already exist in a system? How can I subscribe to various events?
- Who created this or that event? And who can tell me more about it?
- What are my organization’s best practices and conventions for event stream definitions?
- What topic structure should I use to describe the events my application will produce?
- How is this event being used? By whom? By what applications? How often?
- Is the change I want to make to my application or event backward compatible?
- How can I control access to this or that event stream?
- Do capabilities include: replay? recovery? resiliency? traceability? dynamic configuration, testing?
Potential Value Opportunities
Conceptual Value Chain Services
multiple logical domains exist on a logical value chain
...
Compare Event Mesh and Service Mesh Concepts - Solace perspective
Most modern microservices rely on both synchronous and asynchronous interactions, so for most companies it makes sense to deploy both a service mesh and an event mesh. From an infrastructure point-of-view, both can operate in the world of cloud virtualization and Kubernetes, but the event mesh can also connect event nodes that are cloud-native (with or without Kubernetes), on-premises, and even bare metal. In this way, an event mesh expands the reach of connected microservices and gives developers more deployment options. Below is a simple example where Microservice A, driven by synchronous interactions, can inform other microservices of an event having occurred (e.g., “account opened”) by sending an event to an event broker. Other microservices (C, D and E) can perform actions as a result of this event. In general, it’s good design practice to use synchronous interactions when you need to and make as many interactions as possible asynchronous
Potential Value Opportunities
Conceptual Value Chain Services
multiple logical domains exist on a logical value chain
parties can be authorized to participate by role
...
see Modern Platform ODP supports VCE , FMIDAN, STS for more details
Potential Challenges
...