The Pub/Sub architecture, short for Publisher/Subscriber, allows for an asynchronous streaming service based on message queues. It introduces modularity, robustness, and low latency in a network and keeps a backlog of messages. In case of a failure, messages are saved until the end subscriber at the intended destination confirms the delivery.

The Pub/Sub architecture

Pub/Sub is based on relationships between Publishers, who send messages to topics, and Subscribers, who receive the messages through a subscription to topics. The service can push the message to the subscriber's endpoint, or the subscriber can extract it. Once the message is received, the subscriber sends an acknowledgment that allows the service to remove the respected message from an automatically maintained storage.

Why use a Pub/Sub architecture?

Let’s take a look at how the Pub/Sub service can help solve real-world problems.

Any complex application consists of many different microservices with components that work together to provide the application's value. Let's consider a model diagram for a basic eCommerce service that involves interlinked dynamic sub-services, or microservices, acting as its components:

• Front-end user interface
• Packaging and/or warehouse services
• Distribution, shipping, or transportation services
• Updates to the user interface

Let us look at the relationships of a conventional model consisting of the above microservices/components:

As soon as the user places an order, the front-end interface has to notify the packaging services at the warehouse, which the shipping vendor then picks up at every stage of progress, and the user is notified as well. Every step/node needs to be linked to the alert service.

The problem? The interdependent system is vulnerable to failures.

1. Service failure

If one of the nodes or services collapses, there is no framework to recover the order details and start again.

2. Downtime loss

If any of the services, most probably managed by third-party vendors, face downtime for testing or maintenance, the communications between all the services will have to halt and shut down.

3. Scalability resistance

If more warehouses or multiple shipping services have to be installed, with numerous incoming orders, the network would have to determine which warehouse for packaging to contact, but it has no built-in mechanism to ensure knowledge of the traffic directed towards a given warehouse. One might get overwhelmed while others idle.

4. Integration challenges

If a new service, such as data analytics, is added to the network, many components will require modification to connect to the new service node, which is a tedious task.The continuous nature of the network where one component comprehensively depends on others is also known as an asynchronous network. A Pub/Sub architecture solves these problems by adding an element of asynchronization. Instead of an intricate, interdependent network, the Pub/Sub streaming service acts as a robust, consistent, failure resilient, serverless, and reliable message broker that is responsible for communication between all the microservices or nodes.

The new architecture, courtesy of a pub/sub service, means that every component is linked to the central broker, removing the challenges brought about by a complex, interdependent relationship; this is known as loose coupling between system components. Even if the subscriber gets disconnected, all the messages are stored until a connection is re-established and acknowledged.

Advantages of Pub/Sub streaming service

On top of being Integration and scalability-friendly, Pub/Sub streaming has several advantages.

1. Modularity

The decoupling of microservices or allows an increase in the extent to which flexibility and variety can be added to the nodes.

2. Maintainability

Independent services or components are easier to debug, improve and update due to the availability of asynchronous operations.

3. Reusability

Loosely coupled components are much easier to re-use in other software or networks.

4. Dependability

Since the messages are stored in a backup log until an acknowledgment from the end-point subscriber is received, accounting for any failure makes it highly dependable.

Similarly, there are multiple subscription models that end-users can subscribe to.

• Exclusive mode only allows a single Subscriber.
• Shared mode allows multiple Subscribers with retention and recoverable message delivery mechanism on failure.
• Failover mode also allows multiple subscribers. However, messages are only sent to one master consumer at a given instance in time.

Summary

The asynchronous nature of the Pub/Sub streaming service solves several challenges, including Integration, Scalability, and enhancing quality aspects such as Modularity, Maintainability, Reusability, and Dependability.

Macrometa Pub/Sub

Macrometa offers an extensive range of ready-to-go industry solutions that leverage geo-replicated Pub/Sub streams and real-time data aggregation, transformation, and analytics.