Publisher / Subscriber Systems

In software architecture, publish–subscribe is a messaging pattern where senders of messages, called publishers, do not program the messages to be sent directly to specific receivers, called subscribers. Instead, published messages are characterized into classes, without knowledge of what, if any, subscribers there may be. Similarly, subscribers express interest in one or more classes, and only receive messages that are of interest, without knowledge of what, if any, publishers there are.

Let's tease apart what wikipedia says about pub/sub systems and see how these concepts map to the underlying architecture of HOPE.

Publishers

Senders of messages, called publishers, do not program the messages to be sent directly to specific receivers, called subscribers.

  • The term "protocol" is used to describe the message class, which is mapped to an actual runtime generated .NET class.
  • Publishers and subscribers are called "receptors."
  • Receptors can be publishers, subscribers, or both.

Messages

Published messages are characterized into classes.

  • Carriers consisting of a protocol (the actual name of the class containing the data) and the signal, the data itself.
  • Classes are described declaratively in XML, translated into C# code, and instantiated at runtime.
  • Publishers and subscribers do not "know" the runtime instance structure (as in, there usually is no pre-determined interface) thus type checking is done at runtime.
  • Publishers and subscribers manipulate the instance data using the "dynamic" keyword.
  • A publisher receptor declares what protocols it emits.  This is done mainly to support the visualizer in showing the protocol pathways between receptors (subscribers and publishers.)

Subscribers

Subscribers express interest in one or more classes, and only receive messages that are of interest, without knowledge of what, if any, publishers there are.

  • A subscriber receptor declares what protocols it receives.  This information is used by the visualizer to show protocol pathways

Subscriber Messages

In the pub/sub model, subscribers typically receive only a subset of the total messages published. The process of selecting messages for reception and processing is called filtering.

  • The protocols the subscriber receptor declares are used to filter subscriber receptors so that the carrier is "received" only by receptor subscribers interested in the carrier's protoocol.

Filtering

There are two common forms of filtering: topic-based and content-based.

  • Hope uses both topic-based (protocol) and content-based (signal) filtering.
  • In addition, HOPE will eventually incorporate more complex structure/content-based filtering, implementing concepts from the open source project Ceptr.

Topic Filtering

In a topic-based system, messages are published to "topics" or named logical channels. Subscribers in a topic-based system will receive all messages published to the topics to which they subscribe, and all subscribers to a topic will receive the same messages. The publisher is responsible for defining the classes of messages to which subscribers can subscribe.

  • Internally, HOPE maintains a map of logical channels that connect publisher receptors to subscriber receptors.  This map is updated at runtime as receptors are added, removed, and moved in and out of membranes (see below.)
  • As mentioned above, receptors define the protocols that they publish and those to which they subscribe.

Content Filtering

In a content-based system, messages are only delivered to a subscriber if the attributes or content of those messages match constraints defined by the subscriber. The subscriber is responsible for classifying the messages.

  • Subscriber receptors can optionally provide lambda expressions (as Func<bool, dynamic> types) that qualify the reception of a carrier, allowing the subscriber receptor to specify constraints on the signal.

Brokers

In many pub/sub systems, publishers post messages to an intermediary message broker or event bus, and subscribers register subscriptions with that broker, letting the broker perform the filtering.

  • HOPE implements two kinds of message brokers:
    • All receptors within a single collection use a "collection broker" for registration and and topic/content (protocol and signal) filtering.
    • "Membranes" maintain separate receptor collections.  A membrane is a broker that is user-configured (rather than configured by the receptors) to allow topics (protocols) to "permeate the membrane", either inward or outward.
    • Membranes do not currently support content filtering.

Broker Queuing and Prioritizing

The broker normally performs a store and forward function to route messages from publishers to subscribers. In addition, the broker may prioritize messages in a queue before routing.

  • The collection broker will normally forward a carrier immediately or, when the visualizer is used, it creates an Action<> lambda expression that is evaluated when the carrier reaches it target receptors.
    • If a receptor for the carrier exists (topic/content filtering evaluates to "true") the carrier is processed.
    • If no qualified receptors exist, the collection broker will either discard the carrier or queue the carrier (this is currently determined by how the publisher receptor creates the carrier.)
    • Queued carriers are processed once a qualified subscriber receptor comes into existence in some "reachable" receptor collection.  Reachable means either within the receptor collection of the publisher or through one or more membranes to another receptor collection.
  • The membrane broker can block a protocol (impermeability) or can be permeable, allowing the carrier to possibly reach the subscriber receptors in other receptor collections. 
    • If no qualified receptors exist, the collection broker will either discard the carrier or queue the carrier (this is currently determined by how the publisher receptor creates the carrier.)
  • Carrier prioritization is currently not implemented.

Subscriber Message Initialization

Subscribers may register for specific messages at build time, initialization time or runtime.

  • Subscriber registration occurs during initialization and at runtime (dynamic subscription.) 
  • Publisher registration occurs during initialization and at runtime (dynamic publishing.)
  • Subscription and publishing are determined by the receptor implementation -- the user does not have direct control over this process unless the receptor implements some interface to the user (UI, for example) that allows the user to change the pub/sub registration.
  • Membranes provide a mechanism for the user to create receptor collections and determine the permeability of carriers by topic (protocol.)

Dynamic Topology

Publishers are loosely coupled to subscribers, and need not even know of their existence. With the topic being the focus, publishers and subscribers are allowed to remain ignorant of system topology.

  • The topology of the system is dynamic, determined (either by the user or programmatically) by the current configuration of receptor collections.

Scalability

Pub/sub provides the opportunity for better scalability than traditional client–server, through parallel operation, message caching, tree-based or network-based routing, etc.

  • Carriers can be processed in parallel.
  • Carriers can be transmitted between physical systems.
  • Receptors can be implemented across systems:
    • distributing workload
    • providing redundancy
  • Receptors can operate asynchronously:
    • performing internal operations asynchronously
    • interfacing with other services (such as web services) asynchronously
  • Coordinating workflow in a distributed, asynchronous environment has many complexities which are only superficially addressed at the moment:
    • Guarantee of receipt and work -- did a receptor receive the carrier and complete the work?
    • Error handling -- did a receptor complete the work without error?
    • Error recovery -- how is an error handled?
    • Guarantee of sequential operation -- carrier C1 must be processed by receptor R before carrier C2 is processed
      • This is often an issue during initialization -- for example, guaranteeing the database tables/views exist before operating on those tables/views.
    • Query - Response coordination -- a receptor publishes a carrier and requires the result of the operation before continuing with its own process.
    • Dependencies -- a receptor requires multiple responses (usually from different receptors) before proceeding with its own process.
    • Required receptors -- what receptors are required in the collection for a receptor to perform its work, and how are these required receptors instantiated?

Issues / Concerns

The most serious problems with pub/sub systems are a side-effect of their main advantage: the decoupling of publisher from subscriber.

Testing, Traceability, and Repeatability

A pub/sub system must be designed carefully to be able to provide stronger system properties that a particular application might require, such as assured delivery.

  • Testing a HOPE applet is difficult due to the dynamic nature of the topology, both within and across membranes (receptor collections.)
  • Traceability across an asynchronous distributed system is complex.
  • Repeatability across an asynchronous distributed system is often not guaranteed.
  • Other problems (load surges, slowdowns, IP broadcast storms) have already been observed, especially when running applets that process thousands of web-scraping or web service operations.

Tightly Coupled Semantics

Pub/sub are tightly coupled, via event subscriptions and patterns, to the semantics of the underlying event schema and values.

  • It has already been seen that pub/sub pushes the issue of re-usability onto the semantics of the underlying event schema and values (protocol and signal.)
  • This often leads to minor variations (similar to database views) of the same signal, requiring unique schema (protocols.)
  • All these minor variations increase the semantic complexity and maintainability of the entire system and is an area in which work is being done to reduce this semantic complexity.

Security

For pub/sub systems that use brokers (servers), the agreement for a broker to send messages to a subscriber is in-band, and can be subject to security problems. Brokers might be fooled into sending notifications to the wrong client, amplifying denial of service requests against the client. Brokers themselves could be overloaded as they allocate resources to track created subscriptions.

Even with systems that do not rely on brokers, a subscriber might be able to receive data that it is not authorized to receive. An unauthorized publisher may be able to introduce incorrect or damaging messages into the pub/sub system. This is especially true with systems that broadcast or multicast their messages. Encryption (e.g. Transport Layer Security (SSL/TLS)) can prevent unauthorized access but cannot prevent damaging messages from being introduced by authorized publishers.

  • HOPE does not currently implement any encryption or Transport Layer Security.
  • Malformed or malicious carriers can be easily introduced into a HOPE applet.

Islands of Behavior

From observations of developing HOPE systems:

Pub/sub implements "islands of behavior"

  • This requires careful thought and planning to tease apart behaviors so that they can be distributed across multiple receptors.
  • This in turn increases the complexity of coordinating workflow (responses and dependencies) between islands of behavior.
  • The advantage is that this enforces an implementation that is inherently distributable and capable of asynchronous operation.
  • The disadvantage is that we are inherently lacking programming tools to:
    • Construct the scaffolding templates that are common for all islands of behavior (receptors.)
    • Assist in the thought processes necessary for reducing behavior to core tasks.
    • Assist in the assimilation of responses to form coordinated workflows.
  • In actual usage, receptors can be categorized as "high-level" and "low-level" (and numerous in-between categories)
    • High-level receptors issue carriers to low-level receptors and then perform work when all necessary responses are received.
    • High-level receptors often intermix operations that can be performed asynchronously by low-level carriers along with operations that must be performed sequentially by low-level receptors.
    • Low-level receptors (at the lowest level) have no dependencies on other receptors (though other receptors may subscribe to the incoming and outgoing protocols) and perform work autonomously.