Getting Started

Complex Event Processing

The primary goal of BoboCEP is to take streaming data that enters a system in a serialised and uncorrelated manner (i.e. simple events), and detect temporal patterns using it to infer the occurrence of some higher-level phenomenon. A complex event may be generated if a pattern of a phenomenon is fulfilled with relevant data. See Phenomena for more information.

On pattern fulfilment, an action may be taken in response which, in turn, leads to the generation of an action event representing what happened during action execution and whether it was successfully executed. See Actions for more information.

Complex events and action events are added back into the system’s data stream, where they can be used for further pattern detection.

Example

An office may wish to detect the phenomenon of an office fire through various data patterns that could infer its occurrence. Patterns may include:

1. A sharp rise in temperature sensor readings, followed by smoke detection within 1 minute of the rise in temperature.

2. Significant movement away from working spaces and towards the fire exit.

3. Or, it may simply be the pressing of the fire alarm, with no further correlations necessary.

When any of these patterns are fulfilled by the expected data, a number of actions may be triggered as a response: fire alarms sound, sprinklers are activated, and so on.

BoboCEP Architecture

The architecture of BoboCEP is based on the information flow processing (IFP) architecture proposed by [CM2012]. This architecture is extended by enabling state updates to be synchronised across multiple instances of BoboCEP across a network for fault tolerance.

BoboCEP architecture and external systems. Subsystems within dashed border are the core subsystems for a given BoboCEP instance. Dashed arrows represent data exchange to and from external systems (blue, grey).

Subsystems

BoboCEP is powered by a BoboEngine instance, which maintains data flow through its subsystems, as follows.

• Receiver. The entry point for data into the system. Its purpose is to validate incoming data and then format it into a simple event. It also consumes complex events and action events and introduces these event types into the data stream.

  • Data of Any type enters BoboReceiver via the add_data method.

  • If the data is an instance of type BoboEvent, then it is passed to BoboDecider as-is.

  • Otherwise, Receiver will wrap the data in a new BoboEventSimple instance e, where e.data is the data passed via add_data, and will send e to Decider instead.

• Decider. Manages runs, which represent patterns that have not yet received all of the data necessary to indicate the occurrence of some phenomenon.

  • A BoboPhenomenon contains one or more BoboPattern instances and a BoboAction instance.

  • A BoboRun is created when the first predicate of a BoboPattern is satisfied with some BoboEvent.

  • Once a run has had its final predicate satisfied, Producer is notified.

• Producer. Generates complex events when it receives notification from Decider that a run has completed and, therefore, a phenomenon has been observed which the complex event represents.

  • When a Run has completed, BoboProducer will produce a BoboEventComplex instance, indicating the observation of the phenomenon that the pattern is evidence of.

  • Producer passes the complex event to Forwarder, in order to trigger the phenomenon’s corresponding BoboActon. It also passes the complex event to Receiver, which adds it to the data stream.

• Forwarder. Forwards actions passed to it by Producer and executes them. This may involve communication with external services. Each action leads to the generation of an action event that details what occurred during action execution.

  • BoboForwarder the complex event to determine which action is needed to execute.

  • It executes the action and produces a corresponding BoboEventAction that contains reference to the complex event which led to its execution.

  • The BoboEventAction is passed to Receiver and added to the data stream.

• Distributed. This architecture is extended by enabling state updates to be synchronised across multiple instances of BoboCEP.

  • See Distributed for more information.

Quick Start

The key components to getting started with BoboCEP are as follows.

1. Define the Phenomena that you would like to model by defining one or more patterns per phenomenon. Use BoboPatternBuilder for defining patterns to make things much easier.

2. Define Actions that should be executed if a phenomenon were to be triggered. Allocate an action to a phenomenon if you wish, or leave it blank.

3. Decide whether you want BoboCEP to be Distributed or not, and use one of the setup classes to help with setting up the system engine and all of its components: BoboSetupSimple and BoboSetupSimpleDistributed are provided for these purposes.

Check out the Examples page for various ways to set up BoboCEP and connect it to external systems e.g. Flask.

To explore the API in more detail, see Source Code.

Why “Bobo”?

Bobo is the name of Mr Burns’ childhood teddy bear that features in the episode “Rosebud” of The Simpsons.

References

CM2012

Cugola, G., & Margara, A. (2012). Processing flows of information: From data stream to complex event processing. ACM Computing Surveys (CSUR), 44(3), 15.