Event Service

1. Introduction

Event Service allows to do operation on Redis stream. This service allows faster insertion and retrieval of time-series sensors data in stream as well as to discard data which is out of order. This service offers four API’s to:

  • ‘insert’ in Redis stream,

  • ‘get’ from Redis stream,

  • ‘get’ from Redis stream by ID

  • ‘delete’ from Redis stream by IDs

1.1 Intended Audience

The intended audience of this document is anyone who wants to have an overview of TCUP Event Service. After going through this document, the user will understand the capability of TCUP Event Service in IoT platform.

2. Key Concepts

In order to use TCUP Event Service, the user needs to understand the basic concepts of the service. Please refer to the following section:

2.1 Stream

Stream is a new data type introduced with Redis 5.0. The following are some important features of stream data type:

  • This is an append only data structure, once the data is inserted into streams it cannot be altered.

  • Supports both in order as well as out of order data insertion.

  • Data is inserted in streams as key-value pair.

  • Once data is inserted in stream, a unique ID is generated which is composed of two parts namely <millisecondsTime>-<sequenceNumber>

  • Stream insertion is very fast.

  • Stream can be used as Event Source.

2.2 Insert Data in Stream

API is available to insert data in stream. Bulk observation data can be entered by specifying the stream name and event ID. This API can also set the stream expiry time using date or duration.

2.3 Query Stream Data

API is available to query specific stream event(s) data by specifying stream name and other parameters like start time, end time of the observed data against a stream etc. This service also allows to query in order or out of order data.

2.4 Query Stream Data by ID

API is available to query specific stream event(s) data by specifying stream name and event ID of the observed data against a stream. This service also allows to query in order or out of order data.

2.5 Delete Stream Data

API is available to delete single, multiple or all data/ events of a given stream.

3. Functional Capabilities

The Event Service provides the following functional capabilities:

  • Supports both in order as well as out of order data based on user requirements.

  • As of now SOS and flat JSON is supported as payload format for insertion in streams, it supports bulk observation insertion as well.

  • Response will be flat JSON while the query is performed on stream.

  • Query response will come in stream (chunk) therefore huge number of records can be fetched for the client if needed.

  • User can perform time series query using start time and end time of observation. Count based queries with latest and old values are also supported.

  • User can set the stream expiry time using date or duration.

  • User can also query with sensor/ feature, if sensor/ feature is used as streamName while creation, query with streamID is also possible.

  • User can perform filtering operations, the supported operations are >,>=, <, <=, ==, !=, like.

  • User can also perform aggregation operations, supported functions are sum, avg, count, max, min.

  • User can subscribe to a stream for the latest data.

  • User can delete from stream using specific streamID’s separated by comma. An entire stream can also be deleted.

  • Multitenancy is supported.

  • Subscribe via SSE/ websocket to ensure no loss of messages.

4. Purpose/Usage

TCUP Event Service is useful for the applications where out of order data needs to be discarded. It can be used to store events generated from MR/CEP. Window data stream can be created such as last ‘N ‘number of messages, last 7 days, last 1 day, last 1 month data etc. This service can be used to join streams and also generate events on key expiry time.

5. Examples

Considering a situation where alerts get generated for patients when their systolic pressure is greater than 130 and diastolic pressure is greater than 90. Complex Event Processing (CEP) rule is used to generate the alerts when the condition is satisfied.

In case later we want to track the patients whose conditions were satisfied to find out the number of patients who needed medical intervention, CEP alone will not serve the purpose. One action rule can be written which will consume the data from CEP output topic exchange and post the data to stream. Here all the events which satisfies the condition will get inserted into the stream. These events can be tracked using the APIs provided in Event Service such as time series query, query with sensor or feature by applying filtering or aggregation on those events etc.

6. Reference Document

Please refer to the following documents for more details about this service:

  • API Guide