Sensor Observation Service

1. Introduction

Sensor Observation Service (SOS) provides the sensor data management functions in TCS Connected Universe Platform. This service supports any type of sensor and observations and facilitates storage, persistence, query and retrieval of sensor observation data.

This service enables data ingestion from sensors, devices and IoT gateways to backend data stores. It provides facility for registering new sensors and features to the platform and insert observations and measurements from registered sensors.

1.1 Intended Audience

The intended audience of this document is anyone who wants to have an overview of TCUP Sensor Observation Service. After going through this document, the user will understand the capability of TCUP Sensor Observation Service in IoT platform.

2. Key Concepts

In order to use the Sensor Observation Service, a user needs to understand some of the basic concepts and building blocks of the service. Please refer to the following section for the concepts:

2.1 Feature

Any real world entity, ‘asset’ or ‘thing’ is referred to as a ‘Feature of Interest’ in SOS. e.g. a room, an engine etc.

User can use API to create feature in SOS by specifying the unique feature name. They can also add metadata like geo location of the features and create hierarchy of features by specifying the details. APIs are also available to query, update and delete features.

2.2 Sensor

A sensor is anything that can observe a real-world entity and can generate observations or measurements related to one or more properties of a feature as an output. Sensor can be of multiple types:

2.2.1 Type of Sensor

  • Atomic sensor vs Composite sensor

    • Atomic sensor - A single sensor which is generally used for specific purpose e.g. temperature sensor.

    • Composite sensor - A sensor which is composed of other sensors, generally used for multiple purposes e.g. weather station (comprises of temperature sensor, humidity sensor etc.).

This type of sensor can be created by specifying parameter ‘isComposite’ as “Y” or “N”.

  • Movable sensor vs In-situ sensor

    • Movable sensor - Sensor installed in a movable platform/device e.g. sensor installed for vehicle tracking.

    • In-situ sensor - Sensor installed in a fixed position/platform which is not moving e.g. weather station sensor.

This type of sensor can be created by specifying parameter ‘isMobile’ as “Y” or “N” in JSON.

  • Physical sensor vs Soft-sensor

    • Physical sensor - A physical device installed in a physical location.

    • Soft-sensor - Software program/bot hosted in a real/virtual system acts as a sensor.

2.2.2 Operation on Sensor

Users can create, update, query and delete sensor using APIs. Sensor can be created by specifying the following:

  • sensor name - Sensor name must be unique

  • observation properties (output details to be observed) - Sensor can have one or more observed properties.

  • feature name (optional)

  • geo location (optional) - SOS supports both global and local location information. Position-global data can have maximum 1 set of values whereas position-local data can have multiple set of values.

  • other metadata (optional)

2.3 Observation Property

This refers to output properties/ parameters that needs to be captured by a sensor.

Example: A temperature sensor can have a observed property like temperature or temp. The sensor will capture/ measure the value of that observed property during observation.

2.4 Observation

An observation is the value of observed property captured by a particular instance of a sensor and is always associated with a timestamp. SOS provides APIs to insert, search and delete observation. User can post a single observation or can upload in bulk using CSV file.

Based on time, observation can be of two type: Instantaneous observation vs Long-running observations(on basis of Event time)

  • Instantaneous observations - The start time instance when sensor captured the observed property value.

  • Long-running observations - If observation start time and end time is different , SOS will consider it as a long-running observation and then the event time is considered as start time to end time.

Record time - The time when the data is actually inserted to the database.

Valid till time - The validity end time of that observation data.

Note

Observation once entered in the system cannot be modified.

2.5 Capabilities

Capability is an interface for SOS service which provides an overview of what this service can do. In other words it is a way to discover the service. The corresponding API allows to fetch all available components of SOS like sensor, feature of interest or any kind of output etc.

2.6 Batch Operations

Following batch operations are allowed

BATCH INSERT

BATCH UPDATE

BATCH DELETE

Feature

Yes

Yes

No

Sensor

Yes

Yes

No

Observation

Yes

No update operation

Yes

3. Functional Capabilities

The Sensor Observation Service provides the following functional capabilities:

  • Sensor registration – SOS allows a user to register a sensor along with its observed properties.

  • Feature registration – Allows to register all real-world entities, ‘things’, assets etc. along with their properties.

  • Data capture – SOS provides APIs for ingesting observations or events as time series against sensors and corresponding features.

  • Query of sensor data – SOS provides a rich set of query of Sensor and Observation data, typically parametrized by time stamp values, time ranges, Sensor IDs, Feature IDs, Geo-spatial coordinates and many more. Moreover, data sets returned can be filtered based on property values and meta-data as well.

  • SOS provides support for any arbitrary sensor or observation type, including blobs and unstructured sensor observations.

  • SOS provides support for location based query (geo-spatial queries).

  • SOS comes in two variants. One variant uses Hadoop and the other uses relational database as back ends.

  • Real-time sensor feeds – SOS can stream real-time data into clients (such as browsers) directly from the server in form of feeds. The clients need to subscribe to the feed and SOS then pushes live data directly to the client, without requiring the client to call SOS query APIs.

  • SOS is designed to support large volume of sensor observations and measurements from multiple applications. SOS leverages big data technologies to provide the ability to handle large number of sensors and large volume of sensor data. It uses a distributed in-memory cache to enhance query performance.

  • Ensure security of data at rest and protects sensitive data.

4. Purpose/Usage

SOS is a key component of TCUP since it provides basic functionality required for describing the real-word and capture / ingestion of data from the real-world. Many applications can be built simply using SOS only.

Some of the common usage patterns are as follows:

  • Transmission of measurements/ event data from Edge devices / gateways using SOS API calls, persistence of the observation data within SOS’ databases and retrieval of the same using SOS query APIs.

  • Uploading of data from files, such as CSV files, consisting of time-series data into SOS.

  • Generating visualization/ graphs/ charts using SOS query APIs or real-time feeds. Data from SOS query APIs is returned as JSON documents and can be easily rendered using any charting or graphing utility.

  • Analytics programs can pull data from SOS service using query APIs and generate and execute models. For example analytical programs written in languages such as Python can call SOS query APIs to retrieve historical data.

5. Example

Consider an example of an Engine Monitoring System. The Engine has an Electronic Control Unit (ECU) attached and this ECU is connected to a gateway via the CANbus. The gateway is connected to TCUP via a wireless internet connection. The gateway runs an embedded Linux operating system.

A data acquisition software is running on the gateway. This software uses Sensor Observation Service to push data to TCUP cloud. The application is modelled in TCUP as follows:

  • The engine is the Feature of Interest. The Engine Serial Number is mapped on to the Feature ID.

  • The ECU is considered as the Sensor. The ECU serial number is considered as the Sensor ID

  • Engine Speed, Engine Power Temperature, Pressure etc. are observed properties of the sensor.

The data acquisition software collects the data from the CANBus and pushes the data to TCUP using the SOS insert observation API.

An application on the cloud side uses SOS Get Observation API to retrieve data and generate reports as needed.

6. Reference Document

Please refer to the following documents for more details about this service:

  1. API Guide