MQ Service

Introduction

In TCUP (TCS Connected Universe Platform) the Message Queue service allow users to create consumer groups and manage ot modify the offsets(index for message stored in the underlying broker i.e apache kafka) with multiple parameters so that user can retriev old data/messages(which is already processed/consumed once) as per the requirement.

This service also provides credentials for other TCUP services or applications deployed on TCUP to access kafka elements(topic/consumer groups).

Purpose of the Document

The following section describes how to work on MQ service. After going through this document user will able to do the following activities using MQ portal

  • View user specific broker Account

  • Create/ Delete consumer group

  • Update offset(message pointer) details for a consumer group

  • Create/ Delete binding between a consumer group and topic.

  • Get total consumer group statistics and usage details.

Reference Document

Please refer to the following documents to get more details on Message Queue service:

  • To understand the basic concepts of MQ please refer to the Concept guide.

  • For API details of MQ please refer to the API guide .

API Details

All the APIs below are authenticated based on TCUP api-key. This key is used to uniquely identify each tenant and its application. In order to access service API the following custom header has to be passed as request header:

Custom Request Header

  • x-api-key : A mandatory header to pass along with every API call

List of API Services:

Accounts

  • create an mq-account

  • Get list of mq accounts

  • Delete an mq-account

consumerGroups

  • Create with different parameters

  • Get list of consumerGroups

  • Get the details of a consumerGroup by consumerGroup name

  • Update a consumerGroup by reseting the offset value

  • Delete a consumerGroups

Bindings

  • Create a binding between a consumerGroup and a topic

  • Delete a binding between a consumerGroup and a topic

The following RESTful APIs are present in swagger for accessing the service:

../_images/mq.PNG

Before we proceed with the operation on consumerGroups, TCUP Admin will first register the user to access MQ service with a unique x-api-key which is a one time process. User API is used to create and register a user which can only be accesed by admin. In order to create a user, the admin will use the following user API:

Create User

  • POST http://<domain name>/api/mq/v2.0/users

The following input JSON is used to create a user:

{
  "apiKey": "NZOlaIbIQikF9FoNbdbLsOwK9M=",
  "appKey": "",
  "restriction": ""
}

  • On clicking on SEND button of rest client with the above created JSON and POST http method, the user gets created and gets the repsonse from the API as mentioned in the image below:

../_images/mq1.PNG

Note

Above API service can only be accessed by TCUP admin.

Operation on Accounts

Create an MQ-Account

This API is used to create an mq-account in the underlying message broker. This API is accessible to the admin users only. The following is the URL for MQ create accounts:

  • POST http://<domain name>/api/mq/v2.0/accounts

** The following is default template for creating accounts:

{
        "apikey": "",
        "username": "",
        "password": "",
        "vhost": "",
        "host": ""
}

  • Description of create account JSON parameters is defined in the table below:

    Parameter

    Required

    Values

    Description

    apikey

    true

    string

    key that uniquely identifies a user.

    username

    true

    string

    username to access the underlying broker.

    password

    true

    string

    password to access the underlying broker.

    vhost

    false

    string

    vhost is deprecated.

    host

    true

    string

    host where the broker is running.

  • The following is the response list:

    Name

    Value

    Http Method

    POST

    URL

    /accounts

    HTTP Return Status

    200 Ok, 400 Bad Request, 403 Forbidden:Invalid API key, 404 Invalid Payload, 500 Internal Server Error

    Content Type

    application/json

  • On clicking on SEND button of rest client with above create JSON and POST http method, the account gets created and gets the repsonse from the API as mentioned in the image below:

../_images/mq2.PNG

Note

Above API service can only be accessed by TCUP admin.

Get List of MQ Accounts

This API is used to get the details of the message-queue accounts for a given x-api-key. The users can view their account details in the account details section of the MQ service portal page as shown:

../_images/mq3.PNG

The user can also get the same from swagger API as displayed below. The following is the URL for getting MQ account details:

  • GET http://<domain name>/api/mq/v2.0/accounts

  • Click on GET /v2.0/accounts API in swagger and hit Try it out! button, it wil display the corresponding account details as below:

../_images/mq4.PNG

Delete an MQ Account

This is used to delete all the details regarding the message-queue account user for a given user. The following is the URL to delete the MQ account details:

  • DELETE http://<domain name>/api/mq/v2.0/accounts/<apikey>

  • Just provide the apikey of the user in the url path and click on the SEND button of rest client.DELETE http method, the mq-account corresponds to the user will be deleted and user will get the repsonse from the API as mentioned in the image below:

../_images/mq5.PNG

Note

Above API service can only be accessed by TCUP admin.

Operation on consumerGroups

Create a consumerGroups

This API is used to create a consumerGroup in the underlying message broker. The following is the URL for creating queue:

  • POST http://<domain name>/api/mq/v2.0/consumerGroup

Following input JSON is used to create a consumerGroup:

{
     "name": "Q1"

}

Note: Valid characters for consumerGroupName are alphanumeric characters, periods (.), asterisk(*), underscores (_), and hyphens (-).

  • Following is the response list:

    Name

    Value

    Http Method

    POST

    URL

    /api/mq/v2.0/consumerGroup

    HTTP Return Status

    200 Ok, 400 Bad Request, 403 Forbidden:Invalid API key, 404 Invalid Payload, 500 Internal Server Error

    Content Type

    application/json

The following two methods are used to create a consumerGroup using Message Queue Service:

  • Using MQ Service API swagger

  • Using MQ Service portal

Using MQ Service API Swagger

  • On clicking on Try it out! button of the API swagger with the above created JSON and POST http method, the consumerGroup gets created through API and gets the repsonse from the API as shown in the following image:

../_images/mq6.PNG

  • Description of create queue JSON parameters is defined in the table below:

    Parameter

    Required

    Values

    Description

    name

    true

    string

    Name of the queue.

Using MQ Service Portal

You can also create consumerGroup through MQ Service portal as shown in the image below:

  • Go to home page of MQService portal and click on Create new Queue button as shown below:

../_images/mq7.PNG

  • Provide the consumerGroup name and click on the Register button to create the consumerGroup.

../_images/mq8.PNG

Note

consumergroup will not be visible in the portal utill it is binded with a topic

Get List of consumerGroup

This API is used to list all the consumerGroup along with the topics inside the consumerGroup that are created by a particular tenant in the underlying message broker. The consumerGroup list is displayed as shown in the image below:

  • GET http://<domain name>api/mq/v2.0/consumerGroup

  • Click on GET list of consumerGroup API in swagger and hit Try it out! button, it wil display all the consumerGroup created by the user as below:

../_images/mq9.PNG

User will get the following JSON:

../_images/getmq1.PNG

All the consumerGroup created by a particular user can also be viewed from the MQ Service portal as below:

../_images/mq10.PNG

Get Details for a consumerGroup

This API is used to provide the details of a particular consumerGroup created by a user along with the number of messages stored in the topic partition(i.e lag), current position off the consumer in the broker(cuurent offset), position of the last writtrn message in the broker and other details.

  • GET http://<domain name>/api/mq/v2.0/consumerGroup/<consumerGroup name>

  • Provide the name of the consumerGroup.

../_images/getq2.PNG

  • Click on GET /consumerGroup/<consumerGroup name> API in swagger and hit Try it out! button, it wil display all the details for the given consumerGroup created by the user as below:

../_images/mq11.PNG

  • If there is multiple topics present inside a consumergroup, then user can filter the group details by providing topic name as well.

  • Provide the name of the consumerGroup and the topic by which the group to be filterd.

../_images/getq3.PNG

  • Click on GET /consumerGroup/<consumerGroup name> API in swagger and hit Try it out! button, it wil display all the details for the given consumerGroup created by the user as below:

../_images/getq4.PNG

Delete a consumerGroup

This API is used to delete a given consumerGroup created by a particular user. The following is the URL for deleting a consumerGroup:

  • DELETE http://<domain-name>/api/mq/v2.0/consumerGroup/<consumerGroup name>

  • Click on DELETE /consumerGroup/<consumerGroup name> API in swagger, provide the name of the consumerGroup to be deleted and hit Try it out! button, it will delete the given consumerGroup created by user as shown in the image below:

../_images/mq12.PNG

To delete a consumerGroup form MQService portal follow the below steps:

  • Click on the delete beside the name of a particular consumerGroup as shown below:

../_images/mq13.PNG

  • Click on the Delete button to delete the consumerGroup.

../_images/delete.PNG

Binding consumerGroups

A binding is a relationship between an topic and a consumerGroup. This can be simply read as the consumerGroup being interested in messages from this topic.

Create a consumerGroup-bind

This API is used to bind a consumerGroup with a given topic in the underlying message broker. The following is the URL to bind a consumerGroup:

  • PUT http://<domain name>/api/mq/v2.0/consumerGroup/binding/<consumerGroup name>/<topic name>

  • Click on PUT /consumerGroup/binding/<consumerGroup name>/<topic name> API in swagger, provide the name of the consumerGroup and topic name created by user to be binded and hit Try it out! button, it will bind a consumerGroup with a given topic created by user as shown in the image below:

../_images/mq14.PNG

  • The following is the response list:

    Name

    Value

    Http Method

    PUT

    URL

    consumerGroup/binding/<consumerGroup name>/<topic name>

    HTTP Return Status

    200 Ok,400 Bad Request, 403 Forbidden:Invalid API key, 404 Invalid Payload, 500 Internal Server Error

Delete a consumerGroup-bind

This API is used to delete binding of consumerGroup with a given topic in the underlying message broker. The following is the URL to bind a consumerGroup:

  • DELETE http://<domain name>/api/mq/v2.0/consumerGroup/binding/<consumerGroup name>/<topic name>

  • Click on DELETE /consumerGroup/binding/<consumerGroup name>/<topic name> API in swagger, provide the name of the consumerGroup and topic name created by user to be unbinded and hit Try it out! button, it will delete the binding of consumerGroup with topic created by user as shown in the image below:

../_images/mq18.PNG

  • The following is the response list:

    Name

    Value

    Http Method

    DELETE

    URL

    consumerGroup/binding/<consumerGroup name>/<topic name>

    HTTP Return Status

    200 Ok,400 Bad Request, 403 Forbidden:Invalid API key, 404 Invalid Payload, 500 Internal Server Error