Cloud Sink with Azure Event Hub

In this Jumpstart we will create a new Microsoft Azure Event Hub with Kafka support and configure it. We will then use the system:inmation Cloud Sink object as a producer to send a message to Azure Event Hub using the Kafka protocol.

Prerequisites

To complete the examples in this Jumpstart the following things are required:

  • Azure account

  • system:inmation installation (v1.54 and above) including DataStudio

Preparing Azure Infrastructure

  1. Login to Azure Portal. Click on   Create a resource   button

  2. Add a new Event Hubs

    Event Hubs namespace
    Figure 1. Adding a new Event Hubs namespace
  3. Make sure you have the following parameters set using the suggested values or use the below screenshot as a guide:

    Kafka protocol is supported only in Standard pricing tier
    • Enable Kafka - Checked

    Create Namespace
    Figure 2. Event Hubs Namespace configuration
  4. Click   Create   and wait until the resource is deployed

  5. When Event Hub is up and running, select it under   All Resources  .

  6. Create a new Event Hub. Give the object a name and make note of it (this will also be used in the configuration of the Cloud Sink object and as a Kafka Topic).

    A new Event Hub
    Figure 3. Adding a new Event Hub
  7. Select the Event Hub object you just created and find the connection string to configure under the RootManageSharedAccessKey Policy. Make a note of it for later when configuring the the Cloud Sink object.

    Connection String
    Figure 4. Obtaining Connection String
  8. Download the cacert.pem certificate file available here

Cloud Sink configuration

  1. In DataStudio create a new Cloud Sink object by selecting a Connector and right-clicking. Select Admin  New  History  Cloud Sink from the context menu. Give the object a name a click Create to create it in the I/O Model.

  2. Select the Cloud Sink object and in the Object Properties panel, open the Configuration property section.

  3. Under Kafka Producer Parameters enter for the Topic property, the Event Hub name from Preparing Azure Infrastructure: Step 6

    Cloud Sink Topic configuration
    Figure 5. Cloud Sink Kafka configuration
  4. Under Global Configuration Properties, enter the following values for the parameters:

    Azure Kafka service is listening to port 9093
    Cloud Sink Global configuration
    Figure 6. Cloud Sink Global configuration
  5. Click Apply in the Object Properties to save configuration.

  6. To check that the interface is working, double-click on Cloud Sink object’s Faceplate and enter a string in the Write Value dialog.

    Writing to Cloud Sink
    Figure 7. Writing a value to Cloud Sink Item Value
  7. Now check the Azure Event Hub metric to see that the message has been captured.

    Metric Result
    Figure 8. Azure Event Hub metric with the number of incoming messages
  8. Alternatively you can install the Azure Event Hub Explorer extension for Visual Studio Code and check the result there

    Explorer Result
    Figure 9. An incoming message reported by Azure Event Hub Explorer extension