Cloud Sink with Azure IoT Hub

In this Jumpstart we will create a new Azure IoT Hub, which supports MQTT communications, and configure it. Next we are going to use system:inmation Cloud Sink as a publisher to send a message to Azure IoT Hub using the MQTT protocol.

Prerequisites

To complete the examples in this Jumpstart the following things are required:

  • Azure account

Azure account which is going to be used should be initially configured, meaning that there already should be a valid Azure subscription and all the necessary permissions to add and manage resources. There also should be at least one Resource Group. More information about Resource Groups is available on the the official Microsoft Documentation webpage
  • Visual Studio CODE (for configuring a device with SAS Token Authentication)

  • OpenSSL (for configuring a device with X.509 CA Authentication)

  • system:inmation with Data Studio

Preparing Azure Infrastructure

  1. Login to Azure Portal and click on the   Create a resource   button

  2. Find and add a new IoT Hub

    IoT Hub
    Figure 1. Adding a new IoT Hub
  3. Make sure you have the following parameters set using the suggested values or use the below screenshot as a guide:

    On the Basics tab enter the following parameters.

    On the Size and Scale tab enter the following parameters.

    • Pricing and scale tier - Any desired option. For this example we will use the Free tier option.

    Free tier allows to send up to 8000 messages, which is sufficient for testing purposes
    Basics
    Figure 2. Basic configurational tab
    Scale and size
    Figure 3. Scale and Size configurational tab
  4. When finished, select the   Review + create   tab and then click   Create   and wait until the resource is deployed

In case of encountering any troubles, this official Microsoft Quickstart can be used as an alternative example of how to create a new IoT Hub

Creating a device with SAS Token Authentication

  1. Go to the Azure Portal and select the IoT Hub resource, which was created in Preparing Azure Infrastructure. You can find it in   All Resources  

  2. Find the connection string to configure under the iothubowner Policy ( IoT Hub  Shared access policies  iothubowner  Connection string-primary key ). A connection string should be similar to this:

    HostName=inmationMqttTest.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=cHJpbWFyeSBvciBzZWNvbmRhcnkga2V5Ow==

    Make a note of it for later, we will need this connection string to generate the SAS Token.

    Connection string
    Figure 4. Obtaining Connection String for IoT Hub
  3. Open Visual Studio CODE and go to Extensions tab to install the Azure IoT Tools extension

    Azure IoT Tools
    Figure 5. Azure IoT Tools: a Visual Studio CODE extension
  4. To connect Visual Studio Code to the IoT Hub, open the Explorer tab in Visual Studio CODE and open AZURE IOT HUB. You should see something like this:

    Set Connection string
    Figure 6. Setting Connection String to access IoT Hub through Visual Studio CODE extension

    Click on → Set IoT Hub Connection String and insert the Connection String obtained on Azure portal in the previous step into the popup input box.

  5. Now go back to the Azure Portal and select the IoT Hub resource (The one from Preparing Azure Infrastructure).

  6. Click on IoT Devices to create a new device ( IoT Hub  IoT devices  + New ).

    New device
    Figure 7. Creating a new IoT Device
  7. Enter a Device ID name and keep a note of it, we will need it later. In this example indevice is used a a name. Set Authentication type to Symmetric key and leave everything else as it is. Click   Create   button.

    Create a device
    Figure 8. Configuring IoT Device
  8. Return to Visual Studio CODE. Refresh the Device section and your newly created device should appear in the list.

    Right-click on it and select Generate SAS Token for Device option.

    Generate SAS Token
    Figure 9. Generating SAS Token with Visual Studio CODE extension

    Insert an expiration time in hours in the input box, for example 24 hours. Generated SAS Token will be automatically placed in the clipboard as well as written to the output console. Make a note of it since it will be needed later for Cloud Sink configuration (SAS Token Authentication). An example of SAS Token is presented below:

    SharedAccessSignaturesr=inmationMqttTest.azure-devices.net%2Fdevices%2Findevice&sig=X%2B3r3iqIOu%2B9jPkmcy7pJ496RiPL8scvplo9A%2F18g2k%3D&se=1572964996
    When SAS Token exipers, it will no longer be accepted by Azure MQTT Broker and a new one should be generated. It is possible to specify a longer period of time when generating a token, but it is higly recomended to use X.509 Authentification instead.
  9. Now we need to obtain a DigiCert Baltimore Root certificate.

    First of all, download certs.c from the Azure IoT SDK for C.

    Simple copy-pasting may not work as some extra line-breaker characters might appear and corrupt the file. It is recommended to use browser Save as…​ command or use any download tool which can accept URL as a file source.

    This file contains hardcoded certificate data used for application development. To create a valid certificate, we need to extract certificate data and create a certificate file in a correct way.

    Since the instruction given by Microsoft on how to obtain a DigiCert Baltimore Root certificate is very convoluted, there is a help script available as a part of esi-cloud-interface-initializer.lua library, which is available as a part of inmation. In order to get the necessary certificate:

    1. Open a new console in DataStudio ( Ctrl+Shift+U )

    2. Run the following lua command:

      local path_to_c_file = "C:\\Jumpstart\\certs.c" -- path to certs.c file
      return require("esi-cloud-interface-initializer"):CERT(path_to_c_file)
    3. An output like File [C:\Jumpstart\certs.cer] created indicates successful certificacte generation.

    Alternitevely this file can be created manually. Using the .c file downloaded earlier, you need to remove all C-related code form it. Also make sure you have -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- lines included, removed the " marks at the beginning and end of every line, and removed the \r\n characters at the end of every line. When the content is modified properly, file extension should be changed from .c to .cer.

Cloud Sink configuration (SAS Token Authentication)

  1. In DataStudio create a new Cloud Sink object by selecting a Connector and right-clicking Admin  New  History  Cloud Sink from the context menu. Give the object a name a click Create to create it in the I/O Model.

  2. Select the Cloud Sink object and in the Object Properties panel, open the Configuration and select MQTT Publisher Interface.

  3. Under MQTT Publisher Parameters fill the Topic property:

    Table 1. MQTT Publisher Parameters
    Parameter Value Reference Example

    Topic

    devices/ + [ Device ID ] + /messages/events/

    S3:S7

    devices/indevice/messages/events/

  4. Under Connection Parameters, enter the following values for the parameters:

    Table 2. Connection Parameters
    Parameter Value Reference Example

    Broker Address

    [ Hub name ] + .azure-devices.net

    S2:S3

    inmationMqttTest.azure-devices.net

    Broker Port

    8883

    8883

    Client ID

    [ Device ID ]

    S3:S7

    indevice

    Cloud Sink Topic configuration
    Figure 10. Cloud Sink MQTT configuration
  5. Under the Credentials property groups, configure the parameters as instructed below:

    Table 3. Connection Parameters
    Parameter Value Reference Example

    Username

    [ Hub name ] + .azure-devices.net +

    [ Device ID ] + /?api-version=2018-06-30

    S2:S3 S3:S7

    inmationMqttTest.azure-devices.net

    /indevice/?api-version=2018-06-30

    Password

    [ Generated SAS Token ]

    S3:S8

    SharedAccessSignature

    sr=inmationMqttTest.azure-devices.net%2Fdevices%2Findevice&

    sig=X%2B3r3iqIOu%2B9jPkmcy7pJ496RiPL8scvplo9A%2F18g2k%3D&

    se=1572964996

    Cloud Sink Credentials and SSL parameters
    Figure 11. Cloud Sink Credentials and SSL configuration
  6. Under SSL/TLS support import Certificate Authority file

    Click Import File on property Certificate Authority and select certs.cer file obtained in this step

    Cloud Sink Certificate Authority property
    Figure 12. Certificate Authority property
    Selecting Certificate Authority file
    Figure 13. Selecting Certificate Authority file

    If everything is done correctly, it should be possible to view certificate details by clicking View

    Certificate details
    Figure 14. Certificate details
  7. Click Apply in the Object Properties to save configuration.

Creating a device with X.509 CA Certificate Authentication

In this tutorial we are going to use self-generated Certificate Authority (CA) certificate. For production environments it is recommended to purchase X.509 certificates from a root certificate authority (CA).
  1. Go to the Azure Portal and select the IoT Hub resource, which was created in Preparing Azure Infrastructure. You can find it in   All Resources  

  2. Click on IoT Devices to create a new device ( IoT Hub  IoT devices  + New ).

    New device
    Figure 15. Creating a new IoT Device
  3. Enter a Device ID name and keep a note of it, we will need it later. In this example indevice is used as a name. Set Authentication type to X.509 CA Signed and leave everything else as it is. Click   Create   button.

    Create a device
    Figure 16. Configuring IoT Device
  4. Download and extract from the archive OpenSSL biniaries. For this example OpenSSL Binaries Win-64 1.1.1d were used.

  5. First we need to generate a new CA (Certificate Authority) root certificate. To do that follow the instructions:

    1. Open Command Prompt terminal (Windows  Start  Windows System  Command Prompt ) and change directory to the folder where openssl.exe file is located

      cd "C:\\openssl-1.1.1d-win64"
    2. Use the following command to generate a new CA root certificate:

      openssl req -x509 -newkey rsa:2048 -keyout rootKey.pem -nodes -out rootCA.pem

      You will be promted to provide some certificate data (such as Country, City, Organisation Name, etc.) Here is the example of how those fields are filled

      Country Name (2 letter code) [AU]:DE
      State or Province Name (full name) [Some-State]:NRW
      Locality Name (eg, city) []:Cologne
      Organization Name (eg, company) [Internet Widgits Pty Ltd]:inmation
      Organizational Unit Name (eg, section) []:
      Common Name (e.g. server FQDN or YOUR name) []:TEST ROOT CA
      Email Address []:

      As an otput you should obtain two files rootCA.pem (CA root certificate) and rootKey.pem (CA root private key).

      Do not close Command Prompt terminal as we will need it later
  6. The created CA certificate needs to be uploded to Azure and then verified. This verification procedure is knowns as Proof of Possession.

    1. Login to Azure Portal. Select your IoT Hub resource

    2. Upload generated rootCA.pem certificate to Azure ( Iot Hub  Certificates  + Add ).

      Upload Root certificate
      Figure 17. Uploading Root Certificate to Azure
    3. The certificate should now appear in the certificate list. Click on the newly added certificate to open its properties, next click Generate Verification Code (see Figure 15: Button 1) and copy generated data Verification Code (see Figure 15: Button 2)

      Verification
      Figure 18. Generating verification code and uploading verification file
    4. Go back to the Command Prompt terminal used in step 5

    5. Create a new verification key by using the following command:

      openssl genrsa -out verification.key 2048
    6. Next, we need to create a Certificate Signing Request by executing the following command

      openssl req -new -key verification.key -out verification.csr

      When creating the Certificate Signing Request, we again are going to be requested to provide some certificate data. But in this case we need to specify the verification code obtained in step 6.c as the "Common Name" certificate field. Here is the example of how this should look like:

      Country Name (2 letter code) [AU]:DE
      State or Province Name (full name) [Some-State]:NRW
      Locality Name (eg, city) []:Cologne
      Organization Name (eg, company) [Internet Widgits Pty Ltd]:inmation
      Organizational Unit Name (eg, section) []:
      Common Name (e.g. server FQDN or YOUR name) []:7A69A4702DA903A41C3A5BC5575A8E3F49BEC5E5BA2D4CE1
      Email Address []:
      
      Please enter the following 'extra' attributes
      to be sent with your certificate request
      A challenge password []:
      An optional company name []:
    7. Now creating the Proof of Possession certificate with the following command:

      openssl x509 -req -in verification.csr -CA rootCA.pem -CAkey rootKey.pem -CAcreateserial -out verificationCert.pem -days 1024 -sha256

      As an output you should obtain a new file verificationCert.pem

      Do not close Command Propmt terminal as we will need it later
    8. Next import the generated verificationCert.pem file (see Figure 15: Button 3) and click   Verify   button

      Verified
      Figure 19. Root Certificate has been successfully verified
  7. Next task is generating the Device certificate.

    1. Use the same Command Prompt terminal we’ve been using through this tutorial (the last time it was used on step 6.g) to create a new Key and Certificate Signing Request

      openssl req -nodes -newkey rsa:2048 -days 365 -keyout device.key -out device.csr

      Again, we need to follow the procedure of providing certificate data. In this case we need to specify the Device ID set in step 3 as the "Common Name" certificate field. Here is the example of how this should look like (used name is indevice):

      Country Name (2 letter code) [AU]:DE
      State or Province Name (full name) [Some-State]:NRW
      Locality Name (eg, city) []:Cologne
      Organization Name (eg, company) [Internet Widgits Pty Ltd]:inmation
      Organizational Unit Name (eg, section) []:
      Common Name (e.g. server FQDN or YOUR name) []:indevice
      Email Address []:
      
      Please enter the following 'extra' attributes
      to be sent with your certificate request
      A challenge password []:
      An optional company name []:

      Also pay attention to the device.key, the Device Key file, which will also be used for Cloud Sink configuration (X.509 CA Authentication) later.

    2. And finally we can sign Device certificate with CA root certificate we created in step 5.b by running this command:

      openssl x509 -req -days 365 -in device.csr -CA rootCA.pem -CAkey rootKey.pem -CAcreateserial -out device.crt

      As an output we have device.crt certificate, which is going to be used for Cloud Sink configuration (X.509 CA Authentication).

  8. For Cloud Sink configuration we will also need DigiCert Baltimore Root certificate (certs.cer), the same that was used for SAS Token Authenticated devices. This step will explain how to get it.

Cloud Sink configuration (X.509 CA Authentication)

  1. In DataStudio create a new Cloud Sink object by selecting a Connector and right-clicking Admin  New  History  Cloud Sink from the context menu. Give the object a name a click Create to create it in the I/O Model.

  2. Select the Cloud Sink object and in the Object Properties panel, open the Configuration and select MQTT Publisher Interface.

  3. Configure MQTT Publisher Parameters and Connection Parameters:

    Table 4. MQTT Publisher Parameters and Connection Parameters
    Parameter Value Reference Example

    Topic

    devices/ + [ Device ID ] + /messages/events/

    S5:S3

    devices/indevice/messages/events/

    Broker Address

    [ Hub name ] + .azure-devices.net

    S2:S3

    inmationMqttTest.azure-devices.net

    Broker Port

    8883

    8883

    Client ID

    [ Device ID ]

    S5:S3

    indevice

    Cloud Sink Topic configuration
    Figure 20. Cloud Sink MQTT configuration

    Configure Credentials and SSL parameters:

    Parameter Value Reference Example

    Username

    [ Hub name ] + .azure-devices.net +

    [ Device ID ] + /?api-version=2018-06-30

    S2:S3 S5:S3

    inmationMqttTest.azure-devices.net

    /indevice/?api-version=2018-06-30

    For x.509 Authentication algorithm password field stays clear
  4. Under SSL/TLS support import necessary certificates

    Following the same procedure of importing a certificate as shown in this step, three certificates have to be imported to the corresponding properties:

    • Certificate Authority - DigiCert Baltimore Root certificate (certs.cer) (from step 8)

    • Client Certificate - Device certificate (device.crt) (from step 7.b)

    • Client Key - Device Key file (device.key) (from step 7.a)

    Cloud Sink Credentials and SSL parameters
    Figure 21. Cloud Sink Credentials and SSL configuration
  5. Click Apply in the Object Properties to save configuration.

Transferring data with Cloud Sink

  1. To check that the interface is working, double-click on Cloud Sink object’s Faceplate and enter a string in the Write Value dialog.

    Writing to Cloud Sink
    Figure 22. Writing some value to Cloud Sink Item Value
  2. Now check the Azure IoT Hub metric to see that the message has been sent

    Metric Result
    Figure 23. Azure IoT Hub metric with the number of incoming messages
  3. Alternatively, Azure IoT Hub extension for Visual Studio Code can be used to monitor device messages (the one, which was used for SAS Token generation). To start monitoring a device open AZURE IOT HUB in the Explorer tab, right-click on the necessary device and then select Start Monitoring Build-in Event Endpoint

    Message Monitoring
    Figure 24. Enabling message monitoring
    Explorer Result
    Figure 25. An incoming message reported by Azure IoT Hub extension
A single device cannot be connected multiple times to Azure IoT Hub. If a new device with the same Device ID is connected from somewere else, all previous connections will be abandoned.

After deleting a Cloud Sink object, some certificates might be left in the corresponding certificate stores. In order to remove old certificates run the following command in a console:

return require("esi-cloud-interface-initializer"):CLEAN()

Executing this command will not affect any certificates, which are currently being used by other Cloud Sink objects