The TCP Datasource Object

The Datasource object represents external data sources in the I/O model of the system and can be configured to be the interface to OPC servers or any other data producing source. When the Datasource is configured as a TCP stream it provides an interface to any port that streams data. All that is required is a hostname and port number and then a Lua script to handle the incoming data. The TCP datasource acts as the client connecting to a TCP server and you can connect to any server accessible in your data source network. It is important to know the format of the data being streamed from the TCP source as this determines how the received data will be parsed. The Lua script that is embedded in the datasource will be scripted to parse the data. For this example we will create a simple TCP server object to stream data and then connect to it using the TCP datasource.

TCP Server Simulation

To set up the TCP stream Datasource we first need a TCP server to connect to. For the purposes of this tutorial we will set up our own simulation TCP server using an ActionItem. We will configure this server to stream simple messages that can be received by the Datasource acting as a client. To begin we will first need to create a local Connector under the Core. For instructions on how to do this, please visit the Infrastructure Hands-on page on How to Add a Connector. Once the Connector is visible in the I/O Model tree, select it the right-click and select Admin → New → Data Processing → Action Item from the context menu.

Create Action Item for simulation TCP Server
Figure 1. Create Action Item for simulation TCP Server

In the Create Object wizard give the ActionItem a name and select the Dedicated Thread Execution box.

Create Object Wizard
Figure 2. Create Object Wizard

Click on "…" for the Lua script body and enter the following script in the script Editor display

local socket = require('socket')

local HOST = "127.0.0.1"
local PORT = 8020

-- create a TCP socket and bind it to the host
local server = assert(socket.bind(HOST, PORT))

local function log(msg)
	if syslib then
		syslib.setvalue(syslib.getselfpath(), msg)
	else
		print(msg)
	end
end

log(string.format("Ready! Listening on %s:%s", HOST, PORT))

local clients = {}

while true do
	if syslib then
		-- object exists and it's enabled
		if (nil == syslib.getobject(syslib.getselfpath())) or (false == syslib.getself():enabled()) then
			return 0x0
		end
	end

    server:settimeout(1)
    local client = server:accept()

    if client then
		log("Client connected.")
		client:settimeout(1)
        table.insert(clients, client)
    end

    if #clients > 0 then
        for _idx, _client in pairs(clients) do
            local full, err, part = _client:receive('*a')
            local buff = full or part

            if err and err == 'closed' then
                table.remove(clients, _idx)
                log("Client disconnected.")
            elseif err ~= 'timeout' then
                log('Error: ' .. err)
            end

            if buff and #buff > 0 then
                log(buff)
            end

			_client:send('Message #1')
            socket.sleep(2)
            _client:send('Message #2 after 2 seconds.')
        end
    end
end

Click OK to add the script and then click Create in the Create Object wizard. The Simulation TCP server will be created in the I/O Model and should return a message indicating that the server is running and listening for connections.

Simulation TCP Server in I/O Model
Figure 3. Simulation TCP Server in I/O Model

If we take a moment to examine some of the details of the Lua script, you can see that the object uses the supported Lua socket library to create a TCP socket and bind it to the localhost (127.0.0.1) on port 8020. The server then listens out for clients on this port and accepts connections. When a client connects it will display the message "Client connected" in the faceplate of the object. One a client is connected it will send two messages, Message #1 and then Message #2 after two seconds.

This simple Simulation TCP server can also be run outside of an inmation environment in a console application. In these cases the messages are returned to the console. NOTE: The server will remain running unless the item is disabled or deleted.

Creating and Configuring the TCP Stream Datasource

Once the Simulation TCP Server is created and running, we can now create a TCP Stream datasource to connect and process the streamed data. We will create the Datasource under our Connector by selecting the Connector in the I/O model, right-clicking and selecting Admin > New > External Interfaces > Datasource from the context menu.

Create New Datasource
Figure 4. Create New Datasource

In the Create Object wizard, first give the Datasource a name then click Next >.

Create Datasource Wizard
Figure 5. Create Datasource Wizard

In the Server Type section of the wizard, select Generic TCP stream from the drop-down menu and fill in the hostname and port for the server. For this you can enter localhost and port 8020, as was configured for the simulation server.

Server Type Settings
Figure 6. Server Type Settings

Click on the "…" next to the Lua script body and enter the following script in the Lua script Editor.

local item_name = '_Debug'
local debug_path = string.format("%s/%s", syslib.getselfpath(), item_name)

local items = {
	{
		path = debug_path,
		class = syslib.model.classes.Variable,
		ObjectName = item_name,
		["ArchiveOptions.StorageStrategy"] = 1
	}
}
syslib.mass(items, syslib.model.flags.MassBatchFlags.FAIL_IMMEDIATELY)

local res = {
    connect = function() syslib.setvalue(debug_path, 'Connected') end,
	received = function(data) syslib.setvalue(debug_path, 'Recv: '.. data) end,
	sent = function() syslib.setvalue(debug_path, 'On sent') end,
	error = function(err) syslib.setvalue(debug_path, 'Error: '.. err) end,
}

return res

Click OK to save the script and then click Create in the Create object wizard. The Datasource will be created in the I/O model and should connect to the Simulation TCP Server we created in the previous step. As part of the Lua script, a Variable object named Debug was created under the item (using the syslib.mass function). The received data from the TCP server is written to this item and historized.

TCP Stream Datasource and Debug Object in I/O Model
Figure 7. TCP Stream Datasource and Debug Object in I/O Model

The second part of the script handles the incoming data from the stream. For the correct functioning of the TCP Datasource object the script should return a table (in the above example 'res') that contains some predefined fields.

These fields access callback functions which are invoked in certain circumstances. The main callback functions are:

  • connect - invoked when a connection is established

  • sent - invoked when data has been sent

  • received - invoked when data has been received

  • error - invoked when something happens to cause the connection to be terminated

In the example Lua script earlier, these fields are assigned to functions that output to the Debug item under the Datasource. Add the Debug item to a History Grid display to see a typical series of messages after connection to the TCP server

History Grid Display showing initial connection and received data
Figure 8. History Grid Display showing initial connection and received data

As well as the callback functions, other parameters can be defined in the table returned by the Lua script.

  • host - the hostname for the server, if set here, this will override the Remote hostname property of the Datasource object

  • port - the port for the server, if set here, this will override the Port property of the Datasource object

  • timeout - the communication timeout in milliseconds, 10000 by default

  • reconnect - the reconnect period in milliseconds, 10000 by default

In this example, the Server was configured to send alternating messages every two seconds. This is a simple example and often the data from a TCP source in production could involve more complex data structures including XML or JSON files. However, when you have prior knowledge of the structure of the data that comes from the TCP server, the Lua script can be customized to parse the received data. The Using Lua Scripting Jump Start contains an example of parsing xml data (from a http source) that can be used as a basis for handling XML data from a TCP server. The system also supports a Lua library for parsing JSON data.