Azure Machine Learning Studio Using Python

Azure Machine Learning Studio supports Python and R script execution, which enables the user to read and write the data from the system:inmation, by creating inmation http Python client for example. Large or reusable scripts can be stored in the zip files. Zip files can be imported and saved as datasets in the Azure ML Studio.

Below, an example script of inmation http Python client creation and a few basic functions for accessing data stored in the DataStudio. Script can be saved as zip file and uploaded from the local computer as data set to Azure ML Studio.

import json
import requests
import datetime
import pandas as pd

class Client:
    baseURL = "http://localhost:8002"
    options = {
        "auth": {
            "username": '',
            "password": '',
            "authority": 'inmation',
            "grant_type": "password"
        }
    }

    def __init__(self, baseURL, options):
        self.baseURL = baseURL
        self.options = options

    def reqHeaders(self):
        headers = {
            "Content-Type": 'application/json'
        }
        options = self.options or {}
        auth = options["auth"] or {}
        headers["username"] = auth["username"]
        headers["password"] = auth["password"]
        headers["authority"] = auth["authority"]
        headers["grant_type"] = auth["grant_type"]
        return headers

    def createHistoryRequestItem(self, path, aggregate = None):
        agg = aggregate or 'AGG_TYPE_INTERPOLATIVE'
        return {
            "p": path,
            "aggregate": agg
        }

    def createItemValue(self, p, v, q, t):
        return {
            "p": p,
            "v": v,
            "q": q,
            "t": t
        }

    def readHistoricalData(self, items, start_time, end_time, intervals_no):
        reqBody = {
            "start_time": start_time,
            "end_time": end_time,
            "intervals_no": intervals_no,
            "items": items
        }
        headers = self.reqHeaders()
        url = "{}/api/v2/readhistoricaldata".format(self.baseURL)
        print("inmation Web API URL: '{}'".format(url))
        print("inmation Web API headers: '{}'".format(headers))
        print("inmation Web API body: '{}'".format(json.dumps(reqBody)))
        r = requests.post(url, data = json.dumps(reqBody), headers = headers)
        return r

    def write(self, itemvalues):
        reqBody = {
            "items": itemvalues
        }
        headers = self.reqHeaders()
        url = "{}/api/v2/write".format(self.baseURL)
        print("inmation Web API URL: '{}'".format(url))
        print("inmation Web API headers: '{}'".format(headers))
        print("inmation Web API body: '{}'".format(json.dumps(reqBody)))
        r = requests.post(url, data = json.dumps(reqBody), headers = headers)
        return r

Functions from the Zip file

Functions from the Zip file can be invoked by adding Execute Python Script tile. Example code for invoking the ReadHistoricalData endpoint:

import pandas as pd
from inmationhttpclient import Client

baseURL = "http://localhost:8002"
options = {
    "auth": {
        "username": '%USERNAME%',
        "password": '%PASSWORD%',
        "authority": 'inmation',
        "grant_type": "password"
    }
}

start_time = "2018-01-01T00:00:00.000Z"
end_time = "2018-02-01T00:00:00.000Z"
intervals_no = 31
items = [
    {
        "p": "%PATH%",
        "aggregate": "AGG_TYPE_INTERPOLATIVE"
    },
]

client = Client(baseURL, options)

def azureml_main(dataframe1 = None, dataframe2 = None):
    r = client.readHistoricalData(items, start_time, end_time, intervals_no)

    print("JSON")
    jsonRes = r.json()
    print(type(jsonRes))
    dataFrameData = {}
    if isinstance(jsonRes, dict):
        print("r.json is an dictionary!!")
        data = jsonRes['data']
        if isinstance(data, dict):
            print("data is an dictionary!!")
            items = data['items']
            if isinstance(items, list):
                print("items is a list!!")
                for item in items:
                    objPath = item['p']
                    itempath = objPath.split('/')
                    objName = itempath[-1]
                    intervals = item['intervals']
                    v = []
                    q = []
                    t = []
                    print(type(v))
                    dataFrameData[objName + '_v'] = v
                    dataFrameData[objName + '_q'] = q
                    dataFrameData[objName + '_t'] = t
                    print(type(v))
                    if isinstance(intervals, list):
                        print("intervals is a list!!")
                        for interval in intervals:
                            v.append(interval['V'])
                            q.append(interval['Q'])
                            t.append(interval['T'])

    print(v, q, t)
    dataframe = pd.DataFrame(data = dataFrameData)
    return dataframe

Arguments to be filled in: item paths, start and end time, intervals and aggregation. The zip file, which stores the inmation http Python client creation script should be connected to the Python script tile. After these files have been connected click run on the Python script tile.

Access input data

After successful execution green tick mark will be displayed at the top right corner of the screen.

To view the imported data, click the right mouse button, choose Result Data set, Visualize option. Various statistics about each column are displayed. Data can be modified and filtered, missing values deleted or replaced.

Store model data to system:inmation

After incoming data has been shaped and machine learning module created, output can be stored to the DataStudio. Deploy another script execution tile in ML studio, connect the tile with data output and the zip fil. Create a new item in the I/O model, copy the full property path, fill it in the example script item path place holder and click on run.

After successful run, values are set to the item in the DataStudio.

import pandas as pd
from inmationhttpclient import Client
import json

baseURL = "http://localhost:8002"
options = {
    "auth": {
        "username": '%USERNAME%',
        "password": '%PASSWORD%',
        "authority": 'inmation',
        "grant_type": "password"
    }
}

client = Client(baseURL, options)
p = "%PATH%"

def azureml_main(dataframe1 = None, dataframe2 = None):
    colnames = dataframe1.columns
    scores = dataframe1[colnames[6]]
    timestamps = dataframe1[colnames[0]]
    itemValues = []
    for index, timestamp in timestamps.iteritems():
        v = int(scores[index])
        t = int(timestamp)
        itemValues.append(
            {
                "p": p,
                "v": v,
                "t": t
            }
        )

    print(itemValues)
    r = client.write(itemValues)
    print(r.json())
    return dataframe1

What can your company create using Azure ML Studio and system:inmation?

We at inmation hope that the data flow easiness between system:inmation and the Azure ML Studio enables our customers to freely work with machine learning. Create your experiments and do not forget to share your successes with us.