Connecting Devices via HTTP using Node-RED to Kaa IoT Platform

In the realm of the Internet of Things (IoT), connecting devices to the cloud is crucial for enabling data collection, processing, and analysis. While various protocols and technologies exist for device-to-cloud communication, HTTP remains a popular choice due to its simplicity, ubiquity, and well-established client and server libraries.

In this comprehensive tutorial, we'll delve into the process of connecting devices via HTTP using Node-RED, a popular open-source visual programming tool, to the Kaa IoT Platform, a powerful and scalable IoT platform.

Challenge

While the Kaa IoT platform offers direct HTTP connectivity for seamless data transfer, the flexibility of this approach comes with certain limitations. For instance, data must be formatted in a simple JSON structure, requiring additional configuration if more complex data structures are involved. This can be particularly challenging when integrating data from other platforms via HTTP, especially when dealing with batch data transfers.

To address these scenarios, Kaa provides a powerful message handler that enables you to pre-process and transform data before it's published to the Kaa platform. This not only streamlines the data flow but also enhances data quality and ensures seamless integration with your existing systems. Furthermore, there are numerous situations where you may need to perform in-depth message handling before sending data to the Kaa IoT platform.

Node-RED comes to the rescue, effortlessly simplifying data processing.

Also read the Tutorial: Connecting Devices to the IoT Platform with Node-RED & MQTT

This brief tutorial demonstrates how Node-RED can be utilized for the initial data transformation before forwarding it to the Kaa IoT Platform.

Example of the message

For instance, imagine a scenario where a pressure sensor communicates periodically, transmitting measurements within a single message over a specific time frame.

Message example:

    {
    "DeviceType": "pressure_sensor",
    "DeviceId": "PS875035",
    "MessageType": "batch_msg",
    "Data": {
        "start": "2023-12-15 08:00:00",
        "interval": 600,
        "values": [
            35,
            37,
            37,
            38,
            40,
            39,
            42,
            43,
            45,
            32
            ]
        }
    }
                    

In this message, we have the following parameters:

• DeviceType: the sensor type;
• DeviceId: sensor ID;
• MessageType: message type. In our case, it is a batch message (a message that contains some amount of data);
• start: this field contains the date-time of the first measured value;
• interval: the interval between the measurements;
• values: contains an array of measured values.

The measured data needs to be organized into a time series to facilitate displaying data through timestamped graphs.

To achieve this, two parameters are necessary:

• The data itself.
• Timestamp: a timestamp indicating which point in time the data relates to.

When sending data to the Kaa IoT platform, there are two timestamp options to consider:

Returning to our current message, in our case, we only have the timestamp when the first measurement occurred and know the interval between subsequent measurements. To properly publish this packet to the Kaa IoT platform, we need to convert this message into a series of messages. Each message within this series will contain the measured value along with the timestamp indicating when the measurement occurred.

Node-RED flow configuration

To implement the logic described above, the most convenient option is to use Node-RED.

Let's go to Node-RED and create our flow.

To accept HTTP POST requests, we need to add a node

During the next step, we need to configure this node.

URL: http://<tenant-id>.nodered.kaaiot.com:1880/data (where <tenant-id> is your tenant ID):

If the request has the header “Content-type” = “application/json”, the message in the debug window will look like this:

We can see that it is an Object.

If the “Content-type” header is absent, the message will look like the text and need to be parsed:

To convert it to an object, we have to use JSON.parse(msg.payload).

We will use MQTT requests to send the data to the Kaa IoT Platform.

Let’s look at our message.

We have the DeviceType and DeviceId. To send to the URL (it can be found here) mqtt://mqtt.cloud.kaaiot.com/kp1/<app-version-name>/dcx/<endpoint-token>/json, we have to know the app-version-name and endpoint-token.

The most effective approach is to create separate applications for devices categorized under different DeviceTypes. Each device type corresponds to its own dedicated application.

Further reading: Kaa Concepts: Applications and application versions.

As usual, each DeviceId possesses a unique value, making it feasible to utilize it as an endpoint token.

Further reading: Kaa Concepts: Endpoints.

First of all, we have to create an Application and an Application version.

Let's proceed with configuring the timestamp settings as mentioned earlier.

Click on 'Base configuration' and then on the 'epts' button. Configure the following settings:

• Select Autoextract option to allow the platform to autoextract the data from the messages;
• Set Path (Path to the timestamp value in the payload. Nested keys can be separated by dot);
• Select Format (Format of the timestamp value);
• Select Fallback strategy.

Note: Different application versions can have different configurations. That is why we have to repeat the configuration for our application version.

Now we can Create the endpoint. Go to Device Management on Kaa Cloud, select our application, click Add device, and set the DeviceId as Endpoint token:

Now we have the application version and endpoint token so our MQTT URL to publish messages to the endpoint is mqtt://mqtt.cloud.kaaiot.com/kp1/cf2p1cmgthh95ci5h4kg-v1/dcx/PS875035/json

We can go back to Node-RED and continue our flow configuration.

Let’s create the function node where we will parse the HTTP body. We will convert the original message into an array of messages each with its own timestamp and value.

    var messagesToSend = [];    // messages to send to the Kaa platform

    if (msg.payload.DeviceType == "pressure_sensor") {

        var firstMeasurementTimestamp = new Date(msg.payload.Data.start).getTime();
        var measurementInterval = msg.payload.Data.interval;

        for (var i = 0; i < msg.payload.Data.values.length; i++) {
            
            var message = {};
            message.payload = {};
            message.payload.timestamp = firstMeasurementTimestamp + i* (measurementInterval * 1000); // timestamp in milliseconds
            message.payload.value = msg.payload.Data.values[i];
            message.token = msg.payload.DeviceId;

            messagesToSend.push(message);
        }
    
    } else {
        node.warn(`Unknown device type ${msg.DeviceType}`);
        return null;
    }

    return [messagesToSend];
                    

In order to publish data to the Kaa IoT Platform, we can utilize nodes available in the Kaa library. For publishing data to the time-series, we require the Kaa Timeseries node and the Kaa MQTT Connector node.

Now, let's proceed to configure the Kaa MQTT Connector node. We have to set the Application, Application Version, and Client ID:

Do not forget to add the HTTP response node to create a response to the original HTTP message.

Alright, as a result, we have the next flow:

Let’s send a message to the Postman (use your tenant ID instead of <tenant-id>).

As the final step of the tutorial, if we navigate to the Device Management section on the Kaa IoT platform and inspect our endpoint, we will be able to view the data.

The source code for the Node-RED flow

[
    {
        "id": "87467f57faa3605d",
        "type": "debug",
        "z": "bea8ae04524ada71",
        "name": "raw_request",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "x": 290,
        "y": 120,
        "wires": []
    },
    {
        "id": "add8c3c39028b4bf",
        "type": "http in",
        "z": "bea8ae04524ada71",
        "name": "",
        "url": "/data",
        "method": "post",
        "upload": false,
        "swaggerDoc": "",
        "x": 100,
        "y": 180,
        "wires": [
            [
                "87467f57faa3605d",
                "96859fa0fd3f2c18",
                "1ee7f320da06fb83"
            ]
        ]
    },
    {
        "id": "96859fa0fd3f2c18",
        "type": "function",
        "z": "bea8ae04524ada71",
        "name": "parse HTTP body",
        "func": "var messagesToSend = [];    // messages to send to the Kaa platform\n\nif (msg.payload.DeviceType == \"pressure_sensor\") {\n\n    var firstMeasurementTimestamp = new Date(msg.payload.Data.start).getTime();\n    var measurementInterval = msg.payload.Data.interval;\n\n    for (var i = 0; i < msg.payload.Data.values.length; i++) {\n        \n        var message = {};\n        message.payload = {};\n        message.payload.timestamp = firstMeasurementTimestamp + measurementInterval * 1000; // timestamp in milliseconds\n        message.payload.value = msg.payload.Data.values[i];\n        message.token = msg.payload.DeviceId;\n\n        messagesToSend.push(message);\n    }\n    \n} else {\n    node.warn(`Unknown device type ${msg.DeviceType}`);\n    return null;\n}\n\nreturn [messagesToSend];",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 310,
        "y": 180,
        "wires": [
            [
                "26a2d5fc0d5db2ac",
                "796b3f7d1b2efe39"
            ]
        ]
    },
    {
        "id": "e285c3056ed51a28",
        "type": "comment",
        "z": "bea8ae04524ada71",
        "name": "HTTP server",
        "info": "",
        "x": 110,
        "y": 120,
        "wires": []
    },
    {
        "id": "26a2d5fc0d5db2ac",
        "type": "debug",
        "z": "bea8ae04524ada71",
        "name": "messages to KaaIoT",
        "active": false,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "x": 560,
        "y": 120,
        "wires": []
    },
    {
        "id": "1ee7f320da06fb83",
        "type": "http response",
        "z": "bea8ae04524ada71",
        "name": "HTTP server response",
        "statusCode": "200",
        "headers": {},
        "x": 320,
        "y": 240,
        "wires": []
    },
    {
        "id": "796b3f7d1b2efe39",
        "type": "kaa-timeseries",
        "z": "bea8ae04524ada71",
        "x": 540,
        "y": 180,
        "wires": [
            [
                "09081b9b248f9536"
            ]
        ]
    },
    {
        "id": "09081b9b248f9536",
        "type": "kaa-connector",
        "z": "bea8ae04524ada71",
        "connection_type": "MQTT",
        "domain": "cloud.kaaiot.com",
        "application": "cf2p1cmgthh95ci5h4kg",
        "version": "v1",
        "client_id": "mqttClient",
        "x": 760,
        "y": 180,
        "wires": [
            []
        ]
    }
]