Version 7


    M2M (machine to machine) and IOTA (distributed ledger) communication face challenges in delivering information from (a) one source to many servers, (b) listening to abrupt events, and (c) distributing small packets of information in high volumes over unreliable networks. MQTT (Message Queuing Telemetry Transport) is a protocol that aims to meet the demands of IoT communication requirements. It is a very lightweight and fast protocol and takes very few bytes to connect to a server, hence, it can be easily implemented on low-end microcontrollers, consuming very low power for its operation.

     

    The MQTT Protocol Model

    The core elements of the MQTT system are the Clients, Servers (referred to as Brokers), Sessions, Subscription, and Topics. It uses the Publish-and-Subscribe (Pub/Sub) model which decouples the client sending messages (referred to as publisher) from another client who receives the messages (referred to as the subscriber).  The process of subscription and publishing is handled by a Server also referred to as a Broker.

     

    Brokers run topics which collect messages about specific events from clients and forward them to interested clients who are filtered by the topics to which they have subscribed. A typical system consists of one broker and many clients. Each client communicates to a single broker. The client can be subscriber or publisher.

     

    MQTT uses TCP/IP to connect to the broker. Once an MQTT client connects to the broker, it will remain connected even if it is not sending any data.

     

    MQTT clients send a keep-alive message usually at 60-second intervals, which tells the broker that the client is still connected. All clients are required to have a unique client name. This client name is required by the broker to track subscriptions. The publisher publishes to a certain topic and the subscriber subscribes to single or multiple topics. The topic name must be at least one character long and it is a UTF-8 string. Topics have a tree structure and use forward slash as a delimiter. For example: Home/Room/Temperature

     

    MQTT supports 3 QoS (Quality of Service) levels. They are:

     

    • QoS - 0 default no guarantee in the delivery of the message.
    • QoS - 1 message may get duplicates but guaranteed delivery.
    • QOS- 2 confirmed delivery no duplicates of the message.

     

    When a client publishes a message to a broker, it sends the topic, message QoS, and Retain Flag. By default, a retain flag is set to false, i.e. the broker does not retain the message.  If the retain flag is true, the broker will retain the last message published by the client. MQTT supports web sockets to receive data directly into a web browser by using a JavaScript client. The broker places the MQTT packet into a web socket envelope and the client unpacks the MQTT packet from the web sockets envelope.

     

    MQTT Standard Packet Structure

    The table below shows the MQTT standard packet structure:

    The fixed header consists of the control field and the packet length field. The minimum size of the packet length field is 1 byte when the total length of the message is less than 127 bytes.

     

    MQTT Packet = control + length + protocol name + Protocol Level + Connect Flags + Keep alive + Payload

    The Control Field

    The 8-bit control field is divided into two 4-bit fields and contains all of the protocol commands. The first 4-bits are a message type field and the other 4-bits are control flags.

    MQTT Control Field Structure

     

    Packet Type Examples:

     

    Connect=0001=1

    Connack=0010=2

    Disconnect=1110=14

     

    MQTT control Packet type

     

    The table below shows the control packet type and is represented as a 4-bit unsigned value:

     

    Name

    Value

    Direction of flow

    Description

    Reserved

    0

    Forbidden

    Reserved

    CONNECT

    1

    Client to Server

    Client request to connect to Server

    CONNACK

    2

    Server to Client

    Connect acknowledgment

    PUBLISH

    3

    Client to Server or Server to Client

    Publish message

    PUBACK

    4

    Client to Server or Server to Client

    Publish acknowledgment

    PUBREC

    5

    Client to server or server to client

    Publish received

    PUBREL

    6

    Client to server or Server to Client

    Publish release

    PUBCOMP

    7

    Client to Server or Server to Client

    Publish complete

    SUSCRIBE

    8

    Client to server

    Client subscribe request

    SUBACK

    9

    Server to Client

    Subscribe acknowledgment

    UNSUBSCRIBE

    10

    Client to Server

    Unsubscribe request

    UNSUBACK

    11

    Server to Client

    Unsubscribe acknowledgment

    PINGREQ

    12

    Client to Server

    PING request

    PINGRESP

    13

    Server to Client

    PING response

    DISCONNECT

    14

    Client to Server

    Client is disconnecting

    Reserved

    15

    Forbidden

    Reserved

     

    Here, the CONNECT value is in hex. In decimal, it is given as:

     

    0x10=0001000=16 decimal

    Similarly, CONNACK value in decimal is 0x20=0010000=32 decimal.

     

    Flags

    Most of the Flag bits are reserved and intended for future use. The table below shows the flag bits that are currently used. A Publish message uses these flag bits.

     

    Control Packet

    Fixed header flags

    Bit 3

    Bit 2

    Bit 1

    Bit 0

    CONNECT

    Reserved

    0

    0

    0

    0

    CONNACK

    Reserved

    0

    0

    0

    0

    PUBLISH

    Used in MQTT 3.3.3

    DUP

    QoS

    QoS

    RETAIN

    PUBACK

    RESERVED

    0

    0

    0

    0

     

    UP=Duplicate delivery of a PUBLISH Control Packet

    QoS= PUBLISH Quality of Service

    RETAIN= PUBLISH Retain Flag

     

    Variable Length Header

     

    Messages or commands that require additional control information make use of this field. This field is sometimes not present in an MQTT message.

     

    Bit

    7

    6

    5

    4

    3

    2

    1

    0

    Byte 1

    Packet Identifier MSB

    Byte 2

    Packet Identifier LSB

     

    Payload

    The payload is the final part of the packet and is required by some of the control packets, i.e. CONNECT PUBLISH, SUBSCRIBE, SUBACK and UNSUBSCRIBE, and other control packets that do not require payload.

     

    CONNECT Packet

    The Connect packet is the first packet sent from the client to the server after a network connection is established.

     

    The table below shows an example of the CONNECT Packet:

    CONN and RL are the connect packet fixed header and they are given by the table below:

     

    Bit

    7

    6

    5

    4

    3

    2

    1

    0

    Byte 1

    MQTT  Control Packet type (1)

    Reserved

     

    0

    0

    0

    1

    0

    0

    0

    0

    Byte 2

    Remaining length

     

    In the above table, CONN is 10 in hex and the remaining length field (RL) is 12, which is the length of the variable header plus the length of the payload.

     

    PLEN is the protocol length, given as 0004. MQTT is the protocol name which is described in the table below:

     

     

    Description

    7

    6

    5

    4

    3

    2

    1

    0

    Byte 1

    Length MSB (0)

    0

    0

    0

    0

    0

    0

    0

    0

    Byte 2

    Length LSB (4)

    0

    0

    0

    0

    0

    1

    0

    0

    Byte 3

    ‘M’

    0

    1

    0

    0

    1

    1

    0

    1

    Byte 4

    ‘Q’

    0

    1

    0

    1

    0

    0

    0

    1

    Byte 5

    ‘T’

    0

    1

    0

    1

    0

    1

    0

    0

    Byte 6

    ‘T’

    0

    1

    0

    1

    0

    1

    0

    0

     

    LVL is the protocol level and 04 is the default value specified by the table below:

     

     

    Description

    7

    6

    5

    4

    3

    2

    1

    0

    Byte 7

    Level (4)

    0

    0

    0

    0

    0

    1

    0

    0

     

    FL is the connect flag and the value is 02 since we are not giving any username and password, only a clean session. Connect flag bits are shown in the below table.

     

    Connect Flag Bits

    Bit

    7

    6

    5

    4

    3

    2

    1

    0

     

    User Name Flag

    Password Flag

    Will Retain

    Will QoS

    Will Flag

    Clean Session

    Reserved

    Byte 8

    X

    X

    X

    X

    X

    X

    X

    0

     

    KA is the ‘keep alive’ field. It is used by the client to send the keep alive signal to remain connected to the broker even if the client is not sending any data. The keep alive is a time interval measured in seconds. It is the responsibility of the Client to ensure that the interval between the Control Packets does not exceed the keep alive value. The keep alive value here is 3C, which is a 60-second interval.

     

    CIDLEN is the client identifier length. The client identifier identifies the Client to the Server. Each client connecting to the server has a unique Client Id. Here, the Client id is ABCDEF and the length is 06 given by CIDLEN.

     

    PUBLISH Packet

    A publish control packet is sent from a Client to Server or from a server to client. Here, the fixed header for the publish message is given as 30, and the remaining length (RL) field is 10. ELEMENT14 is the topic name and its length in hex is given as 00 09, and hello is the message to be published.

     

    SUBSCRIBE Packet

    The table below shows the subscribe packet:

    The value of the fixed header field for the subscribe packet is 82, which identifies it as a subscriber. The remaining length field is given as 0E. The packet identifier, id PKID, is user specific and the user can give any value; it is 2 bytes. Next is the topic length and topic name, i.e. element14. QS is the quality of service sent to the server; the value of QS given here is 00.

     

    MQTT is used in real-world applications

     

    MQTT is widely used since it is open and royalty free.  It is used in Facebook Messenger to achieve lower power consumption and reduced latency. It is also used in various Smart City applications involving Environment and Traffic Monitoring, Building Automation, asset Tracking and Management, and Medical Services.

     

    Major MQTT brokers available for implementation and testing are Microsoft Azure IoT, Amazon AWS IoT, IBM Bluemix, ThingStudio, ThinkSpeak, among others.