Skip navigation

It appears I have not anticipated the issues that I faced in this project challenge. Unfortunately I had to devote lot of time in researching Azure documentation to fulfill my goal of providing a mobile app for both Android and iOS using Xamarin Forms technology. But the final result is that I could not meet my goals. Even after compromising to provide a set of Console apps, I couldn't meet all the goals. The issues I faced are as follows:

 

  1. Not able to access Azure services through Xamarin Forms. Azure API is not yet servicing Xamarin Forms.
  2. There are many Xamarin examples available but they have been done using Xamarin Native programming technique. That is the Azure APIs are available for native Android platform. So I will have to do the GUI in Native Android but using Xamarin technology. I am not conversant with Android native GUI techniques. So I couldn't dare to do it. It would take much more time. May be I will pursue that line hereafter as the pressure of challenge will not be there any more.
  3. Unfortunately the Azure documents were changing as new services were added. Few things that I have done some 6 months ago, I don't find those guides any more. I had problems in reading Azure Storage Tables. Sending data to storage etc.
  4. Since I was using the Free Azure account, suddenly I found that I had only $10 credit. This happened as I was trying the Azure IoT Suite. The built-in project consumed so many resources that all my $150 credit was gone in just 3 days. It was bit late by the time I realized it.
  5. In between the release of new iOS, new Windows and associated updates to Xamarin all contributed to delays as these updates were not completely smooth. I am still having issues with iOS simulators.
  6. Also struggle with RTC (Reat Time Clock) in STM module after the daytime saving changes. It was not able to reach to the RTC site. Took some time to debug and change the site name.
  7. Mean time I had few health issues and had heavy work load at office last week.

 

I too had high hopes on this project scope but unfortunately I couldn't meet my goals. But I will continue to work on this idea and hope I will be able to make it work finally soon.

 

I wish good luck to those who completed their project.

Storing the device sensor data that is sent to the IoT Hub can be stored either into a Blob or into a Storage Table. Saving in a Table is more easy to comprehend. The process is as follows:

 

  1. Create an Azure Function App named esm-device-msgs2table-appattached to an Azure Storage account named esmiotstorage.
  2. Then created an Event Hub Namespace named esm-event-hub-ns.
  3. In this namespace created an Event Hub named esm-event-hub.
  4. Then created Azure Function code as follows:
    1. Go to the function app created earlier.
    2. Click on Add New Function.
    3. Selected Custom Function link (at the bottom of the page)
    4. Then select the tile named Event Hub Trigger - C#.
    5. Set the name of the function as StoreDeviceMsgs2Table.
    6. Set the other parameters and click Create.

 

This function is supposed to pick the all messages and store in a table. But it was not doing that. This also need to be debugged. If this picks up all the messages and stores in the table, then this table can be read through a .NET app and displayed as historical data.

In this blog I detail the procedure to separate the device messages that contain temperature alert. Once these are separated, they can be channeled to separate cloud storage and also trigger actions to intimate the user about the alert.

 

The messages with alert are sent to a critical queue and read from that. This is done as described in the Azure document

 

  1. First a Service Bus Messaging Namespace is created in Azure Portal as follows:
    1. Go to +New -> Enterprise Integration -> Service Bus. This shows the Create Namespace panel.
    2. .Here created a name space named esm-sb-ns.
  2. Then copied the connection string for this name space to text file.
  3. In this name space then created a Queue named esm-iot-critical-q.
  4. Then in the IoT Hub named esmiothub, created a custom endpoint with following attributes:
    1. Name: esm-critical-q-ep
    2. Type: Service Bus Queue
    3. Service Bus Namespace: esm-sb-ns
    4. Service Bus Queue: esm-iot-critical-q
  5. Then created a Route in the same IoT Hub with following attributes:
    1. Name: esm-iot-critical-route.
    2. Data Source: Device Messages
    3. Endpoint: esm-critical-q-ep
    4. Query String: TemperatureAlert="true"
  6. Then created a .Net Console app named vc_read_alerts. The code for which is shown in the screenshot below.

Code to Read Alerts

 

 

Then both vc_read_alerts and vc_device_live_data are run simultaneously. The STM unit was also powered up. It is expected that all messages with "No Alert" are to be picked up by vc_device_live_data and the messages with "Alert Occurred" will be picked up by the vc_read_alerts program. Unfortunately alert messages were not picked up by vc_read_alerts. This can be seen in the screenshot below. The upper part is the output of vc_device_live_data and the lower part is that of vc_read_alerts. I have created alert by moving the sensors to near a warm light bulb. This needs to be debugged.Console app - Read Alerts

In previous blogs, I have shown the Main Menu of the Mobile App. In this blog I will explain the "Setup New Device" option in that menu. Below is the screenshot of this page on Android Emulator running Marshmallow image.

Mobile app - Setup New Device Page

 

The various components are described here:

  1. Device Model #: The format of this value is two letters followed by two numbers. This will be used in forming the Device Id for cloud access purpose. These indicate as follows,
    1. The 1st letter indicates the Connectivity as follows.
      1. C - Cellular only - When only this connectivity is possible, the WiFi details pane (shown in grey in the above picture) will be disabled as those details need not be input.
      2. W - WiFi only - When this connectivity is possible, the WiFi panel will be enabled.
      3. B - Both Cellular and WiFi connectivity
    2. The 2nd letter indicates the Cloud Service available as follows,
      1. M - Microsoft Azure (Currently this is the only option coded)
      2. A - Amazon Web Services (This service is under examination)
      3. G - Google Web Services (This is not yet considered and meant for future expansion)
    3. The last two digits indicate the version of the hardware board, where the 1st digit is the Major version and the 2nd digit is the Minor version.
  2. Device Serail #: This can take any number of digits. This also will be used in setting the Device Id for cloud access purposes.
  3. WiFi Details pane: This pane will be enabled only if the Model # doesn't start with C.
    1. WiFi SSID: This is the WiFi Network Name.
    2. WiFi Password: Though this is shown in clear text in the screenshot, it will actually be masked with a button to show it momentarily (which is the general practice now for mobile apps).
  4. Unit of Temperature: The button with red border indicates the current selection.
  5. Threshold Temperature: This value is the temperature, in the units selected above, when crossed an alert will be sent to the contact details that will entered next.
  6. Alert Mobile #: This is the mobile phone number to which SMS text message will be sent if cellular connectivity is enabled on the board. However push notifications will anyway be sent for all configurations. The mobile number will be verified by sending a notification to it. Until it is confirmed no alerts will be sent to it.
  7. Alert E-mail Address: When an alert situation occurs an email will be sent to this address if given. This email will be validated by sending a confirmation email. Until it is confirmed no alerts will be sent.
  8. "PROVISION NEW DEVICE" Button: This button will be enabled only after the Model # and Serial # are entered as these two values are needed to formulate the Device Id under which the device is provisioned in the cloud service. When this button is tapped, the designated cloud service will be contacted and the Device Id will be registered with it so that the device can send sensor data to be stored in the cloud. After registering the device with the cloud service, all the details will be sent to the device via Bluetooth so that the device initializes with the parameters and start sending data to the cloud.

 

The two fields above the button are for development purposes and will not appear in the Release version of the app.

C

In this blog post I will describe the program that shows live data after formatting it to general user readable form as shown in the following screenshot.

Console app - Show Live Data

 

Each data received is shown in two rows. The top row shows the Date & Time the sensor data was sampled and the data message number (after the device was powered). The bottom row shows the Temperature, Humidity % and whether the temperature read is above the Threshold Temperature set (which is 86 F). I have created the alert situation by exposing the temperature sensor to warm light bulb. Note the Message numbers 14, 15 & 16. The temperature read is above 86 F and indicates that the Alert Occurred.

 

The console program name is vc_device_live_data and this is to be run in the cmd window or double-clik the vc_device_live_data.exe file.

 

I would like to share few web links that form the basis for my code. I forgot to share them in the earlier blogs. They are:

Console App - Setup New Device - Go to the section on Create a device identity.

Console App - Show Live Data - Go to the section on Receive device-to-cloud messages.

In previous blog post #5 I have shown the Setup New Device page in Mobile App. But unfortunately I found that Azure services are not easily available from Xamarin Forms. There are many Xamarin, the tech that I want to use for cross-platform apps, examples on Azure site but they are all for either .NET console apps or Xamarin Native code. The Xamarin Forms is so appealing that I completely neglected native coding with Xamarin. Hence I chose to work with .NET Console Apps for the present. The following screenshot shows the console app named vs_add_new_device.

Console app - Setup New Device

 

This program is to be run in the cmd window. This asks for the values for the various parameters needed one by one. The format is same as explained earlier.

 

  1. First it will ask for the Device Model #. The 1st letter can be either C, W or B. If it is C then WiFi details will not be asked later on. The 2nd letter can be either M, A or G. Currently only M is allowed which stands for Microsoft Azure services. The last 2 digits indicate the hardware board version.
  2. Next Device Serial # will be asked.
  3. If the Device Model # doesn't start with C, then Wi-Fi Network Name and Wi-Fi Password will be asked. Though the password is shown in clear text, it will not be displayed in the Release mode.
  4. Then Unit of Temperature and Threshold Temperature will be asked. For Unit of Temperature only C or F values are allowed.
  5. Then Mobile Number for Alert Text Messages and E-mail Address for Alert Messages will be asked for. In the Console app, these entries will not be verified by sending confirmation messages. That is skipped for now.

 

Once all the details are entered, the entered values will be displayed and the Azure IoT Hub will be contacted to register the device whose device id is formed by appending the model number and serial number to a fixed set of letters. Once the device id is successfully registered, the Azure IoT Hub will return a device primary key that is to be used by the device to connect to Azure IoT Hub and send sensor data to be stored in cloud.

 

Unfortunately I couldn't lay immediately on code examples to use Bluetooth to send all the required data to the device. Hence I thought of copying the Device Access Key to the mobile app and send it to the device. That is why the access key is displayed here. But that effort also was not successful as I was not able to communicate with the device from phone's Bluetooth. So for the present I hard coded the device access key into the device code.

Device-to-Cloud Communication:

In the last blog I have explained the code that reads the sensors data, process it and sent to Cloud storage, in this case it is Azure Services.

 

First created an Azure IoT Hub using the Azure portal as described here. I am using my Biz Spark subscription for this.

 

Then provisioned a device in the above IoT Hub and got the Device Connection String, which is set in the device software in the azure1_config,h as shown below. Actually this device name / id will follow a format which is yet to be decided. The provisioning will final be done from a Mobile App.

#define AZUREIOTHUBCONNECTIONSTRING "HostName=esm-iot-hub.azure-devices.net;DeviceId=P-NUCLEO-AZURE1-Test;SharedAccessKey=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

 

Currently used Device Explorer utility provided by Microsoft is used to confirm that the data is received by the IoT Hub as shown in the following image.

Device Explorer showing the data as received by the IoT Hub

 

The data received by IoT Hub will be stored in Azure Storage / SQL database which will  be displayed by the Mobile App using Azure App Service. Currently this part is being designed and coded.

 

Mobile App Menu Contents:

In the last blog I have shown the glimpse of the Mobile App Main Menu, which is show below. The functionality is explained here.

Main Menu of the Mobile App (Android version is shown)

 

When the app is started it will be showing the Home page as shown below

Home Page of the Mobile App (Android Version)

 

Tapping the 'Hamburger' icon at the top left corner shows the Main Menu shown earlier. Tapping the [NEW DEVICE] button will allow to add a new device to the Azure IoT Hub. Tapping the [ABOUT] button will show the details about the app. Tapping the [START RECIVING DATA] button will start displaying the data as it is received by the IoT Hub. This will be in the form of a formatted table.

 

The Main Menu options are described below:

  1. Home (Live Data) - Shows the Home Page. This is used to return to the Home Page from any other page.
  2. Alerts - This will display the current and past alerts received. On app start, if there is an active alert the Alerts page will be displayed automatically.
  3. History - This system saves past 7 days data in the cloud storage. This past data will be shown here date-wise. This also includes the data received currently till the time of showing this page.
  4. Manage Cloud - This enables removing an existing device from IoT Hub. If no data is received from a device continuously for 2 days, that device will be 'disabled'. Such disabled device also can be enabled again here. Until it is enabled the data will not be received by the IoT Hub.
  5. Manage Device - This enables the setting of Wi-Fi credentials, IoT Device Connection String, Temperature Unit, Threshold Temperature and other necessary parameters in the device. This connects to the device through the Bluetooth.
  6. App Settings - This allows change of password and other app parameters be set.
  7. Setup New Device - This will walk you through the steps to setup a new device.
  8. About and Help - This shows the details of the app and help topics.

 

Currently researching on the components needed to connect the mobile app to Azure services. Once it is finalized more app pages will be designed and shown here.

This is the 3rd blog post for the "IoT on Wheels Design Challenge"

 

Temperature Alert Logic:

  1. Related Parameters:

    1. Temperature Unit: This can take only two values, 0 for Celsius and 1 for Fahrenheit. Default value will be 1 (Fahrenheit).
    2. Temperature Threshold: If the sensed temperature is above this value, alert mechanism will be activated. Default value will be set to 85 F.
    3. Desired Telemetry Interval:This sets the how frequently the data is transmitted to cloud. This will be set as follows:
      1. 5 seconds: During setup of the device. That is when a new device is being setup to connect to cloud through the mobile app.
      2. 15 minutes: This will be the normal frequency at which the temperature data is transmitted to cloud. This long duration is considered to save data transmission fees as the device will have to connect to cloud through a cellular connection.
      3. 5 seconds for 15 minutes: When an alert occurs the temperature data will be transmitted to cloud every 5 seconds. Also an email and / or text message will be sent to the contact information stored. But this will occur only for 15 minutes or till some action is taken. For now I have no way of knowing that the driver has opened the windows or started fan / air-conditioning either manually or remotely. This logic may change once the system is adapted.
    4. Mobile number and / or Email address of Contact Person: The alert details will be sent to this contact.
    5. Wi-Fi Connection Details: SSID and password to connect through Wi-Fi. This Wi-Fi connection will be used to connect to the mobile app and / or transmit data to cloud when it is available. Currently the password stored will be in clear text but it will be encrypted in future adaptations.
    6. Azure IoT Hub Connection String: This is got from Azure when the device is registered with the IoT Hub. This also will be in clear text for now.

 

Coding for Generating a Temperature Alert:

  • Sensor Board Readings:

    1. The sensors of the Sensor Board are read in the function FillTheModelInstance(void) in the file AzureClient_mqtt_DM_TM.c.
    2. Besides Temperature & Humidity sensors there are also Accelerator and Gyroscope. All values are read in this function only.
    3. The temperature is read in Celsius. So if the Temperature Unit is set to Fahrenheit then the read value is converted to Fahrenheit as shown in the picture below.
    4. If the Temperature is above the TemperatureThreshold then the TemperatureAlert is set to True otherwise it is set to False. The value of this TemperatureAlert will be used by Azure to activate Alert functionality.

Convert Temperature to given Units and set Temperature Alert status

  • Send Sensor Data to Cloud (Azure):

    1. The temperature data is transmitted to Azure by the function SendSNSData_Original(void) in the file AzureClient_mqtt_DM_TM.c.
    2. The following picture shows the data being serialized and transmitted.
    3. The ellipses in the picture contains the code to actually transmit the data and error processing.
    4. A word about naming of the function. The actual function is SendSNSData(void). But to manage the different data transmit intervals, this name is used to manage that time interval. When the time is up for sending the data the function SendSNSData_Original(void) will be called. This interval management code is still being finalized.

 

Code showing the data being serialzed and sent to Azure

 

A Glimpse of the Mobile App being developed (Android version is shown):

Main Menu of the Mobile App (Android version is shown)

This is the 2nd blog post for the "IoT on Wheels Design Challenge".

 

STM32 Setup:

  • The boards in the STM32 kit have been arranged as shown below.

STM32 Kit Boards Arrangement

  • Download & Install USB Drivers & Firmware Update:

    1. First go to www.st.com/stm32nucleo. You will find a link to www.mbed.com. Click on it and select NUCLEO-L476RG figure.
    2. Then go to the bottom of the page and click on the first LINK for ST-LINK/V2 driver in the yellow patch.
    3. Now click on Download the latest ST-LINK/V2 drivers.
    4. Go to Get Software section and click Get Software button.
    5. Here you have to Login / Register to my.st.com. This is a necessary step to download the software. Here download the en.stsw_link009.zip file.
    6. Right click this zip file and select Properties.. There check the Unblock and click OK. This is necessary as this is a downloaded file. This action ensures that all files in the zip are available unrestricted.
    7. Now unzip the .zip and run dpinst_amd64.exe as my PC is a 64-bit system. No errors means it is done good.
    8. Now go to step (2) above and now click on the second LINK to upgrade Nuclero firmware and followed "Windows Upgrade Instructions" as follows:
      1. Downloaded en.stsw_link007.zip file, unblocked in its properties and unzipped it.
      2. Then run ST-LinkUpgrade.exe by double clicking on it in "Windows" directory of the unzipped directory.
      3. Now connect Nucleo board, assembled above, to a USB port.
      4. Click on the Device Connect button.
      5. Now that displays the current version of firmware. For me it showed V2.J25.M14STM32 Debug+Mass storage.
      6. Also shows Upgrading to V2.J29.M18.
      7. Now firmware is successfully upgraded.

 

Connection to Cloud:

I am planning to use cloud to store data and manage alerts. Currently I plan to use Azure IoT Hub for this purpose as I have already worked with it as part of me attending various Microsoft IoT workshops and training sessions. I also got Azure subscription as part of my MSDN subscription. That is the reason why I selected Azure as my cloud component. So I already had created an Azure IoT Hub and added this Nucleo as a device. The details of the cloud setup is as follows:

 

  • Name of the Azure IoT Hub: esm-iot-hub
  • Name of the Device Registered: P-NUCLEO-AZURE1-Test (This is for testing purpose only. The actual name will be set from the mobile app which will follow certain format. The format of the Device ID is yet to be decided.)
  • Primary Key of this device is noted for embedding into the NUCLEO firmware.

 

Transmit Data to Cloud:

  1. Download of Necessary Tools:

    1. Earlier registered to my.st.com. Now got the following software from this site.
      1. Downloaded the STM32 ST-Link Utility from this link. This facilitates upgrade of firmware in STM32 base board.
      2. Downloaded the example code FP-CLD-AZURE1 3.0.0 software. This forms the base code for my project as this has got modules to connect to Wi-Fi and Azure IoT Hub. This takes a lot of burden from my back on connecting the device to Azure.
  2. FP-CLD-AZURE1 3.0.0 software:

    1. The file FP-CLD-AZURE1 3.0 RC1.0.zip is downloaded, unblocked and unzipped to C:\ directory.
    2. This will be unzipped to C:\FP-CLD-AZURE1 3.0 RC1.0 directory.
  3. Setting up System Workbench for STM32:

    1. Installed the workbench software.
    2. Then imported the STM32L4xx-Nucleo project into its workspace from the directory C:\FP-CLD-AZURE1 3.0 RC1.0\Azure\Projects\STM32L476RG-Nucleo\Applications\Azure_Sns_DM\SW4STM32\STM32L476RG-Nucleo.
    3. Important files for this project are:
      1. AzureClient_mqtt_DM_TM.c - Azure sample application.
      2. HWAdvanceFeatues.c - Advance features for sensors board.
      3. platform_STM32Cube.c - Platform dependent functions.
      4. azure1_config.h (in the Includes directory) - for setting Wi-Fi credentials & Azure connection string.
  4. Modifications to Example Code:

    1. Set Wi-Fi Credentials: In azure1_config.h file, set the SSID and Password for the home Wi-Fi network in the following constants. This will be changed to a string variable later to connect to different networks.
      1. AZURE_DEFAULT_SSID
      2. AZURE_DEFAULT_SECKEY
    2. Set Azure Connection String: Set the following constant to the device connection string noted when registered it to Azure IoT Hub. This also will be changed to a string later to accommodate multiple devices.
      1. AZUREIOTHUBCONNECTIONSTRING
    3. Testing the Base Code Functionality:
      1. In SW4STM32 set the Wi-Fi credentials and Azure IoT Hub Connection String as described above. Also made some modifications for testing in AzureClient_mqtt_DM_TM.c file. These changes will be detailed in next blog once they are fixed.
      2. Then built the project from Project -> Build All menu option.
      3. Once the project is successfully built, deployed the code to STM32 base board by running CleanAzure1mbedTLS.bat in the directory C:\FP-CLD-AZURE1 3.0 RC1.0\Azure\Projects\STM32L476RG-Nucleo\Applications\Azure_Sns_DM\SW4STM32\STM32L476RG-Nucleo.
      4. Once the code is deployed to base board, it starts running automatically. So pressed the Blue Button on the base board to stop it running.
      5. Then started the Tera Term, set the port to work with and set the baud rate to 115200. Now the Tera Term is ready to receive from the base board.
      6. Now pressed the Black Button to reset the base board and once it starts running noticed the data is being sent to IoT Hub and getting the acknowledgement as shown in the screenshot below. You may notice the Temperature Threshold crossing also in the screenshot. But that logic is yet to be finalized.

 

Tera Term showing the data received from STM32 Base Board

 

Though all this was done a week ago, I couldn't blog it out due to work pressure and other engagements. In next blog I will show how I am capturing the Temperature alerts and also my plan for the mobile app.

 

If you find this blog is interesting, please do like it.

Now a days we have been hearing about death of children being left inside hot cars. None of the vehicles driven today, old or new, have any way of intimating the drivers that either the car interior is getting hot, left a kid in the car or both. Hence this VeTAS project will enable the drivers to get intimation when things are going wrong.

 

As these are basically 'proof of concept' projects, I am aiming to keep it simple with provisions for extensions and improvements. I will make it as modular as possible so that extending it will be easier in future. The following will describe the design and function of this system.

 

Hardware: This system utilizes all the components of the STM32 kit, namely the base board, the sensor expansion board, the Wi-Fi expansion board and the Bluetooth LE expansion board. Additionally I plan to use a Cellular Shield board to communicate wirelessly with outside world. These will be used as follows:

  1. The base STM32-NUCLEO-L476RG board forms the essential part of the system as this contains the firmware of the system.
  2. The temperature and humidity sensor of the ISK01A2 sensor expansion board will be used for monitoring the vehicle inside environment.
  3. The IDB05A1 BT EL board will be used essentially to initial setup of the system from a mobile app on a smartphone. Both iOS & Android, possibly Windows Mobile also, will be supported.
  4. The IDW04A1 Wi-Fi board will be used to connect to nearby hotspots to transfer data to cloud and / or send email and / or messages (SMS / Push Notifications) to intimate the alerts to the user / driver.
  5. The Cellular Shield board will be used to connect cloud and to send alert messages when the Wi-Fi connection is not available. This will be taken up at the end as an optional unit. The complexity of cellular communications availability is one of the major hurdle in working this out.

 

The mobile app will be used to perform initial setup, display live or history data and to manage cloud component. Currently planning to develop using Xamarin Technology for this cross-platform app. Initially Microsoft Azure will be used for cloud component.

 

In the initial setup the following settings will be set in the devices,

  1. Wi-Fi connection details
  2. Device Id
  3. Temperature Unit (C or F) and Threshold Temperature
  4. Mobile number and email address for alert messages
  5. Register the device to the Azure IoT Hub and get the device connection string for connecting the device to cloud.

 

During setup the data will be transmitted every 5 seconds. Once the setup is completed and ensured the data streaming, the data will be streamed every 15 minutes only to save on data transmission fees and battery. But if an alert occurs, the data will be transmitted every 5 seconds for 15 minutes or till the user takes some action, which ever occurs first.

 

Attached herewith is the Block Diagram of the system.

 

I am in receipt of the sponsored kit and will blog about its setup soon.