Skip navigation
1 2 Previous Next

Enchanted Objects

25 Posts authored by: amgalbu Top Member

After building the MagicHat, I was wondering: what if I want to wear the hat at a party? I need a bow tie... so this is a quick tutorial to build the bow tie for your MagicHat

 

IMG_20150626_110333.jpg

IMG_20150626_110655.jpg

IMG_20150626_111033.jpg

IMG_20150626_111105.jpg

IMG_20150626_112729.jpg

IMG_20150626_113052.jpg

 

IMPORTANT NOTES

1. it's better to wear the bow tie with a shirt

2. charge MagicHat battery before the party

3. remember to shave!!

4. say thanks to balearicdynamics  for this wonderful idea

 

This week I completed my project for the Enchanted Objects design challenge. It has been funny even if some external activities (namely work and family :-)) didn't allow me to get a more in-depth view of some aspects.

I will now try to assess whether I met the requirements of this challenge

 

Security

The MagicHat does not publish anything on the Internet. Local connections are encrypted through SSL and access to web UI is protected by user authentication

Accuracy: how well does the proposed solution register the information it is intended to assess?

It's difficult to say: the sensor temperature has an accuracy of 0.1 °C, the earth beat detection algorithm has probably tested and works reliably. The blood pressure detection needs to be verified on a larger number of patients (see below)

Convenience: how easy it will be for non-technical personnel to use ?

Very easy: just power it on!

Cost: how much will it cost?  Is it cost effective?

Regarding the hardware and the software, I tried to reduce to the minimum the hardware requirements to run the whole application and I also used standard components that are widely available and very cheap.

The bill of the building materials is very short: plywood, PVC film and screws

However, some optimization could be implemented. In particular the blood pressure measurement method could be changed to make everything happen on the Arduino Yun, thus eliminating the Arduino Mini board. The initial idea was to show the biosignal (which should - at least in my mind- could have been a "wow factor") and to use the biosignal itself to measure both diastolic and systolic pressure. After some experiences, I realized that there are too many factors affecting the amplitude of the biosignal. A better solution could be the approach implemented by balearicdynamics in its meditech. He builds an external board that generates a square wave which frequency matches the number of earth beats per minute. Processing the square wave will be much simpler, freeing enough Arduino Yun's resources to control both servos and mouth's LEDs

Durability: for example, can it withstand everyday life and yet continue to function normally?

Crash tests will are planned after the completion of the demo video :-)

Even if the MagicHat is not sending any sensitive data out on the Internet, it's always important to make communication as secure as possible.

For this reason, I will implement HTTPS communication between the web browser and NodeJS running on Arduino Yun. Since the communication is now encrypted, we can use basic authentication (were password is just Base64 coded, but not encrypted)

To add HTTPS support, we need to:

 

Install OpenSSL

OpenSSL setup can be downloaded from here. Installation is plain vanilla

 

Create a self-signed certificate

To create a self-signed certificate, open a command prompt window and type the following commands

set OPENSSL_CONF=C:\Program Files (x86)\GnuWin32\share\openssl.cnf

This commands set the path for the OpenSSL configuration file

 

openssl req -x509 -newkey rsa:2048 -keyout c:\temp\key.pem -out c:\temp\cert.pem -days 365 -nodes







req: PKCS#10 certificate request and certificate generating utility.

-x509: this option outputs a self signed certificate instead of a certificate request. This is typically used to generate a test certificate or a self signed root CA.

-newkey arg this option creates a new certificate request and a new private key. The argument takes one of several forms. rsa:nbits, where nbits is the number of bits, generates an RSA key nbits in size.

-keyout filenamethis gives the filename to write the newly created private key to.

-out filename This specifies the output filename to write to or standard output by default.

-days nwhen the -x509 option is being used this specifies the number of days to certify the certificate for. The default is 30 days.

-nodesif this option is specified then if a private key is created it will not be encrypted. When this option is not added, NodeJS will ask for the private key's passphrase when the private key is used

 

Create a server that support HTTPS

The following code creates a server with HTTPS support

 

var options = {
  key: fs.readFileSync('key.pem'),
  cert: fs.readFileSync('cert.pem')
};

var server = https.createServer(options, app);
server.listen(3000, function(){
  console.log('We have started our server on port 3000');
});





 

Install NodeJS's basic authentication module

Since a  lot of the middleware was pulled out of the Express core in v4 and put into separate modules, the basic auth module needs to be explicitely installed

 

npm install basic-auth-connect

 

Implement basic authentication in node

 

var basicAuth = require('basic-auth-connect');

// NEW CODE BEGINS HERE

// replace 'user' and 'pass' with your username and password

app.use(basicAuth('user', 'pass'));
//NEWCODE ENDS HERE

app.use(express.static(__dirname));
app.use(express.static('.'));



Client side changes

The only change required on the client side Javascript is to changes the websocket URL from ws://<ipaddress> to wss://<ip address> to use Secure web sockets. if standard web sockets are used, web browsers will raise a security error because you are trying to use an unsafe connection

 


With  all these changes in place, we can now connect to the MagicHat web interface using HTTPS. Since the certificate is self-signed and not by a trusted certification authority, web browser will complain about that and will warn you about potential risks

22 - Certificate warning.png


By clicking on the"Continue to this website" link, you will be asked for username and password

 

22 - Authentication.png

This is the last week of the challenge

The hardware and software are almost completed, so it's time to build the real hat. In a previous post, I started building some parts of the hat.

First of all, I cut out from plywood a circle that will be the brim of the hat. I then fixed the blood pressure measurements device to the plywood using some self-tapping screws

 

IMG_20150612_084330.jpg

Then I covered everthing with a fabric. It's not very fashion, but that's what my mother has in house

 

IMG_20150620_122836.jpg

 

I then attached the brim to the cone that contains all the electronics

 

IMG_20150620_132025.jpg

 

You can clearly see the LEDs that make up the mouth and the small speaker

Here is more detailed view of the electronic components inside the hat

 

Electronics.jpg

 

Just to recap:

  • Arduino Yun - Atmel side takes care of sampling sensors (temperature, FSR, finger present, pulse)
  • Arduino Yun - Linino side: runs NodeJS, implements Text-To-Speech by invoking a webservice, get Harry Potter-related quotes by invoking a web service
  • Arduino Mini Pro: controls servos and mouth's LEDs

 

Then I fixed (with some hot glue) the temperature sensor to the baseball cap (trust me: it doesn't hurt when you wear the hat)

 

IMG_20150620_230818.jpg

 

Using the same fabric, I cut out a sort of one that covers all the electronic components. Here is the final result

 

IMG_20150620_200822.jpg

Thanks to the Text-To_Speech engine, I can easily add a nice function to the MagicHat: the capability to say wisdom phrases while it's idling


The implementation will be based on a web service that provides quotes as a JSON document

I found a lot of alternatives on www.mashape.com and I selected the quotes service provided by quotbook.com

The URL to invoke to get a new quote has the following format


https://yusufnb-quotes-v1.p.mashape.com/widget/~{search}.json


where {search} is the topic you are interested to get a quote about. To stay on the challenge topic, I will randomly search for the following topics

  • rowling
  • magician
  • harry potter

 

The quote web service will be queried by theNodeJS application

 

function getQuote(topic) {
  var http = require('http');
  options = { 
    host: 'yusufnb-quotes-v1.p.mashape.com', 
    path: '/widget/~' + topic + '.json',
    port: '1338', 
    headers: {
       'X-Mashape-Key': '<your mashape key>',
       'Accept': 'application/json"

     } }; 
 
 
  callback = function(response) { 
  r str = '';
     response.on('data', function(chunk) { 
      str += chunk; 
    }); 
    response.on('end', function() { 
      console.log(str); 
      var obj = JSON.parse(str);
 
 var mp3url='http://tts-api.com/tts.mp3?q='+encodeURIComponent(obj.quote);
      getMp3(mp3url, filename, function() {
        exec('/usr/bin/madplay -A10 ' + filename);
      });
    }); 
  }
 

  req = http.request(options, callback); 
  req.end();
 };
 


Since the Arduino side is aware whether the MagicHat is idling or not, the quote function will be triggered by Arduino itself by sending a new command


Serial1.println("QUOTE");


This command will be parsed by the serial port's data handler

 

// quote topics
var topics = ['rowling', 'magician', 'harry_potter'];

// generate a random int value between low and high (including limits)
function randomInt (low, high) {
    return Math.floor(Math.random() * (high - low) + low);
}

// serial port data handler

yunPort.on('data', function(data) {

  //console.log('data ' + data);
  
  _data = _data + data;
  var eol = _data.indexOf("\r\n");
  if (eol > 0)
  {
    _data = _data.substring(0, eol);
    if (_data.substring(0,5) == "PLAY ")
    {   
      console.log("Received command " + _data.substring(0,5)+","+_data.substring(5));
      var text = _data.substring(5);
      _data = "";

      text = text.replace(/ /g, '_');
      var filename = '/mnt/sda1/' + text + '.mp3';
      console.log("Checking file " + filename);
      if (fs.existsSync(filename))
      {
        console.log("Playing file " + filename);
        exec('/usr/bin/madplay -A10 ' + filename);
      }
      else
      {
        var mp3url='http://tts-api.com/tts.mp3?q='+encodeURIComponent(text);
        getMp3(mp3url, filename, function() {
          exec('/usr/bin/madplay -A10 ' + filename);
        });
      }
    }
    else if (_data.substring(0,5) == "QUOTE")
    {
      getQuote(topics[randomInt(0,topics.length-1)]);   
    }
    else
    {
      if (_ws)
        _ws.send(_data);
      
      _data = "";
    }  
  }
});

amgalbu

19 - MagicHat first words

Posted by amgalbu Top Member Jun 12, 2015

I completed the mouth and the MagicHat now can speak.

I had to make some changes to the circuit for reading audio output. The new version includes two resistors and a capacitor to shift up voltage level by 2.5V

 

19 - Mouth_bb.png

 

Using an headphone, I cut off one of the speakers and added a connector to be connected to the Arduino board

 

Speaker.jpg

 

The video shows MagicHat saying its first word. "Hello". Unfortunately audio quality is not that good due to my smartphone limitations, but it gives the idea!

In the video, you can also see all the hardware ready to be installed

The last missing piece of hardware to complete the MagicHat is the mouth

The mouth will be a simple VuMeter that continously read the line output from the USB audio card and switch on LEDs accordingly

LEDs will be switched on from the center to the sides to give the impression that the mouth is more open when the sound is louder

 

Here is the circuit schematic

 

18 - Mouth.png

 

and here is the flowchart of the code that switches on and off LEDs

 

18 - The mouth.png

 

NOTE: (threshold1 < threshold2 < threshold3 < threshold4)

I went through and interesting article here

 

Basically, what they state is that once you got the correspondence between biosignal ADC reading and maximum blood pressure, you can determine systolic and diastolic pressure by applying a linear equation on the minimum and maximum ADC reading for each heart beat

 

So I have a very simple (yet not optimal) solution to the problem of determining the systolic and diastolic pressure

The flow chart of the process for measuring blood pressure is as follow

 

17 - Blood pressure measurement.png

Not that I can play an mp3 file, let's find a way to build a simple TTS speech

As we live in a connected world, why not leveraging the power out there on the web to overtake hardware limitations?

 

I found an interesting site here

http://tts-api.com/

 

that provides a text-to-speech web service

 

To convert a text programmatically from NodeJS, the URL to invoke is the following one

 

var mp3url='http://tts-api.com/tts.mp3?q='+encodeURIComponent(text);

 

Given this url, we can use NodeJS's http.get to download the mp3 file. Here is the code

 

function getMp3(mp3url, filename, cb)
{
console.log("Getting url " + mp3url);
http.get(mp3url, function(response) {
   if (response.statusCode > 300 && response.statusCode < 400 && response.headers.location) {
     console.log("Redirect detected");
     // The location for some (most) redirects will only contain the path,  not the hostname;
     // detect this and add the host to the path.
     if (url.parse(response.headers.location).hostname) {
       // Hostname included; make request to response.headers.location
       console.log("Redirecting to " +  response.headers.location);
       getMp3(response.headers.location,filename, cb);
     } else {
       // Hostname not included; get host from requested URL (url.parse()) and prepend to location.
     }
   // Otherwise no redirect; capture the response as normal           
   } else {
     response.setEncoding('binary');
     response.on('data', function (chunk) {
       console.log("Data " + chunk.length);
       _data =_data.concat(chunk);
     });
     
     //the whole response has been recieved, so we just print it out here
     response.on('end', function () {
        console.log("End " + _data.length);
       
        var f = fs.createWriteStream(filename);
        f.write(new Buffer(_data, 'binary'));
        f.end();
       
        cb();
     });
   }
});
};

 

Here are two kind of magic:

  1. Redirection: when the above-mentioned URL is requested, the request is redirected to a URL like this one

http://media.tts-api.com/1e4e888ac66f8dd41e00c5a7ac36a32a9950d271.mp3

NodeJS's http.get does not provided redirection-following capabilities, we have to implement the logic using the following lines of code

if (response.statusCode > 300 && response.statusCode < 400 && response.headers.location) {
console.log("Redirect detected");
// The location for some (most) redirects will only contain the path,  not the hostname;
// detect this and add the host to the path.
if (url.parse(response.headers.location).hostname) {
            // Hostname included; make request to response.headers.location
            console.log("Redirecting to " +  response.headers.location);
            getMp3(response.headers.location, cb);
} else {
            // Hostname not included; get host from requested URL (url.parse()) and prepend to location.
        }

 

  1. by default, NodeJS filesystem assumes data received after a GET is a text. In this case, we are receiving binary data, so we have to let NodeJS know we want it to treat data as an octect stream. This is accomplished by means of the following instruction

 

response.setEncoding('binary');

 

Finally, we have to instruct NodeJS to parse for a specific string from on serial port

 

yunPort = new serialPort.SerialPort('/dev/ttyATH0', { baudrate: 115200 });

yunPort.on('data', function(data) {

  console.log('data ' + data);
  if (_ws)  
  {
     _data = _data + data;
    if (_data.indexOf("\r\n") > 0)
    {
      // check if this is a PLAY command
      if (_data.substring(0, 5) == "PLAY")
      {
        var text = _data.substring(6);
        var mp3url='http://tts-api.com/tts.mp3?q='+encodeURIComponent(text);
        var filename = '/mnt/sda1/' + text.replace(' ', '_') + '.mp3';
        getMp3(text, filename, function() {
             var exec = require('child_process').exec;
             exec('/usr/bin/madplay ' + filename);
         });
      }  
      else 
        _ws.send(_data);

      _data = "";
    }
 }
});

 

Now, on the Atmel side I can simply write

 

Serial.println("hello world");

 

to make MagicHat speak! That's Internet power!

After finishing the data acquisition part of the project, let's see how to make the MagicHat a little more... magic

 

To make Arduino Yun speak, I first need an external USB audio card like this one here

 

IMG_20150528_171510.jpg

 

Then, I need to install an mp3 player that will play a file stored on SD card. Installing the mp3 player is just a few packages away...

 

opkg update
 opkg install kmod-sound-core 
 opkg install kmod-usb-audio 
 opkg install madplay


 

Since the control logic runs on the Atmel side of Arduino Yun, we need to devise a way to launch the player. To send data from Atmel to Linino I disabled the bridge, so now Serial1 is read by NodeJS.

So, the firmware running on Atmel will send out (by calling Serial1.println) a string like

 

PLAY helloworld.mp3


 

and NodeJS will launch the mp3 player to play to "helloworld.mp3" file

 

This can easily achieved using the NodeJS's 'exec'  package

 

var exec = require('child_process').exec;
exec('/usr/bin/madplay ' + filename);


I finally succeeded in measuring blood pressure in an almost reliable way. In this post I will show the results of my experiments

 

In order to determine the law between the value returned by the Force Sensor Resistor and the blood pressure, I run some comparison tests. Basically, I measured my real blood pressure with a normal home medical device (see picture) and associate the readings of the medical devices with the corresponding ADC reading of the FSR.

I made a test on myself (I have a blood pressure of about 130) and my wife (which has a blood pressure of 110)

Here are the data I collected

 

14 - Collected data.png

 

In terms of accuracy, it seems quite good. I applied the following formula to determine the accuracy of the instrument:

 

   14 - Formula1.png

 

I got that the blood pressure reading can vary of less the 10, which is not enough for the MagicHat to be approved for use as a diagnostic device, but it's good enough for a gadget. Also, the design can definitely be improved in terms of sensor selection and mechanical building.

Regarding the selection of the sensor, I probably select the wrong one because its range will allow me to measure blood pressure only up to about 150. IN a next design revision a FSR with a maximum load of up to  3 kilos will be selected

 

Given the above data, I have been able to detemine the following linear correlation between ADC reading and blood pressure

 

  14 - Formula2.png

 

Since I will use Arduino's map function, I need the values corresponding to ADC readings 0 and 1023. So

 

ADC reading = 0 -> Blood pressure = -253,24

ADC reading = 1023 -> Blood pressure = 196,88

 

The above-mentioned results have been obtained in parallel with the choice of the most suitable speed of the elastic rubber string that forces the patient's finger. I need to determine whether the blood is still flowing so I have to wait at least 1.5 seconds to determine whether heart beat is still detectable or not. This requires the movement to be slow. However, it's not very comfortable for the patient if it takes a long time just to measure blood pressure...

The best compromise I found was to move the servo for 20 ms and stop it for 500 ms.

 

Here is a video showing the blood pressure measurement device at work

 

In this post we will have a look  at how the client side of the web application is implemented

For the web client I used the Angular JS framework and the graphical components made by Telerik. This components are not free (I have a licence since I used those components at work, but they can be easily replaced with free and/or open source components)

 

The most interesting part is how the web socket is created and how data is processed to update the web page.

The web socket is created by passing the url of thehost we want to connect to (in this case I am using the default IP address of Arduino Yun

 

                $scope.dataService = new WebSocket('ws://192.168.240.1:3000');

 

Then, we can add the handlers for the events we are interested in. For example, I am interested to handle the "data received event"

 

                $scope.dataService.onmessage = function(event){ 
                       console.log('data ' + event.data);
                       $scope.updateData(JSON.parse(event.data));
                };

 

Eventually, you could also get notification whenever the connection is open or when an error occurs

 

$scope.dataService.onopen = function(){ 
                       console.log('Hello');
                };                                            
 $scope.dataService.onerror = function(){ 
                       console.log('Error');
                };

 

The $scope.updateData(data) is a function that takes care of updating the widgets with the data received on the web socket. The JSON.parse(event.data) function creates a Javascript object from the string received. The object structure will look like this

 

{

temperature: 37,

pressure: {min: 0, max: 120},

bpm: 62,

pulse: [0,0,0,...,0,0];

}

 

This object is passed to the   $scope.updateData function

 

$scope.updateData = function(data)
 {
 if (data.temperature)
 $("#gauge_temp").data("kendoLinearGauge").value(data.temperature);

 if (data.pressure)
 {
 $("#bpmin-value").val(data.pressure.min);
 $("#bpmax-value").val(data.pressure.max);
 $("#gauge_bp").data("kendoRadialGauge").value(data.pressure.max);
 }

 if (data.bpm)
 { 
 $("#read_bpm").each(function() {
        $(this).industrial(data.bpm);
 });
 }

 if (data.pulse)
 { 
      if ($scope.pulse.length + (data.pulse.length/10) > 50)
      {
        var howMany = ($scope.pulse.length + (data.pulse.length/10)) - 50;
        $scope.pulse.splice(0, howMany);
 }                    
           
 console.log('Adding values..' + data.pulse.length);
 for (var i=0; i<data.pulse.length; i+=10)
 {
        $scope.pulse.push({ value: data.pulse[i] });
 }
 }
 };

 

The final result is shown in the following video

 

While struggling to complete the blood-pressure measurement device, let's have a look about how to show measured value

 

In this project, measures will be communicated to the user in two way

  1. through a web interface that can be accessed by any device running a web browser
  2. through a Text-To-Speech engine

 

As I'm still waiting for the components required to build the Text-To-Speech engine, in this post I will focus on the web interface

The web interface will be built using node.

The first step to take is to install node and expand the memory. I followed this tutorial to accomplish this task. All steps completed with no issues (except you have to change to opkg configuration file by replacing the line

src/gz barrier_breaker http://download.linino.org/dogstick/all-in-one/latest/packages/

with

src/gz barrier_breaker http://downloads.arduino.cc/openwrtyun/1/packages

 

 

As I like experimenting new approaches to the a problem, I decided to move away from the typical solution based on periodic poll to update the values on the web interface and use web sockets instead. The data flow is shown in picture below

 

12 - Web interface.png

 

Basically, whenever the Atmel 32U4 has updated data, it sends such data (formatted as a JSON document) to the node application that is listening on port /dev/ttyATH0)

If a client is connected on a web socket, the node application sends the same document on the web socket. Upon receiving new data, web client update widgets accordingly

This is not the best approach from a software-engineering point of view (I broke the separation-of-concern principle as the Atmel32U4 should be unaware of data format used by the web client) but it's quick and easy  to implement and to understand

 

First of all, a node package needs to be installed in order to access the serial port from the node application itself

 

opkg node-serial


 

In the node application, first of all we need to create a listening web socket, that passively waits for a client connection.

 

_ws = null;
var wss = new webSocketServer({server:server});
wss.on('connection', function(ws) {
  _ws = ws;
  ws.on('message', function incoming(message) {
    console.log('received: %s', message);
  });
  ws.on('close', function(message) {
    console.log('closed');
    _ws = null;
  });



 

Then, we need to open the serial port that listens for data from Atmel 32U4.

 

_data = "";
yunPort = new serialPort.SerialPort('/dev/ttyATH0', { baudrate: 115200 });
yunPort.on('data', function(data) {
  console.log('data ' + data);
  if (_ws)
  {
    _data = _data + data;
    if (_data.indexOf("\r\n") > 0)
    {
      _ws.send(_data);
      _data = "";
    }
  }
});


 

The CR+LF bytes to detect the end of the JSON document produced by the Arduino sketch, so inside the "data received" event I need to check for the presence of the CR+LF pair. If this the case, I can send data to the web client

 

As I said, the Arduin sketch uses the serial connection (which is typically wrapped by the Bridge library) to send data. In order to have full access to the serial port, we first need to change the /etc/inittab file and remove the link between the serial port itself and the console. This operation is very simple: just open the file /etc/inittab and comment the last line as shown here

 

::sysinit:/etc/init.d/rcS S boot
::shutdown:/etc/init.d/rcS K shutdown
# removed to use Serial1 from Arduino sketch
#ttyATH0::askfirst:/bin/ash --login 

 

Now, in the Arduino sketch, we can initialize the serial port

 

Serial1.begin(115200)


 

and finally write any data we want. In this case, I'm sending a JSON document ready to be sent to the web client

 

Serial1.print("{\"temperature\":");
Serial1.print(hat.getTemperature());
    Serial1.print(", \"bpm\":");
    Serial1.print(hat.getBPM());
    Serial1.print(", \"pressure\":{\"min\":0,\"max\":");
Serial1.print(hat.getPressure());
    Serial1.print("}, \"pulse\":[");
    for (i = 0; i<pulseDataLen; i++)
    {
      if (i > 0)
Serial1.print(",");
      Serial1.print(pulseData[i]);
    }
Serial1.println("]}");


 

The information included in the JSON document are

  1. temperature
  2. heartbeat per minute
  3. blood pressure
  4. heartbeat samples taken by the pulse sensor. In this way, the web client can show an interesting chart with the sampled values

 

In next post, I will talk about the client-side implementation of the web interface. See you soon!

The blood pressure measurement system revealed to be more challenging than I expected...

As a matter of fact, the pulse detection algorithm takes up most of the Atmel 32U4 resources, thus leaving too few time to other tasks. This has some counter-effects, the most noticeable being a non-uniform movement of the continuous rotation servo.

I tried several workaround, like moving the pulse detection algorithm in Timer3 ISR instead of Timer1 ISR (and obviously forcing the Servo  library to use Timer1 instead of Timer3) but the problem was still there. I also tried to skip the Servo library and generate the PWM for the servo using the analogWrite function, but the problem was not resolved. I guess the problem is that the pulse detection algorithm completely disables interrupts, thus causing the PWM signal to jitter randomly

For this reason, I decided to move the servo control logic to another Arduino board. The Enchanted objects challenge's kit included a Arduino UNO board, but unfortunately I can use this board due to size contraints. So the choice was for the Arduino Mini Pro board, which is small enough to accommodate inside the MagicHat.

Despite the additional work this decision involves, there are also some benefits. For example, the mouth control logic I originally removed from the list of features I planned to implement is returning now into play.

The connection between the two boards is very simple:

  • a digital output driven by Arduino Yun tells Arduino Mini to start the blood pressure servo
  • a digital output driven by Arduino Yun tells Arduino Mini to bend the hat

 

 

11 - Boards_bb.png