4 posts

# Simple Automatic Threshold Generation for Embedded Systems

Posted by satyavrat Mar 20, 2017

Very often it so happens in sensing-actuation based projects that an embedded system must take a certain action when the value of a sensed parameter is above a certain threshold. The most basic method of handling such a control loop is by hard coding a threshold or a set point. Then, for the embedded system, it becomes a binary decision. A High and a Low, even for analog inputs. However, one of the problems caused by this method is that the threshold does not change with a change in secondary environmental parameters affecting the primary sensed value.

Take for example, a system to detect whether or not a box has been opened using a light sensor. I would hard code the threshold, and it would work well at my place because its bright throughout the day. Now suppose I take it to another location where the lighting is not as optimal. The threshold will naturally change, but my code will not be able to adapt to this change without updating the firmware, which is a real problem with a system installed in an inaccessible location. But, if the system were able to adapt to the new lighting conditions, it would be able to determine its own threshold for the same.

I've faced this problem a few times, mainly while determining my threshold based on time series data. A different set of environmental parameters call for a new threshold. What these parameters might be is different for every application, so my initial idea of calculating a complex mathematical inclusion which would be expected to include subjective terms such as location was laughable. And all for segregating my data into two parts? That would be like buying an airliner to get to my local grocery store.

After taking a class on statistics, the intuitive value of statistical analysis for my data led me to come up with this small, rather easy algorithm for automatic generation of thresholds for time series data generated by my systems. Obviously, there are some assumptions here.

• The two groups to be distinguished are mutually exclusive.
• The sampling rate of the algorithm is much slower than the slope of the system.
• The algorithm, hence will not be used for data logging or frequency analysis
• There at least two known values for the algorithm to run.

The algorithm is as follows

Required packages : Numpy, Random, Matplotlib.pyplot

1. Store sensor values in an array

Thresholding function :

1. Sort the array in ascending order
2. Get sequential difference by calculating difference between consecutive elements of ascending order array.(Store these values in an array as well) Let's call this array A.
3. Similarly, get a sequential difference array of the sequential difference array (A) calculated in step 2, let's call it B
4. Select the second largest value in B, and multiply it by two. This is your threshold for parameter for grouping. This will ensure that your groups aren't too close together.
5. Select the element in sequential difference array (A) with the largest value. This element corresponds to the difference between the largest value in the 'OFF' group, ie lower threshold and the smallest value in the 'ON' group, ie the higher threshold, and gives you the difference between the two groups. Let's call this M.
6. Generate the dynamic threshold by dividing M by 2 and adding the value of the lower threshold to get your actual dynamic threshold.
7. Plot the graph.

Add time series data to input array (separate function):When your system senses a new value of light intensity (or your selected conditions);

1. Append the value of light intensity to the input array
2. Check the new value(NV) for the following conditions:
1. NV > Highest value in entire input array
2. NV < Lowest value in entire input array
3. Lower Threshold < NV < Higher threshold
3. If any of these conditions are true, call dynamic threshold function()

The resultant graphs look as follows. The first one is using the priory set of input data, and the second and third ones are after appending randomly generated values to the input dataset in two separate iterations.

As with all algorithms, this one does have its own set of drawbacks which are primarily illustrated by the assumptions made.

• This algorithm cannot be used for data logging and auto classification unless the algorithm selects data at a defined interval from the array of logged values.
• If the sampling rate of the algorithm is higher than the time required for the sensor to become stable, the algorithm will not be able to distinctly group the values because consecutive difference values are going to be very similar.

In the future, I plan to get rid of these drawbacks by introducing a more sophisticated and inclusive method of statistical classification.

# Using an Intel Edison and an old Android phone as a Lightweight Image Processing Platform

Posted by satyavrat Feb 10, 2017

One of the first projects I did when I got my Intel Edison in my third year of engineering, was to try and make a home security system. I had seen a lot of people make many varieties in this theme. You had your infra red tripwires, overhead PIR proximity sensors, all which did the job pretty well. What I found a bit lackluster was the actuation bit. An LED, a Buzzer or may be the odd LCD screen with "INTRUDER!" flashing in the ubiquitous black-over-green. Given that I had a lot more horsepower and connectivity options to work with, I tried to make it a bit more "integrated" with the world we live in today. Admittedly, when one builds a project, we don't see it adopted immediately into the desired ecosystem. There's a lot of work and modification involved. However, that doesn't concern makers, because we are pretty happy with prototypes.

To try and see if the prototype itself could work in the real world, I kept the sensing bit pretty standard, a light sensor to read spikes in ambient light. When it did, however, I decided to click a photo using an old Android phone via remote commands from the Edison, store the photo locally, and then upload it to the user's Twitter handle, which would actually alert the user of a break in, wherever he might be in the world.

At the time, I thought this was the pinnacle of awesomeness. And for a while it was. It's been a general observation in my local engineering colleges that anything with a display or anything that moves is considered a successful project. So having a computer automatically detect an intruder and upload the photo to your own Twitter handle became the apple of every teacher's eye. Needless to say, all my classmates hated me.

The process was quite simple. The Android Play Store houses an app called IPCam, which streams the phone's camera output to a local IP on your network. All you have to do is access that stream via Python and grab a snapshot. Once done, upload the snapshot to predefined Twitter handle.

However, it wasn't until I began working at my first internship that I realized that I had missed the true potential of this setup.

I worked at a company specializing in Image Processing and Embedded Systems for a year, during which time I learned how, with some effort, one can train a computer to intuitively understand what it is seeing through the camera lens. Computer Vision and Image Processing are essentially the back end of how our brains process images as well. (Neural Networks also play a big part here, but let's not get into specifics.) I realized that what I had at home was an ideal platform for some lightweight image processing as well, and how I had severely underutilized its potential in the Twitter project!

Image processing is rather elegant in its premise, and like most machine learning algorithms, is almost purely arithmetic. A big plus, because then the intuition behind it is much easier to understand. Border detection, histograms and motion sensing. Although these may sound like complex concepts, break them down to their mathematical roots, and all of them are ridiculously simple.

I plan on making this kind of a rolling blog to document my experiments in image processing. Intermediate proficiency in Python, a few packages, namely OpenCV and Numpy or Pandas, and, if you're planning on integrating that into IoT, maybe urllib or sockets is all you need. Experimenting with Numpy will be a treat. Analyzing pixel values individually may be very slow, but I'm not looking for lightning fast responses. Yet.

# Which Cloud Service should I use?

Posted by satyavrat Feb 5, 2017

1. ThingSpeak :  Thingspeak was the first service I used to connect my device with the Internet, so maybe that's why it's first in the list. Considering the ones I used later on, ThingSpeak had perhaps the easiest learning curve to get exactly what I wanted from a web service. To upload data, store it, see it, and read it from another place. It allows you to have different channels to read data from different sensors from the same device.

It utilizes a RESTful API like many of those on this list, which allows the user to GET and PUT data using the browser alone. With such an easy API, which can be used with curl on Linux based systems like the Raspberry Pi or Intel Edison, you'd imagine it to be pretty bare bones in terms of functionalities. You'd be wrong.

Thingspeak offers integrated MATLAB analysis on incoming data on the cloud, making it a powerful tool for analytics. Statistical analysis of your dataset is a breeze when you're integrated with something as powerful as the almighty MATLAB.

It also provides APIs to interact with Twitter. In addition, it has support for command queuing and automation with event triggers as well.

The only downside I saw in this platform is the messaging rate which is limited to 1 message per 15 seconds. This does hamper time sensitive systems as a delay of over 2 seconds is considerd huge in say robotic control. Although you can circumvent this by spreading control commands over a number of channels, that would require a more complex device side code in addition to security key complications.

2. Dweet.io : I discovered Dweet.io recently while searching for platforms providing lightweight communication between machines which allow a very low dead time. My intentions were to use it for a real-time system which responded to commands in a negligible amount of time, and Dweet.io almost gave me that.

First of all, it disposes off with the whole sign up and register system, and gives you direct access to your own data repository over a URL. Just entering your own unique device name in the URL provides a small chunk of cloud for you to experiment with.

Dweet also uses something called HAPI web API, which makes responses to Dweet requests understandable to humans rather than a convoluted output format. This makes grabbing required chunks of the response incredibly easy.

I haven't experimented with it yet, but it also lets a physical device subscribe to a Dweet stream. From my standpoint, the event driven action ensures low battery use, which is a good alternative to constant polling, much like the TCP messaging stack.

And finally, Dweet's USP is the transmission delays. As it does not have any heavy cloud machinery hungry for data in the back end, it is a mediator in the purest sense, which gives you blazing fast transmission delays. You can almost call it real time. A 1 message per second rate limit is impressive, to say the least. haven't seen it in many other places.

3. Temboo : Temboo is perhaps the most popular and versatile platforms available for free on the Internet, with a number of useful APIs spanning from Twitter and Facebook to Google Drive access and integration of Google analytics tools such as Google BI.

It's entire software stack is devoted to connecting existing web applications such as social networks and cloud storage to physical devices for interaction. Although their hardware support is limited to texas Instruments and Arduino platforms, their impressive list of SDKs more than makes up for it. You can implement your Temboo appication in your favorite language, and connect to a number of services through it

Temboo has a rather easy learning curve if you consider the possibilities it offers. Although i wouldn't recommend it to beginners, it is a great service to utilize if you are familiar with communication protocols, API design and have a good hands on with any other service.

This is actually a huge platform to explore, and a great one to rely on for integrating your IoT solution to your favorite web service.

4. Mediatek Cloud Sandbox : Mediatek labs opened its own proprietery Cloud Service called the Mediatek Cloud Sandbox, and although it's targetted mainly at the Linkit line of SoCs, the APIs are open to anyone who wants to experiment with cloud service integration into their project.

It is more POC oriented, so don't expect a fleet of advanced features to help in industry level deployment. That is actually a good thing, because that makes the Mediatek Cloud Sandbox a great place for beginners to start.

The Linkit line of SoCs prides itself on its low power consumption and good peripheral support. Therefor it is advertised as a a development solution for wearables as well as IoT. The Mediatek Cloud Sandbox reflects this fact by implementing a RESTful API over HTTP to facilitate power resourceful opertions.

As the Linkit SoCs come bundled with GPS based location sensors, the Mediatek Cloud Sandbox has the welcome addition of geo-location logging which can be useful to a number of applications which aim to combine machine learning with geo-sensing abilities.

With a comparatively simple setup, thorough documentation and a fairly easy learning curve, the Mediatek Cloud Sandbox is a great starting point for beginners to start from if they aim to stay with the same service throughout.

# What are Cloud Services? Analogical Understanding

Posted by satyavrat Feb 5, 2017

If on the off chance that you are like me, you're an Instrumentation Engineer, "Jack of all trades" would probably be a very good description of your skills. The "Master of none" part though, is a bit iffy. It's in the four years that an Instrumentation Engineer spends studying his curriculum  that he must convert the "Master of none" part to "Master of one". At least.

I can discuss diodes with my friends from the Electronics branch, I am able to contemplate compilers with those of the Computer Engineering affiliation and can also chit chat about chassis with those of the Mechanical inclination. But when things get too deep, I need to grab a cold one and wait for the storm of esoteric jargon to pass over. A few hailstones here and there are manageable, but the full blown blizzard is a bit too much.

One of these topics, till the recent past was "Cloud Computing". As the Tom to IoT's Jerry, it's pretty hard to actually pry them apart as they form the backbone of any connected application. So with a "When in Rome," attitude, I sat down to look at what the fuss was all about.

Essentially, cloud computing is divided into 3 broad categories, which are;

• SaaS - Software as a Service
• PaaS - Platform as a Service
• IaaS - Infrastructure as a Service

This diagram is pretty apt for showing what these three terms mean in a nutshell.

To explain in short, imagine that you are at a swimming pool, and you're looking at 3 kids who are swimming there.

The first kid is in the baby pool, because he can't really swim, so he's sort of wading around. he's happy being in the water, and  also safe in the knowledge that something like a puddle that's a bit deeper won't kill him.

That's someone who uses SaaS or Software as Service.

The second kid is thrown into the deep end wearing nothing but his trunks by an over-enthusiastic parent. You observe as his head bobs around the pool as he does a perfect freestyle. He's been there for a while, so he knows his way around water twelve feet deep.

That's someone who uses IaaS or Infrastructure as a Service.

The third kid is floating around beside the freestyler, in a tyre and wearing arm wings. He doesn't know how to swim, but he can float. So he's fine while he has the wings and the tyre on, and he can be pretty much wherever the freestyler is, albeit with some constraints on movement.

That's someone who uses Platform as a Service.

What I'd like to concentrate on is the concept of PaaS or Platform as a Service.

PaaS is really for people like me, with an idea, a plan on how to execute it, and basic knowledge of the technologies that entails. PaaS is to IaaS what PlayDoh is to pottery. It gives you pretty much all of the application level facilities an IaaS offers, but in a nice pre-built package that you can customize. In this respect, SaaS is like buying a pot. You can't really say you made it, but you'll be using it.

Coming to IoT. The utopian concept of everything connected to everything, whose first steps have already been taken requires a constant back end operating on the cloud. Ideally speaking, an IaaS is the perfect option to start from scratch and develop your platform from the ground up. But, in my case, that would really need me to know quite a lot of things whose knowledge would suit a degree in itself.

Now what do I do?

That's where PaaS comes into the picture. PaaS is essentially a platform built to allow customization in the application and firmware side of things such as custom filters and algorithms for analytics and different qualities of service and export formats for M2M communication and visualization of data.

I've used quite a few PaaS offered by a number of providers, and it was pretty confusing to choose the right one. What with the different APIs, protocols, storage options and not to mention pricing, it's not a very easy task to know which one you should use unless you know exactly what you expect from a PaaS.

By date: By tag: