Introduction

The Leap Motion Controller is something that can be used as an input device for a computer (desktop/laptop or a single board computer). It can be placed on the desk, or if desired can be strapped onto a virtual reality (VR) headset if you have one. In brief, when it sees your hand or hands, it can analyse them to figure out where your fingers, palms and so on, are in three dimensional space. It can also figure out certain movements such as swipes. You could use it for controlling stuff, or even controlling Python games.

 

It is only just bigger than a USB memory stick, so it is quite compact.

 

Here is a snippet of the possibilities, from the Leap website:

 

I recently noticed Leap Motion controller devices cheap on ebay, so I was curious to try it out. Knowing nothing about them, I was kind-of hoping to somehow get it functioning with ARM boards such as a Pi or BeagleBone Black, but sadly it appears that internally they contain little processing; all the sensor data is streamed via USB to a PC, which does the heavy processing, although apparently Android support is coming soon. Internally, according to online teardowns, there are a couple of CMOS cameras with fish-eye lenses, and a few infra-red (IR) LEDs, and a Cypress chip to stream all the captured data to the PC. The top surface is a plastic IR filter. With the fish-eye lenses, the Leap Motion Controller gets to see a space about the shape of a hemisphere of 600mm radius which is pretty huge. There is more information about how it works on the Leap website.

 

Image source: Leap website

 

Anyway, to get it running on a PC, the desktop core software and app-store download is pretty huge (400MB+), and frankly I wasn’t really interested in running other people’s apps. They are focused toward gaming, desktop actions and so on, but I get the feeling a lot of the ready-made apps are just for the novelty factor today.

 

Writing your own code and controlling your own games or devices sounded like more fun, and for that, there is no need to download the huge core software app bundle. Instead, a software developer kit (SDK) can be downloaded.

 

The developer kit is supported on Windows/Linux/Mac, and as mentioned Android should be coming soon.

 

Anyway, I fired up Ubuntu and tried it out! I was primarily interested in creating apps with JavaScript and Node.js, so that’s what this blog post will discuss.

 

The notes are below, they are not necessarily complete, but contain all the important stuff to get started. If you spot any errors or have any suggestions, please let me know.

 

What is Needed?

Apart from the Leap device (which comes with USB cables), just a x86 or x86-64 PC or single board computer.  This works on any reasonable PC. The Leap website suggests an Intel Core i3 or upward processor, or AMD Phenom II upward, and 2GB of RAM, I tested it on a laptop which met this requirement, but I also tested on a Gizmo2 (which sadly isn’t sold any more) – and that has just a low-power AMD chip and 1GB of DRAM! It seems to work fine in my limited tests.

 

The Gizmo2 and similar compact boards allow for the Leap Motion device to be used for embedded or kiosk applications. Sadly the Intel IoT boards like Joule are end-of-sale too, otherwise it would have been really neat to try that. I think it would have run really well on that!

 

Getting Started: Install Linux

I installed Ubuntu 16.04, and then as root user in a terminal window proceeded to type the following:

apt-get update
apt-get upgrade

 

Install the Leap Software Developer Kit (SDK)

The Linux SDK can be downloaded from: https://warehouse.leapmotion.com/apps/4186/download

Create a folder to store all your leap stuff. From my home folder, I typed:

 

mkdir -p development/leap
cd development/leap

 

Download the SDK as mentioned, and then extract as follows:

 

gunzip Leap_Motion_SDK_Linux_2.3.1.tgz
tar xvf Leap_Motion_SDK_Linux_2.3.1.tar
cd LeapDeveloperKit_2.3.1+31549_linux

 

Check if you’re running 32-bit or 64-bit Linux:

uname -m

 

If the output says x86_64 then that means you’re running 64-bit, also known as x64.

Install the appropriate extracted SDK (either 32-bit which has ..x86.deb suffix, or 64-bit which is …x64.deb suffix):

 

su
dpkg –install Leap-2.3.1+31549-x64.deb

 

The previous command will complete, but with an error about leapd.service not starting.

Follow the instructions here: https://forums.leapmotion.com/t/tip-ubuntu-systemd-and-leapd/2118

 

Start up the service using:

 

systemctl start leapd.service
systemctl enable leapd.service

 

If that ‘start’ command works, but the ‘enable’ command gives an error, don’t worry, that will be resolved in a while.

 

Install Node.js

As root user:

 

apt-get install nodejs
ln -s /usr/bin/nodejs /usr/bin/node
apt-get install npm

 

Install the Leap JavaScript API and examples

As root user:

npm uninstall bufferutil
npm install -g bufferutil
npm install -g leapjs

 

Still as root user:

 

apt-get install git

 

Note that if the above command complains about a leapd file existing, then type:

 

rm /etc/init.d/leapd

 

Then re-issue the apt-get install git command, and select Y to reinstall it if prompted.

 

Go to the development/leap folder, and as non-root user, download the examples,

 

git clone https://github.com/leapmotion/leapjs.git
cd leapjs/examples

 

Then, as root user:

 

npm install -g gl-matrix
npm install -g ws
npm install -g http-server
npm install gl-matrix
npm install ws

 

You’re almost done!

However, install some interesting examples by going to the development/leap folder and typing (as non-root user):

 

git clone https://github.com/leapmotion/javascript.git

 

This creates a folder called javascript, which contains some nice examples

 

Running an Example

Plug in the Leap Motion device, and confirm the service is running by typing:

 

systemctl is-active leapd.service

 

If it is not active (e.g. crashed), start it up using (as root user):

 

systemctl start leapd.service

 

As non-root user, type:

 

cd javascript/v2/connected-fingers
http-server

 

Now open up a browser and navigate to http://127.0.0.1:8080/index.html

You should see a black screen.

Hover your hand over the Leap Motion device to see it appear in the browser!

 

In the terminal, press Ctrl-C to quit at any time.

Here is a 30-second video of me messing about with it:

 

 

Creating your own Example

To create a custom example, I didn’t currently care about graphical output. I wanted to be able to get information that I could use to directly manipulate my computer, or to manipulate attached hardware. For example, it should be possible to swipe to turn on a light! Or control a MIDI synthesizer.

 

So, I took some example code and modified it to use the ‘gesture’ API based on an example on the Leap website. This would allow me to know if the had was doing things like swiping, circular motions, pressing keys in mid-air and so forth.

 

Go to the development/leap folder, and then type:

 

cd leapjs/examples

 

In this examples folder, there is a program called node.js that doesn’t do much except indicate frames received.

I made a copy of that program (I named the copy gesture.js) and edited it in the same folder. Here is the entire code:

 

#!/usr/bin/node
require('../template/entry');


var controller = new Leap.Controller()
controller.on("frame", function(frame) {
  //console.log("Frame: " + frame.id + " @ " + frame.timestamp);
});


var frameCount = 0;
controller.on("frame", function(frame) {
  frameCount++;
  if (frame.valid && frame.gestures.length>0){
    frame.gestures.forEach(function(gesture){
      switch(gesture.type){
        case "circle":
          console.log("Circle gesture");
          break;
        case "keyTap":
          console.log("Key Tap gesture");
          break;
        case "screenTap":
          console.log("Screen Tap gesture");
          break;
        case "swipe":
          console.log("Swipe gesture");
          break;
      } // end switch
    }); // end frame.gestures.forEach




  } // end if (frame.valid


});


setInterval(function() {
  var time = frameCount/2;
  console.log("received " + frameCount + " frames @ " + time + "fps");
  frameCount = 0;
}, 2000);


controller.on('ready', function() {
    console.log("ready");
});
controller.on('connect', function() {
    console.log("connect");
});
controller.on('disconnect', function() {
    console.log("disconnect");
});
controller.on('focus', function() {
    console.log("focus");
});
controller.on('blur', function() {
    console.log("blur");
});
controller.on('deviceConnected', function() {
    console.log("deviceConnected");
});
controller.on('deviceDisconnected', function() {
    console.log("deviceDisconnected");
});


controller.connect();
console.log("\nWaiting for device to connect...");

 

 

You can use the diff command to see what I changed; it wasn’t much!

 

To run this example, as non-root user, first set the file permission:

 

chmod 755 gesture.js

 

 

Then, just type:

 

./gesture.js

 

It will stream on the terminal text output, whenever the hand is swiped etc. It is really basic, but demonstrates how to capture gestures in order to do something with them. Currently it just logs to the console.

 

Summary

With little effort, and low cost, it is possible to capture three-dimensional hand and finger motions into the computer, and do things with them! Unfortunately today this needs a PC or SBC with x86/x64 capability, but hopefully in the future this is possible with ARM SBCs too.

 

As next steps, I might try to get this to control external hardware. It would be great to hear about other people's experiences with this device.