Skip navigation

Arduino

4 Posts authored by: msb4180

In my previous post, I showed how to control a few LEDs using an Arduino board and BitVoicer Server. In this post, I am going to make things a little more complicated. I am also going to synthesize speech using the Arduino DUE digital-to-analog converter (DAC). If you do not have an Arduino DUE, you can use other Arduino boards, but you will need an external DAC and some additional code to operate the DAC (the BVSSpeaker library will not help you with that).

 

Img1.jpg

In the video below, you can see that I also make the Arduino play a little song and blink the LEDs as if they were piano keys. Sorry for my piano skills, but that is the best I can do . The LEDs actually blink in the same sequence and timing as real C, D and E keys, so if you have a piano around you can follow the LEDs and play the same song. It is a jingle from an old retailer (Mappin) that does not even exist anymore.

 

 

The following procedures will be executed to transform voice commands into LED activity and synthesized speech:

 

  1. Audio waves will be captured and amplified by the Sparkfun Electret Breakout board;
  2. The amplified signal will be digitalized and buffered in the Arduino using its analog-to-digital converter (ADC);
  3. The audio samples will be streamed to BitVoicer Server using the Arduino serial port;
  4. BitVoicer Server will process the audio stream and recognize the speech it contains;
  5. The recognized speech will be mapped to predefined commands that will be sent back to the Arduino. If one of the commands consists in synthesizing speech, BitVoicer Server will prepare the audio stream and send it to the Arduino;
  6. The Arduino will identify the commands and perform the appropriate action. If an audio stream is received, it will be queued into the BVSSpeaker class and played using the DUE DAC and DMA.
  7. The SparkFun Mono Audio Amp will amplify the DAC signal so it can drive an 8 Ohm speaker.

 

List of Materials:

 

 

STEP 1: Wiring

 

The first step is to wire the Arduino and the breadboard with the components as shown in the pictures below. I had to place a small rubber underneath the speaker because it vibrates a lot and without the rubber the quality of the audio is considerably affected.

 

Protoboard

Wiring 1

Wiring 2

Wiring 3

 

Here we have a small but important difference from my previous post. Most Arduino boards run at 5V, but the DUE runs at 3.3V. Because I got better results running the Sparkfun Electret Breakout at 3.3V, I recommend you add a jumper between the 3.3V pin and the AREF pin IF you are using 5V Arduino boards. The DUE already uses a 3.3V analog reference so you do not need a jumper to the AREF pin. In fact, the AREF pin on the DUE is connected to the microcontroller through a resistor bridge. To use the AREF pin, resistor BR1 must be desoldered from the PCB.

 

STEP 2: Uploading the code to the Arduino

 

Now you have to upload the code below to your Arduino. You can also download the Arduino sketch from the link below the code. Before you upload the code, you must properly install the BitVoicer Server libraries into the Arduino IDE (Importing a .zip Library).

 

#include <BVSP.h>
#include <BVSMic.h>
#include <BVSSpeaker.h>
#include <DAC.h>

// Defines the Arduino pin that will be used to capture audio
#define BVSM_AUDIO_INPUT 7

// Defines the LED pins
#define RED_LED_PIN 6
#define YELLOW_LED_PIN 9
#define GREEN_LED_PIN 10

// Defines the constants that will be passed as parameters to
// the BVSP.begin function
const unsigned long STATUS_REQUEST_TIMEOUT = 3000;
const unsigned long STATUS_REQUEST_INTERVAL = 4000;

// Defines the size of the mic audio buffer
const int MIC_BUFFER_SIZE = 64;

// Defines the size of the speaker audio buffer
const int SPEAKER_BUFFER_SIZE = 128;

// Defines the size of the receive buffer
const int RECEIVE_BUFFER_SIZE = 2;

// Initializes a new global instance of the BVSP class
BVSP bvsp = BVSP();

// Initializes a new global instance of the BVSMic class
BVSMic bvsm = BVSMic();

// Initializes a new global instance of the BVSSpeaker class
BVSSpeaker bvss = BVSSpeaker();

// Creates a buffer that will be used to read recorded samples
// from the BVSMic class
byte micBuffer[MIC_BUFFER_SIZE];

// Creates a buffer that will be used to write audio samples
// into the BVSSpeaker class
byte speakerBuffer[SPEAKER_BUFFER_SIZE];

// Creates a buffer that will be used to read the commands sent
// from BitVoicer Server.
// Byte 0 = pin number
// Byte 1 = pin value
byte receiveBuffer[RECEIVE_BUFFER_SIZE];

// These variables are used to control when to play
// "LED Notes". These notes will be played along with
// the song streamed from BitVoicer Server.
bool playLEDNotes = false;
unsigned int playStartTime = 0;

void setup()
{
  // Sets up the pin modes
  pinMode(RED_LED_PIN, OUTPUT);
  pinMode(YELLOW_LED_PIN, OUTPUT);
  pinMode(GREEN_LED_PIN, OUTPUT);

  // Sets the initial state of all LEDs
  digitalWrite(RED_LED_PIN, LOW);
  digitalWrite(YELLOW_LED_PIN, LOW);
  digitalWrite(GREEN_LED_PIN, LOW);

  // Starts serial communication at 115200 bps
  Serial.begin(115200);

  // Sets the Arduino serial port that will be used for
  // communication, how long it will take before a status request
  // times out and how often status requests should be sent to
  // BitVoicer Server.
  bvsp.begin(Serial, STATUS_REQUEST_TIMEOUT, STATUS_REQUEST_INTERVAL);

  // Defines the function that will handle the frameReceived
  // event
  bvsp.frameReceived = BVSP_frameReceived;

  // Sets the function that will handle the modeChanged
  // event
  bvsp.modeChanged = BVSP_modeChanged;

  // Sets the function that will handle the streamReceived
  // event
  bvsp.streamReceived = BVSP_streamReceived;

  // Prepares the BVSMic class timer
  bvsm.begin();

  // Sets the DAC that will be used by the BVSSpeaker class
  bvss.begin(DAC);
}


void loop()
{
  // Checks if the status request interval has elapsed and if it
  // has, sends a status request to BitVoicer Server
  bvsp.keepAlive();

  // Checks if there is data available at the serial port buffer
  // and processes its content according to the specifications
  // of the BitVoicer Server Protocol
  bvsp.receive();

  // Checks if there is one SRE available. If there is one,
  // starts recording.
  if (bvsp.isSREAvailable())
  {
    // If the BVSMic class is not recording, sets up the audio
    // input and starts recording
    if (!bvsm.isRecording)
    {
      bvsm.setAudioInput(BVSM_AUDIO_INPUT, DEFAULT);
      bvsm.startRecording();
    }

    // Checks if the BVSMic class has available samples
    if (bvsm.available)
    {
      // Makes sure the inbound mode is STREAM_MODE before
      // transmitting the stream
      if (bvsp.inboundMode == FRAMED_MODE)
        bvsp.setInboundMode(STREAM_MODE);

      // Reads the audio samples from the BVSMic class
      int bytesRead = bvsm.read(micBuffer, MIC_BUFFER_SIZE);

      // Sends the audio stream to BitVoicer Server
      bvsp.sendStream(micBuffer, bytesRead);
    }
  }
  else
  {
    // No SRE is available. If the BVSMic class is recording,
    // stops it.
    if (bvsm.isRecording)
      bvsm.stopRecording();
  }

  // Plays all audio samples available in the BVSSpeaker class
  // internal buffer. These samples are written in the
  // BVSP_streamReceived event handler. If no samples are
  // available in the internal buffer, nothing is played.
  bvss.play();

  // If playLEDNotes has been set to true,
  // plays the "LED notes" along with the music.
  if (playLEDNotes)
    playNextLEDNote();
}

// Handles the frameReceived event
void BVSP_frameReceived(byte dataType, int payloadSize)
{
  // Checks if the received frame contains binary data
  // 0x07 = Binary data (byte array)
  if (dataType == DATA_TYPE_BINARY)
  {
    // If 2 bytes were received, process the command.
    if (bvsp.getReceivedBytes(receiveBuffer, RECEIVE_BUFFER_SIZE) ==
      RECEIVE_BUFFER_SIZE)
    {
      analogWrite(receiveBuffer[0], receiveBuffer[1]);
    }
  }
  // Checks if the received frame contains byte data type
  // 0x01 = Byte data type
  else if (dataType == DATA_TYPE_BYTE)
  {
    // If the received byte value is 255, sets playLEDNotes
    // and marks the current time.
    if (bvsp.getReceivedByte() == 255)
    {
      playLEDNotes = true;
      playStartTime = millis();
    }
  }
}

// Handles the modeChanged event
void BVSP_modeChanged()
{
  // If the outboundMode (Server --> Device) has turned to
  // FRAMED_MODE, no audio stream is supposed to be received.
  // Tells the BVSSpeaker class to finish playing when its
  // internal buffer become empty.
  if (bvsp.outboundMode == FRAMED_MODE)
    bvss.finishPlaying();
}

// Handles the streamReceived event
void BVSP_streamReceived(int size)
{
  // Gets the received stream from the BVSP class
  int bytesRead = bvsp.getReceivedStream(speakerBuffer,
    SPEAKER_BUFFER_SIZE);

  // Enqueues the received stream to play
  bvss.enqueue(speakerBuffer, bytesRead);
}

// Lights up the appropriate LED based on the time
// the command to start playing LED notes was received.
// The timings used here are syncronized with the music.
void playNextLEDNote()
{
  // Gets the elapsed time between playStartTime and the
  // current time.
  unsigned long elapsed = millis() - playStartTime;

  // Turns off all LEDs
  allLEDsOff();

  // The last note has been played.
  // Turns off the last LED and stops playing LED notes.
  if (elapsed >= 11500)
  {
    analogWrite(RED_LED_PIN, 0);
    playLEDNotes = false;
  }
  else if (elapsed >= 9900)
    analogWrite(RED_LED_PIN, 255); // C note
  else if (elapsed >= 9370)
    analogWrite(RED_LED_PIN, 255); // C note
  else if (elapsed >= 8900)
    analogWrite(YELLOW_LED_PIN, 255); // D note
  else if (elapsed >= 8610)
    analogWrite(RED_LED_PIN, 255); // C note
  else if (elapsed >= 8230)
    analogWrite(YELLOW_LED_PIN, 255); // D note
  else if (elapsed >= 7970)
    analogWrite(YELLOW_LED_PIN, 255); // D note
  else if (elapsed >= 7470)
    analogWrite(RED_LED_PIN, 255); // C note
  else if (elapsed >= 6760)
    analogWrite(GREEN_LED_PIN, 255); // E note
  else if (elapsed >= 6350)
    analogWrite(RED_LED_PIN, 255); // C note
  else if (elapsed >= 5880)
    analogWrite(YELLOW_LED_PIN, 255); // D note
  else if (elapsed >= 5560)
    analogWrite(RED_LED_PIN, 255); // C note
  else if (elapsed >= 5180)
    analogWrite(YELLOW_LED_PIN, 255); // D note
  else if (elapsed >= 4890)
    analogWrite(YELLOW_LED_PIN, 255); // D note
  else if (elapsed >= 4420)
    analogWrite(RED_LED_PIN, 255); // C note
  else if (elapsed >= 3810)
    analogWrite(GREEN_LED_PIN, 255); // E note
  else if (elapsed >= 3420)
    analogWrite(RED_LED_PIN, 255); // C note
  else if (elapsed >= 2930)
    analogWrite(YELLOW_LED_PIN, 255); // D note
  else if (elapsed >= 2560)
    analogWrite(RED_LED_PIN, 255); // C note
  else if (elapsed >= 2200)
    analogWrite(YELLOW_LED_PIN, 255); // D note
  else if (elapsed >= 1930)
    analogWrite(YELLOW_LED_PIN, 255); // D note
  else if (elapsed >= 1470)
    analogWrite(RED_LED_PIN, 255); // C note
  else if (elapsed >= 1000)
    analogWrite(GREEN_LED_PIN, 255); // E note
}

// Turns off all LEDs.
void allLEDsOff()
{
  analogWrite(RED_LED_PIN, 0);
  analogWrite(YELLOW_LED_PIN, 0);
  analogWrite(GREEN_LED_PIN, 0);
}






 

Arduino Sketch: BVS_Demo2.ino

 

This sketch has seven major parts:

 

  • Library references and variable declaration: The first four lines include references to the BVSP, BVSMic, BVSSpeaker and DAC libraries. These libraries are provided by BitSophia and can be found in the BitVoicer Server installation folder. The DAC library is included automatically when you add a reference to the BVSSpeaker library. The other lines declare constants and variables used throughout the sketch. The BVSP class is used to communicate with BitVoicer Server, the BVSMic class is used to capture and store audio samples and the BVSSpeaker class is used to reproduce audio using the DUE DAC.
  • Setup function: This function performs the following actions: sets up the pin modes and their initial state; initializes serial communication; and initializes the BVSP, BVSMic and BVSSpeaker classes. It also sets “event handlers” (they are actually function pointers) for the frameReceived, modeChanged and streamReceived events of the BVSP class.
  • Loop function: This function performs five important actions: requests status info to the server (keepAlive() function); checks if the server has sent any data and processes the received data (receive() function); controls the recording and sending of audio streams (isSREAvailable(), startRecording(), stopRecording() and sendStream() functions); plays the audio samples queued into the BVSSpeaker class (play() function); and calls the playNextLEDNote() function that controls how the LEDs should blink after the playLEDNotes command is received.
  • BVSP_frameReceived function: This function is called every time the receive() function identifies that one complete frame has been received. Here I run the commands sent from BitVoicer Server. Commands that controls the LEDs contains 2 bytes. The first byte indicates the pin and the second byte indicates the pin value. I use the analogWrite() function to set the appropriate value to the pin. I also check if the playLEDNotes command, which is of Byte type, has been received. If it has been received, I set playLEDNotes to true and mark the current time. This time will be used by the playNextLEDNote function to synchronize the LEDs with the song.
  • BVSP_modeChanged function: This function is called every time the receive() function identifies a mode change in the outbound direction (Server --> Arduino). WOW!!! What is that?! BitVoicer Server can send framed data or audio streams to the Arduino. Before the communication goes from one mode to another, BitVoicer Server sends a signal. The BVSP class identifies this signal and raises the modeChanged event. In the BVSP_modeChanged function, if I detect the communication is going from stream mode to framed mode, I know the audio has ended so I can tell the BVSSpeaker class to stop playing audio samples.
  • BVSP_streamReceived function: This function is called every time the receive() function identifies that audio samples have been received. I simply retrieve the samples and queue them into the BVSSpeaker class so the play() function can reproduce them.
  • playNextLEDNote function: This function only runs if the BVSP_frameReceived function identifies the playLEDNotes command. It controls and synchronizes the LEDs with the audio sent from BitVoicer Server. To synchronize the LEDs with the audio and know the correct timing, I used Sonic Visualizer. This free software allowed me to see the audio waves so I could easily tell when a piano key was pressed. It also shows a time line and that is how I got the milliseconds used in this function. Sounds like a silly trick and it is. I think it would be possible to analyze the audio stream and turn on the corresponding LED, but that is out of my reach.

 

STEP 3: Importing BitVoicer Server Solution Objects

 

Now you have to set up BitVoicer Server to work with the Arduino. BitVoicer Server has four major solution objects: Locations, Devices, BinaryData and Voice Schemas.

 

Locations represent the physical location where a device is installed. In my case, I created a location called Home.

 

Devices are the BitVoicer Server clients. I created a Mixed device, named it ArduinoDUE and entered the communication settings. IMPORTANT: even the Arduino DUE has a small amount of memory to store all the audio samples BitVoicer Server will stream. If you do not limit the bandwidth, you would need a much bigger buffer to store the audio. I got some buffer overflows for this reason so I had to limit the Data Rate in the communication settings to 8000 samples per second.

 

BinaryData is a type of command BitVoicer Server can send to client devices. They are actually byte arrays you can link to commands. When BitVoicer Server recognizes speech related to that command, it sends the byte array to the target device. I created one BinaryData object to each pin value and named them ArduinoDUEGreenLedOn, ArduinoDUEGreenLedOff and so on. I ended up with 18 BinaryData objects in my solution, so I suggest you download and import the objects from the VoiceSchema.sof file below.

 

Voice Schemas are where everything comes together. They define what sentences should be recognized and what commands to run. For each sentence, you can define as many commands as you need and the order they will be executed. You can also define delays between commands. That is how I managed to perform the sequence of actions you see in the video.

 

One of the sentences in my Voice Schema is “play a little song.” This sentence contains two commands. The first command sends a byte that indicates the following command is going to be an audio stream. The Arduino then starts “playing” the LEDs while the audio is being transmitted. The audio is a little piano jingle I recorded myself and set it as the audio source of the second command. BitVoicer Server supports only 8-bit mono PCM audio (8000 samples per second) so if you need to convert an audio file to this format, I recommend the following online conversion tool: http://audio.online-convert.com/convert-to-wav.

 

You can import (Importing Solution Objects) all solution objects I used in this post from the files below. One contains the DUE Device and the other contains the Voice Schema and its Commands.

 

Solution Object Files:

 

STEP 4: Conclusion

 

There you go! You can turn everything on and do the same things shown in the video.

 

 

As I did in my previous post, I started the speech recognition by enabling the Arduino device in the BitVoicer Server Manager. As soon as it gets enabled, the Arduino identifies an available Speech Recognition Engine and starts streaming audio to BitVoicer Server. However, now you see a lot more activity in the Arduino RX LED while audio is being streamed from BitVoicer Server to the Arduino.

 

In my next post, I will be a little more ambitious. I going to add WiFi communication to one Arduino and control two other Arduinos all together by voice. I am thinking of some kind of game between them. Suggestions are very welcome!

In this post I am going to show how to use an Arduino board and BitVoicer Server to control a few LEDs with voice commands. I will be using the Arduino Micro in this post, but you can use any Arduino board you have at hand.

 

BVS Demo Image 1

 

 

The following procedures will be executed to transform voice commands into LED activity:

 

  1. Audio waves will be captured and amplified by the Sparkfun Electret Breakout board;
  2. The amplified signal will be digitalized and buffered in the Arduino using its analog-to-digital converter (ADC);
  3. The audio samples will be streamed to BitVoicer Server using the Arduino serial port;
  4. BitVoicer Server will process the audio stream and recognize the speech it contains;
  5. The recognized speech will be mapped to predefined commands that will be sent back to the Arduino;
  6. The Arduino will identify the commands and perform the appropriate action.

 

The video above shows the final result of this post. Note in the video that BitVoicer Server also provides synthesized speech feedback. This speech feedback is defined in the server and reproduced by the server audio adapter, but the synthesized audio could also be sent to the Arduino and reproduced using a digital-to-analog converter (DAC). In my next post, I am going to show how to use the Arduino DUE, one amplified and one speaker to reproduce the synthesized speech using the Arduino itself.


List of Materials:



STEP 1: Wiring


The first step is to wire the Arduino and the breadboard with the components as shown in the pictures below.


FritzingProtoboard.jpg


Img2.jpg


Img3.jpg


Img4.jpg

 

The most important detail here refers to the analog reference provided to the Arduino ADC. In my tests, I got better results using 3.3V with the Sparkfun Electrect Breakout. That is why I added a jumper between the 3.3V pin and the AREF pin. If you decide to use the analogRead funcion (for any reason) while 3.3V is being applied to the AREF pin, you MUST call analogReference(EXTERNAL) before you use the analogRead function. Otherwise, you will short together the active reference voltage (internally generated) and the AREF pin, possibly damaging the microcontroller on your Arduino board.


STEP 2: Uploading the code to the Arduino


Now you have to upload the code below to your Arduino. You can also download the Arduino sketch from the link below. Before you upload the code, you must properly install the BitVoicer Server libraries into the Arduino IDE (Importing a .zip Library).


Sketch: BVS_Demo1.ino


#include <BVSP.h>
#include <BVSMic.h>


// Defines the Arduino pin that will be used to capture audio
#define BVSM_AUDIO_INPUT 5


// Defines the LED pins
#define RED_LED_PIN 6
#define YELLOW_LED_PIN 9
#define GREEN_LED_PIN 10


// Defines the constants that will be passed as parameters to
// the BVSP.begin function
const unsigned long STATUS_REQUEST_TIMEOUT = 1000;
const unsigned long STATUS_REQUEST_INTERVAL = 2000;


// Defines the size of the audio buffer
const int AUDIO_BUFFER_SIZE = 64;


// Defines the size of the receive buffer
const int RECEIVE_BUFFER_SIZE = 2;


// Initializes a new global instance of the BVSP class
BVSP bvsp = BVSP();


// Initializes a new global instance of the BVSMic class
BVSMic bvsm = BVSMic();


// Creates a buffer that will be used to read recorded samples
// from the BVSMic class
byte audioBuffer[AUDIO_BUFFER_SIZE];


// Creates a buffer that will be used to read the commands sent
// from BitVoicer Server.
// Byte 0 = pin number
// Byte 1 = pin value
byte receiveBuffer[RECEIVE_BUFFER_SIZE];


void setup()
{
  // Sets up the pin modes
  pinMode(RED_LED_PIN, OUTPUT);
  pinMode(YELLOW_LED_PIN, OUTPUT);
  pinMode(GREEN_LED_PIN, OUTPUT);

  // Starts serial communication at 115200 bps
  Serial.begin(115200);

  // Sets the Arduino serial port that will be used for
  // communication, how long it will take before a status request
  // times out and how often status requests should be sent to
  // BitVoicer Server.
  bvsp.begin(Serial, STATUS_REQUEST_TIMEOUT, STATUS_REQUEST_INTERVAL);

  // Defines the function that will handle the frameReceived
  // event
  bvsp.frameReceived = BVSP_frameReceived;

  // Prepares the BVSMic class timer
  bvsm.begin();
}


void loop()
{
  // Checks if the status request interval has elapsed and if it
  // has, sends a status request to BitVoicer Server
  bvsp.keepAlive();

  // Checks if there is data available at the serial port buffer
  // and processes its content according to the specifications
  // of the BitVoicer Server Protocol
  bvsp.receive();


  // Checks if there is one SRE available. If there is one,
  // starts recording.
  if (bvsp.isSREAvailable())
  {
    // If the BVSMic class is not recording, sets up the audio
    // input and starts recording
    if (!bvsm.isRecording)
    {
      bvsm.setAudioInput(BVSM_AUDIO_INPUT, EXTERNAL);
      bvsm.startRecording();
    }


    // Checks if the BVSMic class has available samples
    if (bvsm.available)
    {
      // Makes sure the inbound mode is STREAM_MODE before
      // transmitting the stream
      if (bvsp.inboundMode == FRAMED_MODE)
        bvsp.setInboundMode(STREAM_MODE);
  
      // Reads the audio samples from the BVSMic class
      int bytesRead = bvsm.read(audioBuffer, AUDIO_BUFFER_SIZE);

      // Sends the audio stream to BitVoicer Server
      bvsp.sendStream(audioBuffer, bytesRead);
    }
  }
  else
  {
    // No SRE is available. If the BVSMic class is recording,
    // stops it.
    if (bvsm.isRecording)
      bvsm.stopRecording();
  }
}


// Handles the frameReceived event
void BVSP_frameReceived(byte dataType, int payloadSize)
{
  // Checks if the received frame contains binary data
  // 0x07 = Binary data (byte array)
  if (dataType == DATA_TYPE_BINARY)
  {
    // If 2 bytes were received, process the command.
    if (bvsp.getReceivedBytes(receiveBuffer, RECEIVE_BUFFER_SIZE) ==
      RECEIVE_BUFFER_SIZE)
    {
      analogWrite(receiveBuffer[0], receiveBuffer[1]);
    }
  }
}





This sketch has four major parts:


  • Library references and variable declaration: The first two lines include references to the BVSP and BVSMic libraries. These libraries are provided by BitSophia and can be found in the BitVoicer Server installation folder. The other lines declare constants and variables used throughout the sketch. The BVSP class is used to communicate with BitVoicer Server and the BVSMic class is used to capture and store audio samples.
  • Setup function: This function initializes serial communication, the BVSP class, the BVSMic class and sets the “event handler” (it is actually a function pointer) for the frameReceived event.
  • Loop function: This function performs three important actions: requests status info to the server (keepAlive() function), checks if the server has sent any data and processes the received data (receive() function), and controls the recording and sending of audio streams (isSREAvailable(), startRecording(), stopRecording() and sendStream() functions).
  • BVSP_frameReceived function: This function is called every time the receive() function identifies that one complete frame has been received. Here I run the command sent from BitVoicer Server. The command contains 2 bytes. The first byte indicates the pin and the second byte indicates the pin value. I use the analogWrite() function to set the appropriate value to the pin.

 

STEP 3: Importing BitVoicer Server Solution Objects

 

Now you have to set up BitVoicer Server to work with the Arduino. BitVoicer Server has four major solution objects: Locations, Devices, BinaryData and Voice Schemas.

 

Locations represent the physical location where a device is installed. In my case, I created a location called Home.


Devices are the BitVoicer Server clients. I created a Mixed device, named it ArduinoMicro and entered the communication settings. NOTE ABOUT ARDUINO MICRO: it uses RTS and DTR so you have to enable these settings in the communication tab. I also created a SystemSpeaker device to synthesize speech using the server audio adapter.


BinaryData is a type of command BitVoicer Server can send to client devices. They are actually byte arrays you can link to commands. When BitVoicer Server recognizes speech related to that command, it sends the byte array to the target device. I created one BinaryData object to each pin value and named them ArduinoMicroGreenLedOn, ArduinoMicroGreenLedOff and so on. I ended up with 18 BinaryData objects in my solution, so I suggest you download and import the objects from the VoiceSchema.sof file below.


Voice Schemas are where everything comes together. They define what sentences should be recognized and what commands to run. For each sentence, you can define as many commands as you need and the order they will be executed. You can also define delays between commands. That is how I managed to perform the sequence of actions you see in the video.


You can import (Importing Solution Objects) all solution objects I used in this post from the files below. One contains the Devices and the other contains the Voice Schema and its Commands.


Files:



STEP 4: Conclusion


That is it! I hope you liked it.


You have everything you need to run the demo shown in the video. Note that in the video I started by enabling the ArduinoMicro device in the BitVoicer Server Manager. As soon as it gets enabled, the Arduino identifies an available Speech Recognition Engine and starts streaming audio to BitVoicer Server.


You can follow the recognition results in the Server Monitor tool available in the BitVoicer Server Manager.


In my next post I will show how you can reproduce synthesized speech using an Arduino DUE.




In this project I connect my Arduino Board to a WiFly module (https://www.sparkfun.com/products/10822) from Roving Networks and stream audio to the PC, so that BitVoicer (http://www.bitsophia.com/BitVoicer.aspx) can perform speech recognition. By using a wireless module, I managed to add speech recognition features to my microcontroller without the need of a physical connection with the PC.

 

The LEDs do the same basic things they did in my last project (http://www.element14.com/community/groups/arduino/blog/2012/12/27/speech-recognition-with-bitvoicer-arduino-and-a-microphone), but now I eliminated the PC wire connection, so I can finally have a speech activated Arduino anywhere my wireless network reaches.

 

Here is the YouTube video of the project:

 

 

I used a WiFly module from Roving Networks (https://www.sparkfun.com/products/10822) because I read somewhere that it would work with the XBee Shield I had (https://www.sparkfun.com/products/9976). Unfortunately, I found out the hard way that a diode between the Arduino TX and the WiFly RX was blocking the signal. I had to rip out the diode and add a voltage divider between the TX/RX pins of the Arduino and the WiFly module because the RX pin of the WiFly is 3.3V tolerant and the Arduino TX works at 5V. It does not seem to be the best approach, but it worked and I was able to send and receive data from the module. I found out that Sparkfun released a new version of the XBee Shield (https://www.sparkfun.com/products/10854) and that the diode level shifter was replaced with a more robust MOSFET level shifter. This should solve the diode problem.

 

This is how my project looked like after I added the voltage divider:

 

BitVoicerTest3_800x630.jpg

 

I believe any of the following wireless modules could be used. You just have to find out which one of them best fits your needs:

 

- http://uk.farnell.com/jsp/search/browse.jsp?N=2031+203906&Ntk=gensearch&Ntt=802.11&Ntx=mode+matchallpartial

-          http://arduino.cc/en/Main/ArduinoWiFiShield

-          http://www.rovingnetworks.com/products/RN171XV

-          https://www.sparkfun.com/pages/xbee_guide (you will need a XBee Explorer Dongle to connect the XBee to your computer: https://www.sparkfun.com/products/9819)

-          Many other Arduino WiFi Shields can be found here: http://postscapes.com/arduino-wifi

 

The setup process of the WiFly module can be tricky, so I’m also adding a step-by-step tutorial on how to do it. In my case, I use a regular access point/router from D-Link to implement my private wireless network. This tutorial should work with any access point out there.

 

1.Disconnect your computer from any wireless network
2.Set the GPIO9 pin of the WiFly module to high (3.3V) at power up. This enables adhoc mode on the module. I used the 3.3V source from the Arduino board and an ordinary jumper wire. Right after the power up, remove the wire and follow the next steps. Module datasheet: http://dlnmh9ip6v2uc.cloudfront.net/datasheets/Wireless/WiFi/WiFly-RN-XV-DS.pdf
3.On your computer, search for available networks and connect to the WiFly module network (WiFly-GSX-XX). It may take a few seconds before your computer gets an IP address from the module.
4.Download and run a free software called TeraTerm (http://www.rovingnetworks.com/resources/download/86/Teraterm)
5.Enter the following settings and click on OK: IP Address: 169.254.1.1; TCP port: 2000; Service: Telnet; Protocol: IPv4
6.You should see the word “*HELLO*” on the screen.
7.Type $$$ to enter command mode
8.Type scan and hit enter to scan for available access points
9.If your network uses WPA authentication, use the command set wlan phrase <string> to set the pass phrase. For WEP, set the key using the set wlan key <num> command. My network is WEP secured, so I had to use the second command. The problem is that the key must be entered ASCII/HEX encoded and it MUST have exactly 26 characters. I found this page that converts text to ASCII/HEX: http://www.string-functions.com/string-hex.aspx.
10.Type set wlan ssid <XXXX>, where XXXX is the SSID of you network (look at the results of step 8)
11.Type save
12.Type join <YOUR NETWORK> (look at the results of step 8)
13.The WiFly-GSX-XX will disappear from the windows network list
14.Close Teraterm
15.Reconnect your computer to your local wireless network
16.Turn your Arduino board OFF and back ON
17.Write down the IP number assigned to the WiFly module by your access point (usually, you can find this info on the access point DHCP page)
18.Open Teratem using the new IP address and the other settings from step 5
19.Type $$$ to enter command mode
20.Type set comm close 0 and hit enter
21.Type set comm open 0 and hit enter
22.Type set comm remote 0 and hit enter
23.Type set comm size 1460 and hit enter
24.Type set comm time 1000 and hit enter
25.Type set comm baud 115200 and hit enter (the same baud rate I used in my sketch)
26.Type save and hit enter
27.Type exit and hit enter
28.Close Teraterm

 

Now your WiFly module should be able to communicate with BitVoicer and your PC. The WiFly user’s manual can be downloaded here: http://dlnmh9ip6v2uc.cloudfront.net/datasheets/Wireless/WiFi/WiFly-RN-UM.pdf

 

Here are a couple of pictures of my project:

 

BitVoicerTest3_1_800x600.jpg

 

BitVoicerTest3_2_800x600.jpg

And this is the sketch I used to control the LEDs, capture audio and interact with BitVoicer:

 

#include <BitVoicer11.h>

//Instantiates the BitVoicerSerial class
BitVoicerSerial bvSerial = BitVoicerSerial();

//Stores true if the Audio Streaming Calibration tool
//is running
boolean sampleTest = false;
//Stores the data type retrieved by getData()
byte dataType = 0;
//Sets up the pins and default variables
int pinR = 3;
int pinY = 5;
int pinG = 6;
int lightLevel = 0;

void setup()
{
  //Sets the analog reference to external (AREF pin)
  //WARNING!!! If anything is conected to the AREF pin,
  //this function MUST be called first. Otherwise, it will
  //damage the board.
  bvSerial.setAnalogReference(BV_EXTERNAL);
  //Sets up the microcontroller to perform faster analog reads
  //on the specified pin
  bvSerial.setAudioInput(0);
  //Starts serial communication at 115200 bps
  Serial.begin(115200);
  //Sets up the pinModes
  pinMode(pinR, OUTPUT);
  pinMode(pinY, OUTPUT);
  pinMode(pinG, OUTPUT);
}

void loop()
{
  //Captures audio and sends it to BitVoicer if the Audio
  //Streaming Calibration Tool is running
  if (sampleTest == true)
  {
    //The value passed to the function is the time
    //(in microseconds) that the function has to wait before
    //performing the reading. It is used to achieve about
    //8000 readings per second.
    bvSerial.processAudio(46);
  }

  //Captures audio and sends it to BitVoicer if the Speech
  //Recognition Engine is running
  if (bvSerial.engineRunning)
  {
    //The value passed to the function is the time
    //(in microseconds) that the function has to wait before
    //performing the reading. It is used to achieve about
    //8000 readings per second.
    bvSerial.processAudio(46);
  }
}

//This function runs every time serial data is available
//in the serial buffer after a loop
void serialEvent()
{
  //Reads the serial buffer and stores the received data type
  dataType = bvSerial.getData();

  //Changes the value of sampleTest if the received data was
  //the start/stop sampling command
  if (dataType == BV_COMMAND)
      sampleTest = bvSerial.cmdData;

  //Signals BitVoicer's Speech Recognition Engine to start
  //listening to audio streams after the engineRunning status
  //was received
  if (dataType == BV_STATUS && bvSerial.engineRunning == true)
    bvSerial.startStopListening();

  //Checks if the data type is the same as the one in the
  //Voice Schema
  if (dataType == BV_STR)
    setLEDs();
}

//Performs the LED changes according to the value in
//bvSerial.strData
void setLEDs()
{
  if (bvSerial.strData == "wake")
  {
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    delay(200);
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    delay(200);
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    delay(200);
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    delay(200);
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    delay(200);
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "sleep")
  {
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    delay(200);
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    delay(200);
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    delay(200);
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "RH")
  {
    digitalWrite(pinR, HIGH);
    lightLevel = 255;
  }
  else if (bvSerial.strData == "RL")
  {
    digitalWrite(pinR, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "YH")
  {
    digitalWrite(pinY, HIGH);
    lightLevel = 255;
  }
  else if (bvSerial.strData == "YL")
  {
    digitalWrite(pinY, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "GH")
  {
    digitalWrite(pinG, HIGH);
    lightLevel = 255;
  }
  else if (bvSerial.strData == "GL")
  {
    digitalWrite(pinG, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "ALLON")
  {
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    lightLevel = 255;
  }
  else if (bvSerial.strData == "ALLOFF")
  {
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "brighter")
  {
    if (lightLevel < 255)
    {
      lightLevel += 85;
      analogWrite(pinR, lightLevel);
      analogWrite(pinY, lightLevel);
      analogWrite(pinG, lightLevel);
    }
  }
  else if (bvSerial.strData == "darker")
  {
    if (lightLevel > 0)
    {
      lightLevel -= 85;
      analogWrite(pinR, lightLevel);
      analogWrite(pinY, lightLevel);
      analogWrite(pinG, lightLevel);
    }
  }
  else
  {
    bvSerial.startStopListening();
    bvSerial.sendToBV("ERROR:" + bvSerial.strData);
    bvSerial.startStopListening();
  }
}

 

The BitVocier Voice Schema I used can be downloaded here: http://www.justbuss.xpg.com.br/BitVoicerTest3.zip

 

If you have any question about this project, please post it here, so anyone else can read the answer.

In this project I use BitVoicer’s speech recognition features (http://www.bitsophia.com/BitVoicer.aspx), one Arduino board and one electret microphone (https://www.sparkfun.com/products/9964) to control a few LEDs.

 

The following fritzing schematic shows how I wired my Arduino board:

 

BitVoicerTest2_Fritz_800x600.jpg

 

I’m also adding a few pictures and a YouTube video of the project running.

 

 

BitVoicerTest2_1_800x600.jpg

 

BitVoicerTest2_2_800x600.jpg

Unfortunately, the sketch I wrote for my first post is no longer supported by the new version of BitVoicer. Here is the new sketch:

 

#include <BitVoicer11.h>


//Instantiates the BitVoicerSerial class
BitVoicerSerial bvSerial = BitVoicerSerial();


//Stores true if the Audio Streaming Calibration tool
//is running
boolean sampleTest = false;
//Stores the data type retrieved by getData()
byte dataType = 0;
//Sets up the pins and default variables
int pinR = 3;
int pinY = 5;
int pinG = 6;
int lightLevel = 0;


void setup()
{
  //Sets the analog reference to external (AREF pin)
  //WARNING!!! If anything is conected to the AREF pin,
  //this function MUST be called first. Otherwise, it will
  //damage the board.
  bvSerial.setAnalogReference(BV_EXTERNAL);
  //Sets up the microcontroller to perform faster analog reads
  //on the specified pin
  bvSerial.setAudioInput(0);
  //Starts serial communication at 115200 bps
  Serial.begin(115200);
  //Sets up the pinModes
  pinMode(pinR, OUTPUT);
  pinMode(pinY, OUTPUT);
  pinMode(pinG, OUTPUT);
}


void loop()
{
  //Captures audio and sends it to BitVoicer if the Audio
  //Streaming Calibration Tool is running
  if (sampleTest == true)
  {
    //The value passed to the function is the time
    //(in microseconds) that the function has to wait before
    //performing the reading. It is used to achieve about
    //8000 readings per second.
    bvSerial.processAudio(46);
  }

  //Captures audio and sends it to BitVoicer if the Speech
  //Recognition Engine is running
  if (bvSerial.engineRunning)
  {
    //The value passed to the function is the time
    //(in microseconds) that the function has to wait before
    //performing the reading. It is used to achieve about
    //8000 readings per second.
    bvSerial.processAudio(46);
  }
}


//This function runs every time serial data is available
//in the serial buffer after a loop
void serialEvent()
{
  //Reads the serial buffer and stores the received data type
  dataType = bvSerial.getData();

  //Changes the value of sampleTest if the received data was
  //the start/stop sampling command
  if (dataType == BV_COMMAND)
      sampleTest = bvSerial.cmdData;

  //Signals BitVoicer's Speech Recognition Engine to start
  //listening to audio streams after the engineRunning status
  //was received
  if (dataType == BV_STATUS && bvSerial.engineRunning == true)
    bvSerial.startStopListening();

  //Checks if the data type is the same as the one in the
  //Voice Schema
  if (dataType == BV_STR)
    setLEDs();
}


//Performs the LED changes according to the value in
//bvSerial.strData
void setLEDs()
{
  if (bvSerial.strData == "wake")
  {
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    delay(200);
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    delay(200);
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    delay(200);
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    delay(200);
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    delay(200);
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "sleep")
  {
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    delay(200);
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    delay(200);
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    delay(200);
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "RH")
  {
    digitalWrite(pinR, HIGH);
    lightLevel = 255;
  }
  else if (bvSerial.strData == "RL")
  {
    digitalWrite(pinR, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "YH")
  {
    digitalWrite(pinY, HIGH);
    lightLevel = 255;
  }
  else if (bvSerial.strData == "YL")
  {
    digitalWrite(pinY, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "GH")
  {
    digitalWrite(pinG, HIGH);
    lightLevel = 255;
  }
  else if (bvSerial.strData == "GL")
  {
    digitalWrite(pinG, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "ALLON")
  {
    digitalWrite(pinR, HIGH);
    digitalWrite(pinY, HIGH);
    digitalWrite(pinG, HIGH);
    lightLevel = 255;
  }
  else if (bvSerial.strData == "ALLOFF")
  {
    digitalWrite(pinR, LOW);
    digitalWrite(pinY, LOW);
    digitalWrite(pinG, LOW);
    lightLevel = 0;
  }
  else if (bvSerial.strData == "brighter")
  {
    if (lightLevel < 255)
    {
      lightLevel += 85;
      analogWrite(pinR, lightLevel);
      analogWrite(pinY, lightLevel);
      analogWrite(pinG, lightLevel);
    }
  }
  else if (bvSerial.strData == "darker")
  {
    if (lightLevel > 0)
    {
      lightLevel -= 85;
      analogWrite(pinR, lightLevel);
      analogWrite(pinY, lightLevel);
      analogWrite(pinG, lightLevel);
    }
  }
  else
  {
    bvSerial.startStopListening();
    bvSerial.sendToBV("ERROR:" + bvSerial.strData);
    bvSerial.startStopListening();
  }
}

 

The BitVoicer Voice Schema I used can be downloaded from: http://www.justbuss.xpg.com.br/BitVoicerTest2.zip (you need to have BitVoicer installed to be able open it).

 

 

Now I want to control a few things (lights and ceiling fan) at home. Suggestions on how to connect them to the Arduino are very welcome.

Filter Blog

By date: By tag: