Skip navigation

This guide provides step-by-step instructions for controlling your MATRIX Creator with a discord bot. By the end, you’ll be able to control the Creator’s LEDs, microphones, and ability to run any command you create.

 

Required Hardware

Before you get started, let's review what you'll need.

  • Raspberry Pi 3 (Recommended) or Pi 2 Model B (Supported) - Buy on Element14 - Pi 3 or Pi 2.
  • MATRIX Creator - The Raspberry Pi does not have a built-in microphone, the MATRIX Creator has an 8 mic array perfect for Alexa - Buy MATRIX Creator on Element14.
  • Micro-USB power supply for Raspberry Pi - 2.5A 5V power supply recommended
  • Micro SD Card (Minimum 8 GB) - You need an operating system to get started. NOOBS (New Out of the Box Software) is an easy-to-use operating system install manager for Raspberry Pi. The simplest way to get NOOBS is to buy an SD card with NOOBS pre-installed - Raspberry Pi 16GB Preloaded (NOOBS) Micro SD Card. Alternatively, you can download and install it on your SD card.
  • A USB Keyboard & Mouse, and an external HDMI Monitor - we also recommend having a USB keyboard and mouse as well as an HDMI monitor handy if you're unable to remote(SSH)into your Pi.
  • Internet connection (Ethernet or WiFi)
  • (Optional) WiFi Wireless Adapter for Pi 2 (Buy on Element14). Note: Pi 3 has built-in WiFi.

For extra credit, enable remote(SSH) into your device, eliminating the need for a monitor, keyboard, and mouse - and learn how to tail logs for troubleshooting.

 

Let's get started

We will be using MATRIX Open System (MOS), to easily program the Raspberry Pi and MATRIX Creator in Javascript. The Discord bot will be made using the discord.js module.

 

Step 1: Setting up MOS

Download and configure MOS and its CLI tool for your computer using the following installation guide in the MATRIX Docs: Installation Guide.

 

Step 2: Installing Raspberry Pi Dependencies

In order to use the MATRIX Creator’s mics and stream audio to Discord, the following dependencies must be installed to your Pi for these functionalities.

  • Discord Audio Streaming
    • Libav: sudo apt-get install libav-tools
  • MATRIX Creator Mics
    • ALSA Tools: sudo apt-get install alsa-base alsa-utils

 

Step 3: Creating Your Discord Bot

Discord App Client key and bot creation

  • Save Bot Token

 

Step 4: Creating Your MOS App

To create your own MATRIX Creator Discord app on your local computer, use the command "matrix create Discord-Bot". You will then be directed to enter a description and keywords for your app. A new folder will be created for the app, along with five files inside. The one you will be editing is the app.js file. From here, you can clone the MATRIX-Discord-Bot GitHub repository with the code or follow the guide below for an overview of the code.

 

Step 5: Dependencies & Global Vars

The bot variable is a discord.js client object which allows us to read and respond with the Discord API. To login, the bot will require your Discord app’s bot token. Mic is then defined to grab microphone input from the MATRIX Creator. This will be used to stream that input into a Discord voice channel. To keep track of what voice channels the bot joins, currentVoiceChannel is defined to later hold that information.

var Discord = require('discord.js');//https://discord.js.org
var bot = new Discord.Client();//Discord Bot Object
var token = 'YOUR_BOT_TOKEN_HERE';//Discord Bot Token
var mic = require('mic');//Stream wrapper for arecord
var currentVoiceChannel;//bot's current voice channel

 

Step 6: Creating Discord Commands

To organize commands, each command you make will need to be assigned to a command group. These groups will hold the commands you want in their list array. The addGroupCommand function is where you make and assign a command to a group. For example, a hello command assigned to the 'matrix' group is now able to be used by typing '/matrix hello' in the Discord chat. A callback you define will run once the command is called.

//////////////////////////////////////////////////////////
// Discord Commands
//////////////////////////////////////////////////////////
var commandGroups = {
    'matrix': {command: '/matrix', list:[]},//matrix commands
    'basic' : {command: '/basic', list:[]}//basic chat commands
};


//Command Creator
function addGroupCommand(group, commandName, description, command) {
    //add command to command group
    commandGroups[group].list.push({
        commandName : commandName,//name
        description: description,//desc
        command: command//function to run
    });
}

 

Step 7: Running A Command

The commandSearch function will use the inserted command group and Discord Message to see if any commands in that group were called. If found, the command will run. If not, the user is sent a message with all group commands. The function will return false for the latter.

//Look For & Use Command In Command Group
function commandSearch(group, message){
    var userArgs = message.content.split(' ');//Convert User Arguments Into An Array
    var commandFound = false;//bool on sending help menu
    var commandHelp = commandGroups[group].command+' Commands:\n';//will hold the command group's commands


    //check if command group was called
    if (userArgs[0] === commandGroups[group].command) {
        //Search For Command In Group
        for(i = 0; i < commandGroups[group].list.length; i++){
            //save command and description to help string
            commandHelp += '\n' + commandGroups[group].list[i].commandName +' - '+ commandGroups[group].list[i].description;
            //If command is found
            if( userArgs[1] === commandGroups[group].list[i].commandName){
                //Use command
                commandGroups[group].list[i].command(userArgs, message);
                //Update commandFound
                commandFound = true;
            }
        }
        //If Command Not Found
        if(!commandFound){
            message.reply('```'+commandHelp+'```');//reply with command list
        }
        //command group was found
        return true;
    }
    //command group was not called
    else
        return false;
}

 

Step 8: MATRIX Led Command

To change the MATRIX Creator’s LEDs, the function looks for an input after ‘/matrix led’. The input (LED color) is then inserted into the matrix.led command. A proper usage reply will be sent to the user if they don’t have a parameter in their message.

//////////////////////////////////////////
// MATRIX Command Group
// - Change MATRIX LEDs
addGroupCommand('matrix', 'led', 'Change Color of MATRIX LEDs', function(userArgs, message){
    //Look For Color Input
    if (userArgs.length === 3){
        message.reply('```Using: matrix.led(\'' +userArgs[2]+ '\').render()```');
        console.log(userArgs[2]);
        matrix.led(userArgs[2]).render();//change colors
    }
    //Command Had No/Bad Input
    else{
        //reply command usage
        message.reply('```\nCommand Usage:\n\t'+
        '/matrix led purple'+'        //color name\n\t'+
        '/matrix led rgb(255,0,255)'+'//rgb values\n\t'+
        '/matrix led #800080'+'       //css color'+
        '```');
    }
});

 

Step 9: MATRIX Join Command

The join command is used to stream the MATRIX Creator microphones into a voice channel. The command requires no parameters and will auto join the user’s current channel, the channel is saved in the currentVoiceChannel variable. The audio will have about a 6-second delay during the initial audio stream, but it will shorten as time passes.

// - Listen To MATRIX Mics
addGroupCommand('matrix', 'join', 'MATRIX Joins Your Voice Channel', function(userArgs, message){
    //continue if no args are present
    if(userArgs.length === 2){
        message.reply('Joining Voice Channel');
        //User Must Be In Voice Channel
        if (message.member.voiceChannel) {
            //just move if in voice channel
            if(currentVoiceChannel !== undefined){
                message.member.voiceChannel.join();//join voice channel
                currentVoiceChannel = message.member.voiceChannel;//save joined channel id
            }
            //join and reinitialize mics
            else{
                //join voice channel
                message.member.voiceChannel.join().then(connection => {
                    //save joined channel id
                    currentVoiceChannel = message.member.voiceChannel;
                    //npm mic config
                    var micInstance = mic({
                        rate: 16000,
                        channels: '1',
                        debug: false,
                        exitOnSilence: 0,
                        device : 'mic_channel8'
                    });
                    micInputStream = micInstance.getAudioStream();//mic audio stream
                    //when mics are ready
                    micInputStream.on('startComplete', function(){
                        var dispatcher;//will serve audio
                        dispatcher = connection.playArbitraryInput(micInputStream);//stream mics to Discord
                        console.log('mics ready');
                    });
                    //start mics
                    micInstance.start();
                });
            }
        }
        //User Is Not In Voice Channel
        else{
            message.reply('You need to join a Voice channel first!');
            return;
        }
    }
    //Tell user to use no args
    else
        message.reply('```"/matrix join" has no parameters```');
});

 

Step 10: MATRIX Leave Command

This command will tell the bot to leave the channel saved in currentVoiceChannel. There’s also another command, at the bottom, for getting the link to the MATRIX documentation.

// - MATRIX Leaves Voice Channel
addGroupCommand('matrix', 'leave', 'MATRIX Leaves Current Voice Channel', function(userArgs, message){
    //continue if no args are present
    if(userArgs.length === 2){
        //leave current voice channel
        if(currentVoiceChannel !== undefined){
            message.reply('Leaving Voice Channel');
            currentVoiceChannel.leave();            
            //remove saved voice channel id
            currentVoiceChannel = undefined;
        }
        else
            message.reply('Currently not in a voice channel!');
    }
    //Tell user to use no args
    else
        message.reply('```"/matrix leave" has no parameters```'); 
});
// - MATRIX Documentation Link
addGroupCommand('matrix', 'docs', 'Link To MATRIX Documentation', function(userArgs, message){
    message.reply('https://matrix-io.github.io/matrix-documentation/');
});

 

Step 11: Basic Ping Command

This is a simple ping command to show you how easy it is to create and organize new commands. The command itself will simply reply ‘pong’ to any user that types ‘/basic ping’

//////////////////////////////////////////
// BASIC Command Group
// - A Simple Ping
addGroupCommand('basic', 'ping', 'Reply To User Ping', function(userArgs, message){
    message.reply('pong');
});

 

Step 12: Discord Message Event

This message event is fired whenever a message, that the bot can read, appears. Private messages the bot receives are set to be ignored. Any other message will be used in a for loop that runs the commandSearch function. This loop will compare the message sent with each existing command group, running the command that matches.

//////////////////////////////////////////////////////////
// Discord Events
//////////////////////////////////////////////////////////
//On Discord Message
bot.on('message', function(message){
    //Accept Text Channel & User Messages Only
    if (!message.guild && bot.user.id !== message.author.id){
        message.reply('You need to join a Text channel first!');
        return;
    }


    //Check If User Message
    if (bot.user.id !== message.author.id){
        //Loop through commandGroup groups
        for (var group in commandGroups) {
            //Search for and run command
            if (commandGroups.hasOwnProperty(group) && commandSearch(group, message))
                break;//leave loop
        }
    }
});

 

Step 13: Logging In

The previously defined token is used for allowing the newly made Discord bot to login.

//On Discord Bot Login
bot.on('ready', function(){
    console.log('ready');
});


//Start Discord Bot
bot.login(token);

 

Step 14: package.json

Before deploying to your MATRIX Creator, update your package.json file to have these dependencies. MOS will auto install everything when it installs your app.

{
  "name": "discordBot",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "discord.js": "^11.2.1",
    "mic": "^2.1.2",
    "node-opus": "^0.2.7"
  }
}

 

Github: The entire repository for this guide can be found here:

Matt Reed from RedPepper has used a Raspberry Pi, Microphone, a Creepy Doll and  Google’s Speech Neural Network system to listen into .... Ghosts.

 

 

 

"From October 27–31, we’ll be live streaming the DeepWhisper rig nightly from our offices in historic “Butchertown” Nashville so you can watch for any EVPs that may come through. Just the thing to do at 3am when you can’t sleep." - Matt Reed

 

The DeepWhisper Project  pipes a real-time microphone stream to Google’s Speech Neural Network, which can detect over 110 languages and then we’llimmediately display the results as they come back

 

www.DeepWhisper.io

 

 

 

 

 

 

 

Deep Whisper is Opensource so anyone can hunt their own ghosts.

it runs on Node and its libraries have been optimized for Raspberry Pis. You’ll need a USB microphone, Google Cloud Platform Project Key, a display, and patience.

Matt will upload a full repository link soon but, for now, here are the key code snippets.

 

Connecting to Google Voice Neural Network

You’ll need to have a project set up in the Google Cloud Platform console which will grant you an authentication JSON key that your app will use to connect. Just follow these steps to get that going. Note: you may have to set up billing with Google to proceed.

 

// Authenticate with Google Cloud
const speech = require('@google-cloud/speech')({
  projectId: 'deepwhisper-XXXXXXXX',
  keyFilename: 'Deepwhisper-XXXXXXX.json'
});

 

Streaming mic input to Google

Simply pipe the microphone input to Google, and if the Neural Network detects speech in any of the 110+ supported languages, it will be returned as a string of transcript text.

 

// Connect and listen to USB microphone
const micInstance = mic({
  rate: ‘16000’,
  channels: ‘1’,
  debug: true,
  exitOnSilence: 0
});
const micInputStream = micInstance.getAudioStream();
micInputStream.pipe(recognizeStream);
// Create a real-time recognize stream with Google
const recognizeStream = speech.streamingRecognize(request)
  .on(‘error’, console.error)
  .on(‘data’, (data) =>
  (data.results[0] && data.results[0].alternatives[0])
    ? io.emit('text', { transcript: data.results[0].alternatives[0].transcript })
    : `\nReached transcription time limit, press Ctrl+C\n`);

 

Display the results

Using a simple HTML page with Socket.IO, you can receive the results emitted by the server above and display them immediately. This uses jQuery to set the text and fade it out after five seconds.

 

<script>
  var socket = io.connect('http://localhost:3000');
  socket.on('text', function (data) {
  $('#transcript').text(data.transcript);
  $('#transcript').fadeIn();
  setTimeout(function(){
  $('#transcript').fadeOut(function(){
  $('#transcript').text('');
  });
  }, 5000);
 });
</script>

Source and Project Matt Reed at RedPepper.land

www.DeepWhisper.io

Filter Blog

By date: By tag: