Welcome back for another Design Challenge Project Summary. For those of you who are new to my content, in this series I will pick a single project from the past week’s updates, and write a short summary of the project to date. Over the course of each challenge, I try to revisit each project at least once, and I am sure that some project summaries will get more than one update if they themselves are updated frequently. Some project creators like to keep their own project summary going, and this series is not meant to overshadow those post, but to highlight each project from an outsider's perspective.
The subject of this week’s installment is project Vintage Toy Synthesizer by Liam Lacey (liamtmlacey). The core idea behind this project is to turn a vintage wooden toy piano into a standalone digital hardware synthesiser, keeping the existing key mechanism and enclosure. Liam has set forth the following goals that will need to be completed for this project to be considered a success.
- Develop a digital synthesis engine that can generate and modify synthesized sounds.
- Modify the existing key mechanism so that it can trigger sounds and control velocity modulation on the synthesis engine.
- Add a set of potentiometers, switches and other controls to the existing piano enclosure for controlling the synthesis engine.
- Add an audio output for connecting the instrument to external speakers and recording equipment.
- Add MIDI I/O so that the instrument can be integrated into existing music production systems.
- Keep the existing aesthetic character of the vintage toy piano.
While this sounds like a tall order, Liam is an old-hat at digitizing old keyboards, and has in the past, completed a similar project using Piezo elements and an . In the video below, you can see a demo of this keyboard in action!
To get project Vintage Toy Synthesizer started on the right path, Liam decided to jump straight in and begin working out how he would modify the existing key mechanism to turn them into digital switches. At first he thought of using the same piezo elements that he had used in the past, but decided to take another route that involved using Velostat, a conductive plastic membrane, to handle the switch-like digital input when a key is pressed. This will allow the keys to function in a similar fashion like most other velocity-sensitive MIDI keyboards. The basic flow for this is listed below.
- When a key is initially pressed it opens a switch/circuit that starts a timer
- When the key is fully pressed it closes a second switch/circuit that stops the timer. The time value is used to generate a velocity value, and a note-on message is sent.
- When the key is released it closes the first switch/circuit, generating a note-off message.
With a plan of action figured out, Liam set to work building the Velostat-based key strikers. Holes were drilled, copper tape was put into place, and the Velostat was installed. With the key mechanical and electrical elements in-place, the wiring was tackled, and Liam was able to get a prototype up and working in no time.
As you can see in the video below, the use of Velostat to digitize the keys works out very well, and it is actually very responsive. It is important to note that Liam used an Arduino to get this prototype up and running quickly, and it looks like he will stick with it for the project. Liam does note that some of the keys are less sensitive than he would like, and he does plan on improving this by re-soldering some of the wiring contact points.
Over the next week or so, Liam was able to fine-tune the keyboards mechanical and electrical assembly, and that resulted in a much more responsive key press across all of the keys. Instead of utilizing off-the-shelf pressure sensors, the project uses bare Velostat sheets due to their flexibility, cost, and ability to modify them as needed.
The project is required to utilize a , and Liam has not forgotten this. His decision to use an Arduino to handle the initial MIDI input from the keyboard is due to two reasons, the first being the ability to split task and share the load. This makes it easier to assign the Beaglebone Black as the main sound synthesis engine, freeing up resources to ensure that timings are spot on. The second reason is to make the project more modular. By sending data from the Arduino to the Beaglebone Black via the boards serial ports, it frees up pins on the BeagleBone Black. This makes it easier to switch in other MIDI devices in the future.
As mentioned above, all the processing of key interaction is handled using an Arduino micro-controller, so the only software required for the keyboard mechanical assembly is a single Arduino sketch that needs to handle four main things - processing key presses/releases, processing key velocity, processing key pressure, and sending these three attributes to the BBB as serial-based messages. You can view this code in Liam’s Git Hub Repository for this project.
In the video above you can see a demo of the new keyboard mechanical and electrical assembly being demonstrated using the newly written code for the project. In this demo Liam has connected an to an (through serial) which has been modified with HIDUINO so that it can be recognized as a USB-MIDI controller, sending MIDI messages to Logic Pro.
Liam spent the next few days getting the Beaglebone Black up and running. This proved to be a little more complicated than expected, but in the end the Beaglebone Black is able to receive serial communications from the Arduino that is connected to the keyboard assembly. In this post, Liam walks the readers through using a separate development environment and cross-compilers to get the code up and running on the .
After getting the Beaglebone Black setup, Liam got started on getting the sound synthesis engine development moving forward. Unfortunately a few roadblocks were encountered along the way, some of which have forced some re-design of the project. Initially Liam had planned on building his own DAC for the project, but he finally settled on an off-the-shelf USB DAC. A couple of USB audio adapters were selected, with the winner being the EC Technology USB Audio Adapter which Liam purchased from Amazon.com.
Since the project’s inception, Liam has been planning on using the C++ audio synthesis / DSP library, Maximilian, to develop the synthesis engine, but while cross compiling his code, a few issues arose. The code for Maximilian was not able to be compiled on Liam’s Mac using the cross-compiler he had planned on using, and he was forced to compile it on the BeagleBone Black itself. This plan was also met with some issues involving variables being mistaken as static instead of dynamic. Liam was able to overcome this small issue by removing the initialization of these variables from the code. Once that was done, everything compiled as it should and Maximilian worked somewhat like it should.
As with many projects that utilize common Linux packages that were developed on much more resource heavy machines, Maximilian proved to not be very efficient running on a single-board computer like the BeagleBone Black. Liam ended the project's fourth blog post by saying that he might have to re-write the sound engine for Maximilian into something much more simple, or he might have to use a different audio library all together.
Update: February 16, 2016
While workarounds for the issues with Maximilian stewed in Liam’s head, he began to work on his next update to the project. In what can only be described as a phenomenal update, Liam educated us on the different types of synthesis, and what the core components of a synthesis engine are. With the terminology laid out, he then went on to describe in great detail exactly how he plans on designing and developing the synthesis engine for this project. LIam’s synthesis engine will be based on subtractive synthesis due to this process being less resource heavy, its ease of use, and his past experience.
With the synthesis engine planning out of the way, Liam decided to move onto the MIDI portion of the project. (Post 1, Post 2) While there are several connectors that are able to pass MIDI data, the common five-pin DIN connector was chosen for a few reasons including: It’s the most common found in commercial synthesis hardware, its easy to connect to the Beaglebone Black’s UART pins, and Liam has experience with them from past projects.
In this same post, Liam began working on the MIDI-In and MIDI-Out circuitry. An was used to isolate the MIDI hardware from the MCU hardware to protect the system from ground loops, and voltage spikes. Using an Arduino for testing, a prototype was quickly set up, and as you can see from the above and below images, the circuitry is quite simple.
In this two-post series, Liam also covers how to connect this circuit to the Beaglebone Black, getting the Beaglebone Black and Arduino communicating via serial, and how to handle the MIDI processing on the Beaglebone Black. This pair of post are two of the most comprehensive tutorials on MIDI data acquisition and processing that I have ever read, and while that is subjective, I can not thank Liam enough for such informative and educational post. This whole project is a treasure trove of information on MIDI, Synthesis Engines, and how it all ties together with the Beaglebone Black.
With a significant portion of the MIDI hardware figured out, Liam moved back into the software side of the project. He began developing a general structure of software on the BeagleBone Black that allows the keyboard, MIDI interface, and eventually the panel to communicate with the sound engine. This post covers how he will interface the keyboard, and control panel, and the two pieces of software that liam has began writing to connect everything together called “vintageBrain” and “vintageSoundEngine.”
“The vintageBrain application handles the task of allowing all the separate parts of the synthesiser to communicate. It is a single-threaded application that listens for messages coming from the keyboard, MIDI, and panel serial ports, and sends the messages to the sound engine and possible back to the MIDI serial port. It also handles all the voice and keyboard settings of the synthesiser,” Liam said. “VintageSoundEngine is the more interesting application of the two, as this is where the sound synthesis engine has been developed. It is a multithreaded application where the main thread is responsible for processing any MIDI messages coming from vintageBrain via the datagram socket which are used to trigger and control the sound, however the second thread is used to handle audio streaming and processing.”
Update: March 3, 2016
In the weeks that have passed since we last updated this project summary, Liam has made some significant progress with this project. The biggest leap forward is the now working synthesys engine. In a project update titled, Audio Synthesis Engine Implementation, Liam walks us step by step through the synthesis engine and how it works. Source code is provided to show how he handles processing notes and control messages, as well as how he is mixing the voice inputs.
As you can see, Liam is very thorough in explaining how every part of the synthesis engine works, and to top it all of its easily understandable by someone who has never jumped into the world of synthetic music. This post may be the longest I have seen in any of the design challenges here at Element14, and much like his other post, this one is definitely a must read.
Following the epic synthesis engine post, Liam got to work soldering together a Beaglebone protocape from Adafruit so that he can have more stable and secure connections between his power, ground, and hardware connections. Using some jumper wires and screw terminals, he was able to quickly solder up a purpose built cape that should have no issues with connectivity even when the device is being transported. Incase you are curious, I have pasted the connections that will be made with this cape below.
- Three pairs for connecting my keyboard, panel, and MIDI interface circuits to the BBB via the UART serial pins (both TX and RX for each circuit).
- Two pairs for providing 3.3V power to my three circuits (leaving one terminal currently unused)
- Two pairs for connecting the GND of my circuits to the BBB (leaving one terminal currently unused)
- A spare pair, just incase.
In the next post, the project took another huge leap forward, with Liam designing and beginning to build the synthesizer panel that will sit on top of the old toy keyboard. He says that his original plan was to utilize vintage / old-looking controls, but budget being an issue, this was just not possible. Liam settled on an aluminum knob that featured black and silver aesthetics. This is the far right knob in the image above.
With the knobs and switches picked out, Liam moved on to designing the layout for the actual panel itself. Using a CAD program, the layout came together quickly, and it looks pretty cool. As you can see, control types are grouped together, and then placed into a layout that matches the toy piano’s body. Topping it off, every group is labeled, with the components in each group also being labeled individually.
With the design fully laid out, Liam made the decision to have the panel laser cut in black acrylic instead of wood. This was due to the fact that he did not need to paint the final product, the laser would be able to easily etch the labels into the acrylic, and it would not warp down the road. As you can see in the image above, the panel is not yet finished, but it’s sure looking sharp!
Update: April 12, 2016
In his next update, Liam focused on getting some key connectivity components installed into the piano’s housing. A pair of 5-pin DIN MIDI sockets were flush-mounted to the back panel, and as explained in the post, the decision to use these sockets instead of a more common jack was mainly due to the panel’s thickness. Also installed was a 6.3mm stereo jack, a DC power jack, and a toggle switch. With that task finished, Liam got to work connecting these new connectors to the internal hardware. This included connecting the audio jack to a 3.5mm connector so that it could be plugged into the EC Technology USB audio adapter.
With the back panel now complete, Liam was free to focus on getting the rest of the internal electronics wired up. As you can see in the image above, the choice was made to use three 16-channel multiplexers instead of 8-channel units due to a misconception that six 8-channel multiplexers would not work with the Arduino Pro Mini that is being used to convert the analog signals from the pots to serial data. This also greatly cleaned up the internal electronics layout, and made the project much easier to solder together using stripboard.
After what had to be hours of intense soldering, the back of the panel was fully wired up, and all of the potentiometers connected to the multiplexers. I have done similar wiring before with a custom RGB LED display I hacked together, and I can only imaging the amount of concentration it took to keep track of every wire, and to make sure that everything was in the correct order. This is a truly impressing wiring job! As you can see from the second photo above, the end result was definitely worth it!
Next on Liam’s list was about finishing up the Audio Synthesis Engine, and in an update posted on March 20th, he laid out exactly how he would be handling the voice mode, voice allocation, keyboard, volume, and “vintage” parameters.
The Voice Mode and Voice Allocation parameters are broken down into two different modes, Polyphonic, and Monophonic. Poly mode is implemented using an array that stores an ordered-list of 'free' voices - voices that are not currently playing a note. The number at the beginning of the list always represents the next available free voice.
When in mono mode the system stores a 'stack' of notes that represent all the keys that are currently being held down, so that if a key is released whilst there are still keys being held down the played note is changed to the previously played key rather than just turning the note off.
The keyboard parameters are broken down into sections as well:
- Scale - Controls what particular musical scale is played by the keyboard, in this poject’s case, there are eight scales that can be played.
- Octave - Octave controls the musical octave that the keyboard scale is offset by, where an octave value of 0 sets the bottom key on the keyboard to play middle E (MIDI note 64), with greater octave value adding 12 semitones each time, or lower octave values reducing the notes by 12 semitones each time:
- Transpose - Transpose controls a singular semitone offset applied to the note number, and allows the bottom key on the keyboard to be any musical note rather than just E:
The post continues on to layout a few more parameters, and as with all of his updates, Liam has included the source code to show exactly how he is achieving each part of the puzzle. If you are working on building your own synthesis engine, you definitely have to check out this post.
With the challenge quickly coming to an end, Liam used his next post to document the build process of the keyboard. Not only did he clean things up, but he added all of the knobs, and even built a custom “stand” to keep the synthesizer’s lid open to showcase the internal electronics. This had an added benefit of keeping the “miniature grand piano” feel that the project started out with. This project may be one of the most refined, and aesthetically pleasing projects we have ever seen posted to a design challenge. The decision to go with the acrylic panel over wood was definitely the right one!
In his second to final post, Liam showcased a patch manager application that he wrote in visual basic that allows users to save the sound parameter settings into a 'patch' so that a particular sound can be quickly recalled at a later time. While this is usually handled on the synth itself, Liam decided to write this as an external desktop application to save time, and further complexity of the project.
In this same post, Liam gave us a six minute demo of the music that can be produced, and I for one was blown away at the diverse soundstage that this little synth can produce. They say that the proof is in the pudding, and this video shows that to be true. Liam just posted his final post, and I am just going to leave it here for you to check out on your own. This has been an amazing project, and its documentation is truly second to none. So head over to the link above to find out what the finished project looks like. Great job Liam!
That is going to wrap my summary coverage of project: Vintage Toy Synthesiser. This was such a wonderful project from beginning to end, and I feel that Liam created exactly what he originally set out to create. I want to personally thank Liam for his commitment to finishing the challenge, and sharing such an awesome project with all of us. Tune in next week for another Design Challenge Project Summary here at Element14!