You’ve seen those job listings that are looking for a “JavaScript Rock Star” and likely rolled your eyes. It seems silly to think that whatever skill you have at programming is going to get you featured in Rolling Stone. Besides, code on a screen is not likely to land you the legions of screaming fans that a sick guitar solo will.
Unless you’re Sam Aaron, that is. He’s the creator of Sonic Pi, a live coding language for making music. It’s an open source project that he built which creates music from code in real time. The world of music is taking notice: Rolling Stone described him as “transcending the present” when he performed at the same festival as Grimes.
Sonic Pi works on simple loops of code. You enter a few commands that indicate beats, melodies, and samples to play. These commands are processed through a synthesizer backend called SuperCollider. The essence of it is loop and repeat, but if you change the code during the performance, the music changes too. It’s a simple and powerful way to get anyone up and running with electronic music or basic programming concepts.
“If you're a programmer who knows nothing about music then Sonic Pi can help you with that,” said Aaron. “If you're a musician, who knows nothing about programming, Sonic Pi can help you with that. And if you know nothing about either, you’ve got a fun and exciting journey ahead of you.”
On its face, it’s gloriously simple. So simple, that a 10-year-old child can use it. Under the hood, there are some surprising technical challenges to get a language like this to work.
Beginnings
Before Sonic Pi, Aaron and his friend Jeff Rose had created Overtone, another live coding language built in Clojure. This project grew out his interest in domain-specific languages, which was the subject of his PhD thesis at Newcastle University. He continued working on Overtone as a researcher at Cambridge University. However, the money for that position was running out, so Aaron knew he would have to find something else to do.
That’s when he came across a three month project from the Broadcom Foundation—designers behind the Raspberry Pi’s chip —to build something to get children more involved in coding. “I put my hand up and said, I think I could do something with this. I can perhaps take Overtone and make it lighter, simpler, and reduce the functionality but focus it specifically at kids."
That idea turned out to be more ambitious than he thought. He initially assumed the three month project would be to produce a prototype. But in the first meeting, it turned out they wanted a prototype, a teacher, a school, lesson plans, and a run through in that classroom with his assistance. That meant it had to be a fully functioning, battle tested system ready to be used by a classroom full of children.
Raspberry Pi version 1 was designed to be low cost and accessible, which also meant that it was low powered. Aaron initially tried a straight port of Overtone using Clojure. It took seven minutes for the namespaces to load up. “The implementation of Overtone wasn't designed for speed for the low power of the Raspberry Pi 1,” said Aaron, “but also the Raspberry Pi 1 had very, very poor JVM support at that time. For example, floating points weren't supported on the hardware and were software only. Luckily things have improved a lot since those early days.”
Aaron switched to Ruby, which performed well enough on the hardware and was also flexible. In fact, it’s so flexible, that you can modify the language itself to suit your needs. “Flexible languages like Ruby can be massaged into the domain that you want to work in,” said Aarom. “So Sonic Pi can be seen as taking Ruby as a piece of clay, and molding it into a musical instrument.”
The prototype turned out to be the easy part. The code for the first early version took two weeks to finish. The graphical interface, the language, the run timing were all completed quickly. “It was just a very, very, very simple system,” said Aaron. “All it did was beeping.”
When actual kids got a hold of the prototype, they understood how it worked and wanted more. “What really surprised me in the classes is that the kids would be like, ‘Oh this is a cool drum rhythm or this is a cool bassline,’” said Aaron. “And they were saying, ‘How do I play the drums at the same time as the bass?’ The system hadn't been designed for that. That's not on the UK computer science curriculum. But it was such a common question. I thought, well let's just add it in.”
With concurrency came a lot of under the hood complexity. It revealed some fundamental difficulties that computers have with precise timing. Once he introduced multiple threads, they would quickly drift out of sync.
Keep it simple
Initially, Aaron used POSIX sleep to manage timing. But like most programmed sleep operations, this operation caused the computer to sleep for at least time T, then perform the next operation when the internal language scheduler picked it back up. “Computers currently are only about throughput, not about latency and timing,” said Aaron. Each delay would add up to throw off timing worse and worse as the song progressed.
Aaron’s solution was to build that internal delay into the sleep commands. Each time that you call a sleep command, it tracks how much time it takes to recover. Instead of sleeping for time T, it sleeps for T minus whatever additional time the previous calculations took. It compensates for drift automatically. SuperCollider, the synthesizer that actually played the sounds, was able to accept timestamps. That meant the language would run a little ahead of the sounds that it produced. “It's not managing the time,” said Aaron, “it's managing the idea of time, when things should happen and then setting them slightly ahead of time so that the synthesizer will then honor those timings as well as it can.”
All that hard work to create this timing is largely invisible to the end user. To them, it’s a simple set of live loops that anyone can understand. “The original constraints or the problem,” said Aaron, “which was to build something which might engage 10-year-olds in a computer science lesson, is a fabulous set of design constraints, not just suitable for that environment but also nightclubs. Making Sonic Pi easy to teach in a classroom turned out to also make it easy to perform with after you've had a few beers and you’re on stage at 2:00 in the morning in a nightclub.”
That mindset stuck with him for a couple of other complicated features. Inspired by 80s and 90s drum and bass, Aaron wanted to be able to take a drum break and slice it up into individual drum hit samples. He started with a solution that takes six lines of code, but over the next year, rethinks it to one that uses just four characters. “It's really a simple idea,” said Aaron. “It's one of those things where once you figure it out, its, ‘Ah, I should have done this all along,’ and that's to treat a drum break as a list of drum hits.” You can now take Amen break, process it real quick, and treat it like an array.
Another breakthrough was a system that made it easy to apply effects like reverb or distortion. As a guitarist, I have to buy a pedal or a pedal board, run a bunch of wires, and live with my choices. In Sonic Pi, I just enclose a block in with_fx: reverb or whatever the effect I want is. “Every time it sees this reverb command,” said Aaron, “it goes and buys new pedal and set that effect or codes it in memory and then executes it, does all the wiring automatically.” The system figures out when you’re done with the effect, it garbage collects it. You don’t have to worry about limits, memory usage, or deallocation.
“It's a lot of fun, but the thought processes and the amount of effort it takes to get a complicated idea into a simple form, it's effort,” said Aaron. “But once you've achieved that, then you've got something that is not only simple to explain and maintain and use, but it's something that you can use to then come up with new ideas and build new things much more effectively than you could with the old things.”
The beat goes on
The Raspberry Pi Foundation supported Aaron for several years, and in that time, he’s been able to add plenty of features, but also work on the foundations. “I'm using Ruby, which is not known to be the most efficient language,” said Aaron. A lot of effort has gone into caching duplicated messages and making the underlying communication with SuperCollider, the synthesizer and MIDI engine, very efficient within the constraints of Ruby. But even with the caching, a synthesizer with an active MIDI clock could flood the system with a lot of time messages.
There are limits, which is why he’s now rebuilding the low level foundation in Rust. With the higher throughput that Rust enables, Aaron sees the possibility of an intermediate language in Sonic Pi, something that users never see, but that any user-facing tools compile into. “It means that Sonic Pi can start to become a target platform for other experimental languages,” said Aaron.
He’s been working with Thor Magnusson at Sussex University, the creator of Ixi lang, another live coding language for music, on an interface for Sonic Pi. “It's very minimal in scope but super quick in a workshop to get people making melodies and rhythms,” said Aaron. “There are very few keystrokes. We've built a minimal version of Ixi Lang, which is going to be in the next version of Sonic Pi.”
By strengthening the foundations, Aaron sees the possibility of more high-level libraries built on top of it, even wild things like Markov chaining and AI assistance. Shoring up the foundations of the timing and events, ensuring that the same code produces the same music every time, means that high-level features will remain simple and easy to understand. “If I'm building too much stuff at a high level, it will all just collapse,” said Aaron. That’s important, because down the road, Aaron plans on adding distributed jamming so you and your friend can code in separate ZIP Codes and rock the same tune.
Music++
Lots of folks have used Sonic Pi to create very impressive musical projects. There’s a whole scene in Japan making music in VR spaces with Sonic Pi. A woman in India is making traditional music with it. A woman in Canada made a whole opera. And a guy in Finland has been making incredibly heavy metal music, taking advantage of the precision of the timing to rock out with double kick pedals and crazy time signatures.
Aaron himself performs quite a bit as well, both at music festivals and programming conferences. Since the Raspberry Pi Foundation funding ran out, it’s been one of his main sources of support to continue developing Sonic Pi. He even got to perform at the Royal Albert Hall alongside an orchestra and vocalists. “Being recognized on a similar level is a real achievement for me,” said Aaron. “I always found it very odd that when I went into schools, the computer science teachers were super excited like, ‘Yes! This is a tool we can educate our children.’ Whereas the music teachers often were, ‘Ooh, this is not a real musical instrument.’"
Even Aaron was dubious that it could be a real creative outlet, to replace his main tool, Overtone. It was built for kids, and maybe not as powerful, but the barrier to entry was so much lower. “Although it was much less powerful in a technical sense,” said Aaron, “it was a way more powerful tool because it was much easier to go from ideas in my head to code on the screen, music out of the speakers. That took moments whereas Overtone will take weeks.”
If you’re interested in trying it out, head over to the Sonic Pi website. There’s an active community there, and Aaron has put in a lot of work to make the tutorials efficient and effective. If you like what he’s doing, consider donating to his Patreon or hiring him for your conference. And if you’re looking to make some sweet jams in your bedroom, download, compose away, and share your tunes in the comments.