The live coding language that lets you be an actual rock star
Unless you’re Sam Aaron, that is. He’s the creator of Sonic Pi, a live coding language for making music. It’s an open source project that he built which creates music from code in real time. The world of music is taking notice: Rolling Stone described him as “transcending the present” when he performed at the same festival as Grimes.
Sonic Pi works on simple loops of code. You enter a few commands that indicate beats, melodies, and samples to play. These commands are processed through a synthesizer backend called SuperCollider. The essence of it is loop and repeat, but if you change the code during the performance, the music changes too. It’s a simple and powerful way to get anyone up and running with electronic music or basic programming concepts.
“If you’re a programmer who knows nothing about music then Sonic Pi can help you with that,” said Aaron. “If you’re a musician, who knows nothing about programming, Sonic Pi can help you with that. And if you know nothing about either, you’ve got a fun and exciting journey ahead of you.”
On its face, it’s gloriously simple. So simple, that a 10-year-old child can use it. Under the hood, there are some surprising technical challenges to get a language like this to work.
Before Sonic Pi, Aaron and his friend Jeff Rose had created Overtone, another live coding language built in Clojure. This project grew out his interest in domain-specific languages, which was the subject of his PhD thesis at Newcastle University. He continued working on Overtone as a researcher at Cambridge University. However, the money for that position was running out, so Aaron knew he would have to find something else to do.
That’s when he came across a three month project from the Broadcom Foundation—designers behind the Raspberry Pi’s chip —to build something to get children more involved in coding. “I put my hand up and said, I think I could do something with this. I can perhaps take Overtone and make it lighter, simpler, and reduce the functionality but focus it specifically at kids.”
That idea turned out to be more ambitious than he thought. He initially assumed the three month project would be to produce a prototype. But in the first meeting, it turned out they wanted a prototype, a teacher, a school, lesson plans, and a run through in that classroom with his assistance. That meant it had to be a fully functioning, battle tested system ready to be used by a classroom full of children.
Raspberry Pi version 1 was designed to be low cost and accessible, which also meant that it was low powered. Aaron initially tried a straight port of Overtone using Clojure. It took seven minutes for the namespaces to load up. “The implementation of Overtone wasn’t designed for speed for the low power of the Raspberry Pi 1,” said Aaron, “but also the Raspberry Pi 1 had very, very poor JVM support at that time. For example, floating points weren’t supported on the hardware and were software only. Luckily things have improved a lot since those early days.”
Aaron switched to Ruby, which performed well enough on the hardware and was also flexible. In fact, it’s so flexible, that you can modify the language itself to suit your needs. “Flexible languages like Ruby can be massaged into the domain that you want to work in,” said Aarom. “So Sonic Pi can be seen as taking Ruby as a piece of clay, and molding it into a musical instrument.”
The prototype turned out to be the easy part. The code for the first early version took two weeks to finish. The graphical interface, the language, the run timing were all completed quickly. “It was just a very, very, very simple system,” said Aaron. “All it did was beeping.”
When actual kids got a hold of the prototype, they understood how it worked and wanted more. “What really surprised me in the classes is that the kids would be like, ‘Oh this is a cool drum rhythm or this is a cool bassline,’” said Aaron. “And they were saying, ‘How do I play the drums at the same time as the bass?’ The system hadn’t been designed for that. That’s not on the UK computer science curriculum. But it was such a common question. I thought, well let’s just add it in.”
With concurrency came a lot of under the hood complexity. It revealed some fundamental difficulties that computers have with precise timing. Once he introduced multiple threads, they would quickly drift out of sync.
Keep it simple
Initially, Aaron used POSIX sleep to manage timing. But like most programmed sleep operations, this operation caused the computer to sleep for at least time T, then perform the next operation when the internal language scheduler picked it back up. “Computers currently are only about throughput, not about latency and timing,” said Aaron. Each delay would add up to throw off timing worse and worse as the song progressed.
Aaron’s solution was to build that internal delay into the sleep commands. Each time that you call a sleep command, it tracks how much time it takes to recover. Instead of sleeping for time T, it sleeps for T minus whatever additional time the previous calculations took. It compensates for drift automatically. SuperCollider, the synthesizer that actually played the sounds, was able to accept timestamps. That meant the language would run a little ahead of the sounds that it produced. “It’s not managing the time,” said Aaron, “it’s managing the idea of time, when things should happen and then setting them slightly ahead of time so that the synthesizer will then honor those timings as well as it can.”
All that hard work to create this timing is largely invisible to the end user. To them, it’s a simple set of live loops that anyone can understand. “The original constraints or the problem,” said Aaron, “which was to build something which might engage 10-year-olds in a computer science lesson, is a fabulous set of design constraints, not just suitable for that environment but also nightclubs. Making Sonic Pi easy to teach in a classroom turned out to also make it easy to perform with after you’ve had a few beers and you’re on stage at 2:00 in the morning in a nightclub.”
That mindset stuck with him for a couple of other complicated features. Inspired by 80s and 90s drum and bass, Aaron wanted to be able to take a drum break and slice it up into individual drum hit samples. He started with a solution that takes six lines of code, but over the next year, rethinks it to one that uses just four characters. “It’s really a simple idea,” said Aaron. “It’s one of those things where once you figure it out, its, ‘Ah, I should have done this all along,’ and that’s to treat a drum break as a list of drum hits.” You can now take Amen break, process it real quick, and treat it like an array.
Another breakthrough was a system that made it easy to apply effects like reverb or distortion. As a guitarist, I have to buy a pedal or a pedal board, run a bunch of wires, and live with my choices. In Sonic Pi, I just enclose a block in with_fx: reverb or whatever the effect I want is. “Every time it sees this reverb command,” said Aaron, “it goes and buys new pedal and set that effect or codes it in memory and then executes it, does all the wiring automatically.” The system figures out when you’re done with the effect, it garbage collects it. You don’t have to worry about limits, memory usage, or deallocation.
“It’s a lot of fun, but the thought processes and the amount of effort it takes to get a complicated idea into a simple form, it’s effort,” said Aaron. “But once you’ve achieved that, then you’ve got something that is not only simple to explain and maintain and use, but it’s something that you can use to then come up with new ideas and build new things much more effectively than you could with the old things.”
The beat goes on
The Raspberry Pi Foundation supported Aaron for several years, and in that time, he’s been able to add plenty of features, but also work on the foundations. “I’m using Ruby, which is not known to be the most efficient language,” said Aaron. A lot of effort has gone into caching duplicated messages and making the underlying communication with SuperCollider, the synthesizer and MIDI engine, very efficient within the constraints of Ruby. But even with the caching, a synthesizer with an active MIDI clock could flood the system with a lot of time messages.
There are limits, which is why he’s now rebuilding the low level foundation in Rust. With the higher throughput that Rust enables, Aaron sees the possibility of an intermediate language in Sonic Pi, something that users never see, but that any user-facing tools compile into. “It means that Sonic Pi can start to become a target platform for other experimental languages,” said Aaron.
He’s been working with Thor Magnusson at Sussex University, the creator of Ixi lang, another live coding language for music, on an interface for Sonic Pi. “It’s very minimal in scope but super quick in a workshop to get people making melodies and rhythms,” said Aaron. “There are very few keystrokes. We’ve built a minimal version of Ixi Lang, which is going to be in the next version of Sonic Pi.”
By strengthening the foundations, Aaron sees the possibility of more high-level libraries built on top of it, even wild things like Markov chaining and AI assistance. Shoring up the foundations of the timing and events, ensuring that the same code produces the same music every time, means that high-level features will remain simple and easy to understand. “If I’m building too much stuff at a high level, it will all just collapse,” said Aaron. That’s important, because down the road, Aaron plans on adding distributed jamming so you and your friend can code in separate ZIP Codes and rock the same tune.
Lots of folks have used Sonic Pi to create very impressive musical projects. There’s a whole scene in Japan making music in VR spaces with Sonic Pi. A woman in India is making traditional music with it. A woman in Canada made a whole opera. And a guy in Finland has been making incredibly heavy metal music, taking advantage of the precision of the timing to rock out with double kick pedals and crazy time signatures.
Aaron himself performs quite a bit as well, both at music festivals and programming conferences. Since the Raspberry Pi Foundation funding ran out, it’s been one of his main sources of support to continue developing Sonic Pi. He even got to perform at the Royal Albert Hall alongside an orchestra and vocalists. “Being recognized on a similar level is a real achievement for me,” said Aaron. “I always found it very odd that when I went into schools, the computer science teachers were super excited like, ‘Yes! This is a tool we can educate our children.’ Whereas the music teachers often were, ‘Ooh, this is not a real musical instrument.’”
Even Aaron was dubious that it could be a real creative outlet, to replace his main tool, Overtone. It was built for kids, and maybe not as powerful, but the barrier to entry was so much lower. “Although it was much less powerful in a technical sense,” said Aaron, “it was a way more powerful tool because it was much easier to go from ideas in my head to code on the screen, music out of the speakers. That took moments whereas Overtone will take weeks.”
If you’re interested in trying it out, head over to the Sonic Pi website. There’s an active community there, and Aaron has put in a lot of work to make the tutorials efficient and effective. If you like what he’s doing, consider donating to his Patreon or hiring him for your conference. And if you’re looking to make some sweet jams in your bedroom, download, compose away, and share your tunes in the comments.
crystal is fast ruby-like FWIW 🙂
Been using SP for 2 1/2 years now… I’m neither musical
nor a programmer, but it lets me create stuff I would be
proud to show to the world.
Being shy, I might need a little help with that, but the
community that has built up around SP is very helpful,
I have found they do not judge people on ability, be it
musical or progamming.
If only it had been around 50 years ago, my life might
have been very different.
Paul. (See you on the SP forums). 🙂
Related: Interview with musician (among other things) Dylan Beattie on podcast “.NET Rocks”, episode 1636, 2019-05-23, about the “Rockstar” programming language.
www DOT dotnetrocks DOT com/default DOT aspx?ShowNum=1636
Thanks for sharing this Ryan, this looks amazing.
Could you give more details on the projects that you mention in your last section (Music++)? I’d be especially curious about the “guy in Finland” making heavy metal music, but I don’t seem to be able to find anything about the project on Google or the SP forums.
>> It seems silly to think that whatever skill you have at programming is going to get you featured in Rolling Stone.
You should ask [Thomas Dolby](https://en.wikipedia.org/wiki/Thomas_Dolby) about that.
Also, there is already a “live coding language that lets you be an actual rock star”. It’s called music
Hahahaha You are badass
Great article. I have been using Sonic Pi since version 1 and was hooked on it from the beginning. It has a unique ability to enable you to interact with musical ideas, and quickly to produce sounds and rhythms that can be modified and refined. Version 2 addition of interaction with the outside world whether through midi in/out or OSC messaging can be exploited in many ways. I have even built a Sonic Pi driven Xylophone running on a Raspberry PI and using solenoids controlled by Sonic Pi to hit the notes!
Download and try it for free, and have a look at the in-thread.sonic-pi.net site with other users of Sonic Pi
The music generating languages are so much fun! I recommend giving Orca (from Rekka & Devine) a look too. I cannot comment on other qualities, but it certainly makes you look like an absolute wizard.
Marquette University Engineering school was given an IBM 1620 and, in 1962, a few students started writing programs to create loops of different code. A transistor radio (remember those?) was placed on top of the cabinet over the CPU and tuned to a specific frequency (I’m too old to remember which). The loops created resonance frequencies that were picked up by the radio. We spent many long hours tweaking code to play a few pop tunes and the Star Spangled Banner. How is this related to the topic of the article? Not sure. But it occurred to me when I read it so I thought I’d pass the anecdote along.
So neat! Thanks for sharing
Similar and inventive idea using old tech with new code: https://www.youtube.com/watch?v=4gibcRfp4zA