Not sure what got me started on this project. Perhaps it was listening to too much Meat Beat Manifesto in the late 80’s. Or maybe it was my wife who got me to sit down and watch the Scratch documentary. Whatever the hell it was, I’ve been wanting to make a decent vinyl scratch app with Actionscript for quite some time now. Now this idea isn’t exactly new. Since the release of Flash Player 10, it’s been possible to manipulate the very byte data of sounds with the SampleDataEvent and many have toyed with this idea including Lee Brimelow, Grant Skinner, Andre Michelle, (unfortunately you can’t actually see that one still), and this guy (sorry – the blog is in French and I couldn’t find an ‘about’ section to provide the actual name). All of these (including a similar one I did myself a few years back) pretty much sucked though – and I’ll tell you why…
The Problem
The trouble with digital audio is that its, well, too digitized. That is, when you write bytes to an audio stream, no matter how fast or slow or whether you write forwards or backwards, every single one of those bytes will be distinctly played. The result of that then isn’t so much a ‘sccrrrratcchh’ sound, but rather how you might imagine a fast moving tape would sound – sorta like a turkey all hopped up on meth. But back in the analog ‘real’ world, when you slide a vinyl disc underneath a needle, it won’t play every note distinctly – instead it kind of plays the sounds in between and mixes it all together. Fortunately, there’s a way of emulating the mixing of in between values digitally; with the fancy sounding word ‘interpolation’.
The Process
Now seeing as how I know as much about audio processing as I do quantum topology (and I don’t even know if that’s a real thing or if I just now made it up), I do what everyone does these days – I turned to Google to see if I could find any ready made examples I could just copy. The most comprehensive thing I could find was this C++ / Obj-C example from Jan Kalis. So I downloaded the source and made a half assed attempt at just a straight port to Actionscript. Not really knowing C++, Objective C, or audio processing really made that a rather difficult task and I soon just gave up and went back to googling. While I couldn’t find any better source code, every blog post, forum thread and stack overflow question I came across all mentioned that magic word: ‘interpolation’ – quite often preceded by the word ‘hermite’. Actually, one other place where source code was available was in the Mixxx project (incidentally, Mixxx is a great open sourced DJ software project – I definitely recommend downloading it and playing around if you’re in to that sort of thing). So after digging around through the Mixxx source, I discovered that they were, indeed, using hermite interpolation to create their scratch sound. That then seemed to be the key. Still I was stuck though. I had no idea how to implement hermite interpolation with sound in Actionscript. So back to Google I went. And that’s when I came across this great blog post from Alex Nino. I downloaded the example files and immediately learned two things – how to use interpolation with sound and the very nifty little trick of extracting float samples from a sound’s byte array and storing them in a vector for easier access.
So I turned back to my own project and tried those two tricks and interpolating between a current position and target position based on mouse movement. I compiled expecting to hear amazing things and… nada. I still had my hopped up turkey. I searched around for other interpolation methods and tried 3 or 4 others. All without any change. At this point I cussed and fumed and kicked the floor and swore to give up on the project.
After that, I went back to the Jan Kalis source code. Now that I knew a bit more about storing and accessing sound float values and interpolating those values with Actionscript, all that C++ suddenly made a lot more sense. So with a bit of tweaking, fiddling, tinkering, and fine tuning that cpp into as, I finally came up with the below..
It ain’t perfect, but it’s the best I’ve heard in Actionscript yet. And if you’re interested in the source, you check it all out over on Wonderfl.
wiki wiki…
Last week I have also worked a bit with sound and byteArray for a client project.
I see you have made a great piece of work here!
Keep up the good work!!
Brilliant work!!
If you scratch very slowly it’s not very recognisable as the actual mp3 (i.e. if you try and drag as if to play the song at its original speed), but I guess that’s down to what is actually going on in the implementation. That said it sounds awesome, a job well done sir. I wrote a little 3d turntable app and had scratching in that but wasn’t happy with the result… might have to try and plug in what you have done and see how that sounds!!!
Any idea on the performance of the method you adopted… from my dabblings with sound, some techniques can be quite taxing on the ol cpu.
i.e. can the re-sampling be performed in sub 3ms on a decent machine? Just curious (and lazy as I could just try and profile it myself but super busy at the mo)
once again congratulates on by far the best sounding attempt at this I have heard to date :)
Thank you Paul!
@Ben, Thank you for the comments. You’re right about the slow movement, but it was a bit of tradeoff. There’s still a lot of work that could be done on it. To be honest, I put the source on wonderfl in the hopes that someone who knew more about audio than myself would fork it and make it better. Not really sure about the performance, but it was built on a rather oldish laptop and worked fine there. If you give it a go on your 3D turntables, let me know – I wouldn’t mind checking it out, even if it’s just a video.
And, completely unrelated, but that was some great work on your AGAL Helper as well.
Well as soon as I am any less than stupidly busy with unrelated work, I had planned to upload a demo, so if (hopefully when) I get round to it I’ll be sure to let you know and send you a link.
An awesome side-effect: If you click on the waveform and just keep it still, you get an awesome deep rumble. It sounds a bit like the engine room on the Enterprise.
Wasn’t going to mention it, but that’s a feature – not a bug. Honestly.