ronw ee columbia edu
03/20/2006
Newer version with some bugfixes, better documentation, MashupComposer, and a few more feature extractors. In general the bleeding edge code is available from the darcs repository (see the README).
Finally, a slightly more interesting sorted composition is available here. I just sorted Psycho Killer by average spectral centroid. There is a clear movement from impulsy, wideband drum hits to more stationary, pitched chunks.
03/15/2006
So I went and wasted a day of my spring break hacking away at Mike's mashup generator code to create a prototype version of the MEAP reordering software suite. I give you MEAPsoft-0.1 (precompiled jar file). See the readme for lots more information. Note that javasound does not seem to want to play nice in converting stereo files to mono, so as of now the code only works on mono wave files (at least for me). The code is somewhat hacked together and not commented (or even indented) very well, but it does mostly work.
Just to prove that it does in fact work, I went and recreated those wacky Led Zeppelin pieces. Check out the wave files in http://www.ee.columbia.edu/~ronw/code/MEAPsoft/examples.
Special thanks to Mike for writing extensible, easy to follow code for his mashup generator.
02/05/2006
So the mashup generator did not like to play nice with my computer at all. Something like 20% of the times I tried to run it, my computer just froze completely... Luckily I was able to coax it into doing something through judicious use of bash for loops. I bring you the Dust Brothers doing the Dust Brothers:
This is the result of playing the Fight Club soundtrack straight through, reconstructing the current song using samples from the previous song. The music is relatively simple, no vocals, etc. so the database is populated with clean, mostly monophonic samples. So its not a very exciting mashup, but it sounds pretty cool.
11/21/2005
Speech Separation
Some examples are available here
Epitomes
Sound examples:
short epitome - 1 and 2 second patches - 10 second epitome
long epitome - 5 second patches - 30 second epitome
02/21/2005
I have been working on a method to extract the drum track from a polyphonic piece of music. In the context of the audio kiosk project we have been talking about I see this as one of the tools we can use to extract something useful from music input (say from an ipod) that can then be used to synthesize something new. Right now it can only detect bass and snare drums and takes a little while to run (its not even remotely real time). The algorithm is based on that in this paper. The basic idea is to take the STFT of a template drum sound and try and match the template in the spectrogram of the music. Accuracy is increased by adapting the base template to the most similar drum sound in the audio before searching for matches. The approach has two main drawbacks: The template adaptation method sometimes overfits resulting in a template drum sound that includes bits of other instruments causing the algorithm to miss drum events that occur without the other instruments present in the adapted template. Also, the accuracy is sensitive to thresholds used in template matching.
This has obvious application to the idea of an automatic mashup generator, but I don't know how useful it would be to anything else...