2006-03-27 - MEAPsoft in Matlab
Here's how I used the MEAPsoft tools of RonWeiss to compose segments with Matlab:
- Downloaded and compiled MEAPsoft-0.2. It compiled cleanly on my Powerbook once I changed "cp -a" to "cp -pr" in the Makefile.
Extracted an MP3 file to a 22 kHz WAV file Aimee.wav:
% mpg123 -2 -m -w Aimee.wav ~/Music/Aimee_Mann/Bachelor_No._2/11-Susan.mp3
- Broke into segments with the segmenter:
% java -cp bin/MEAPsoft.jar Segmenter -o Aimee.seg Aimee.wav
- Calculated the default 40-band Mel-spectral features (allocating 250 meg to java):
% java -mx250m -cp bin/MEAPsoft.jar FeatExtractor -o Aimee.feat Aimee.seg
Started Matlab, read in the features with featread.m, sorted them for greedy local similarity with ordercols.m, and wrote out an edit decision list with edlwrite.m:
{{{>> [F,T,D] = featread('Aimee.feat'); >> X = ordercols(D(:,1:10)',1); >> subplot(211) >> imgsc(D(:,1:10)') >> subplot(212) >> imgsc(D(X,1:10)') >> edlwrite('Aimee-order.edl',F(X),T(X,:));}}} http://www.ee.columbia.edu/~dpwe/meap/meapsoft/Aimee-sgrams.jpg
Resynthesized to create new version Aimee-order.wav:
% java -mx250m -cp bin/MEAPsoft.jar Synthesizer -o Aimee-order.wav Aimee-order.edl This is what you get if you sort on the full band, instead of just the bottom 10 bins: Aimee-b.wav
2005-02-14
I'm Dan Ellis, Assistant Professor in the Electrical Engineering department. In my lab, LabROSA, we do research on extracting information from sound, including speech recognition, source separation (or, more generally, sound organization), and music signal analysis.
I'm interested in finding stimulating applications for signal processing and machine learning techniques as applied to musical sound. In my role as a research manager (as distinct from a researcher), I am excited by the way that the enormous 'cool' factor of music/art/avant garde can stimulate people to really stretch technology and make the impossible work. My ideal is for this project to result in a series of hard-core IEEE publications on great new signal analysis techniques, all of which were created as the necessary solutions to problems that came up in creating a weird and uncategorizable art thing piece.
One sort-of vision I had for what we might create was something like a booth in Lerner where you can plug in your iPod, play something in to it, and have the booth respond with an experience (audio/visual?) that incorporates what you are putting in, what you have put in in the past, and what other people have put in recently, or are feeding in at the same time. I particularly like the idea of an evolving space, where people are motivated to return day after day, to refine their own 'personalities' as known to the machine, and to see what else has changed or been added in the interim.
2005-02-14: I'm putting together an EquipmentShoppingList
dpwe ee columbia edu