• Feb272006Class

2006-02-27 Class

* Douglas showed his Milan painting machine. It worked! It was beautiful, particularly from the inside!

* Dan's vision for MEAPsoft:

audio -> (segmenter) -> beat segment list -> (feature calculation) -> beat times + features -> (composer) -> source + dest time references -> (synthesizer) -> new audio

• - constraint: features are constant per segment, segments are variable length - led zep hairway to steven is just reversing the order of the segments in the (composer) - australian poet mashup is just sorting on pitch value in (composer) - original mashup cross-synthesis is using song 1's beat times and placing song 2's segments that are closest in feature space - 2 basic file formats:
  • - segment time file: audio file name, then list of start, duration times, then any number of optional feature columns
    • - each file refers to only one audio file?
    - synthesis control file: list of output times, references to segments of source files that get added in, optional modification operands with arguments (basically an edit decision list)
    • - a given synthesis description may refer to multiple audio sources
  - (synthesizer) is largely the same for everybody - have as a binary, no need to reimplement - (segmenter) is very close to the front-end of Mike's system - Mike to break out?
  • - weird poetry sorting example would require a different kind of segmenter to break on phonemes
  - (feature calculation) is a big place for making changes and variations
  • - Mike's code already contains an example - Maybe a modular architecture so different features can be stacked at will (but will get confused?)
  - (composer) is the obvious place of artistic innovation
  • - sorting on various columns is one of the stock things this structure suggests - or just some kind of nearest-neighbor random walk on some of the feature dimensions

Features

• Mean (auditory) spectrum
• Pitch/chroma
• Loudness
• Amplitude modulation "sharpness"