Exploring Early Music Technology: From Mellotrons to Digital Sampling Innovations

1. 10. Sampling EE2F2 - Music Technology

2. Early Sampling • It’s not a real orchestra, it’s a Mellotron • It works by playing tape recordings of a real orchestra • Each key starts the relevant tape playing • When the key is released, the tape rewinds automatically • The tapes are only 8 seconds long though, so notes can’t be held any longer Listen to the backing violins in this David Bowie recording from 1969.

3. Digital Sampling • With modern computers, it is fairly trivial to digitally record sounds into memory (or disk) and then play them back in response to MIDI commands • To create a digital version of a Mellotron you need to record the instrument(s) separately for every possible pitch. • Extra desirable improvements: • Extending the playback time • Managing the memory/recording requirements • Introducing velocity sensitivity • Modifying the recordings to produce new sounds

4. Memory Requirements • Early digital samplers were heavily restricted in memory • For example • Ensoniq Mirage (1985) • 8 bit resolution • 144 kbytes memory, 32 kHz sample rate • 4.6 seconds recording time • Mellotron (1963) • 35 notes each with 8 seconds of tape • Total = 35 x 8 = 280 seconds recording time • How could just 4.6 seconds be any use?

5. Looping • After an initial attack portion, many sounds don’t actually change that much with time • Instead of sampling several seconds of a sustained note, it would save memory to just repeat (or ‘loop’) a small section • Unlike tape, digital memory can be accessed randomly, so looping is fairly easy Flute sample With looping

6. Example! Advanced Looping • Looping points must be chosen with care to avoid: • Big changes in the volume of the sound (creating a ‘pumping’ effect) • Changes in phase (creating audible clicks) E.g. 1 E.g. 2

7. Example! Pitch vs. Playback Rate • Compare these two flute samples • The waveform shapes are virtually identical • The higher note can be generated by playing the lower note at a faster speed • Being a semitone apart, the ratio between the speeds is 1:21/12 = 1:1.0595 Flute (F4) Time Flute (E4) Time

8. Example! Multi-Sampling • If you speed up a sample: • The pitch rises • The entire frequency response of the instrument is effectively shifted • Result: it starts to sound ‘squeaky’ • If you slow down a sample: • The pitch falls • The frequency response changes too • Often, the resulting sound lacks upper harmonics • It sounds thin, hollow and simply wrong! • Solution: Multi-sampling

9. Velocity Variations • Up to now, our sampler doesn’t respond to velocity • Acoustic instruments respond in terms of: • Amplitude: Higher velocity = Louder • Envelope: The sound can evolve in a different way • Timbre: Spectrum changes with velocity • Amplitude and frequency variation could be achieved using an amplifier and a filter (more later) • Alternatively, record samples of the actual sound for different velocities

10. Example! Cross-Fading • In a multi-sampled set-up, the sample used for any note can depend on: • Pitch • Velocity • The simplest implementation is to allocate each sample a ‘zone’ in pitch/velocity space • This can mean there is a sudden transition when crossing zone boundaries • To prevent this, interpolation (or cross-fading) can be used

11. Sampling Limitations • Sampled instruments can sound very realistic but they do have some notable limitations: • High memory demands • It’s difficult to sample expressive instruments whose sounds can evolve in response to the performer • Only real instruments can be sampled! • A way of addressing some of these issues is to combine sampling and subtractive synthesis techniques • Such instruments are known as sample & synthesis instruments

12. Amplitude and Timbral Control • As noted before, acoustic instruments respond in terms of: • Amplitude: Higher velocity = Louder • Envelope: The sound can evolve in a different way • Timbre: Spectrum changes with velocity • All these effects can be modelled using familiar processes • Amplitude changes can be synthesised using an amplifier • Envelope changes can be synthesised by varying the parameters of an envelope generator • Timbral changes can be synthesised using a filter • All of these are found in a subtractive synthesiser

13. Sample + Synthesis • This diagram is identical to the subtractive synthesiser except that the V.C.O is now a sampler • It can be playing very short loops or entire recordings Multi-sample playback V.C.F. V.C.A. Output L.F.O. Trigger Env. Gen.

14. Example! Using Sample + Synthesis • Some examples of how sample+synthesis can be used: • Instead of recording multiple velocity samples, just record one loud sound and model velocity effects using a filter and amplifier • To apply performance effects like vibrato or crescendos, use the LFO or amplifier respectively • To create entirely new sounds, use the processes in the same way as an analogue subtractive synthesiser

15. Comparison with Analogue Synthesis • Amongst the samples stored in memory, standard waveforms like sines, squares etc. can also be stored • The sample & synthesis structure can, therefore, do everything that a subtractive synthesiser can • Only notable exceptions in early models are large frequency sweeps (portamento or ‘slide’), especially when using multi-samples • The latest instruments can even handle this.

16. Pros & Cons • Pros • Very convincing, realistic sounds are possible • Compared with other techniques, it’s relatively easy to program a sampler • Cons • Hard to innovate and produce novel sounds – only real instruments can be sampled • Very unconvincing performances are possible – some instruments don’t sample well Suitable Unsuitable Pianos Drums Ensembles Solo strings Solo brass Individual voices ‘One-shot’ sounds or relatively inexpressive Expressive sounds. Smooth transitions between notes.

17. Summary • Sampling • Playback of pre-recorded sounds • Pitch can be varied by changing playback rate • Multi-sampling improves quality over whole pitch & velocity range • Sample + Synthesis • All the benefits of high quality sampled sounds plus the ability to form novel instruments • Easily the most popular technique currently used