Interactive AudioManipulation Processor Todd Carter Daniel Ross Stephen Tang
What is the iAMP? • It’s interactive: the iAMP is an FPGA-based processor that lets YOU perform real-time manipulation of an input audio source. • More specifically, using a joystick, the user can graphically position the location of a sound source (such as a CD player) in real-time.
Overview • Physical Setup • User Interface • video output • joystick input • Audio Path • audio cues • how iAMP localizes sound • A-D and D-A
VGA Monitor joystick iAMP Setup Speaker Speaker Speaker Speaker
Left Right VGA Monitor joystick iAMP Setup iAMP (Altera UP1) D/A 1 Audio Amp Audio Source A/D Front Left D/A 2 Right Rear
Joystick.vhd UI.vhd Video.vhd Dac_interface.vhd Dac_port.vhd Adc_port.vhd Audio_Manipulator.vhd Dac_interface.vhd Dac_port.vhd Ram_interface.vhd Ram_port.vhd Design Overview Main.vhd
speaker cursor center icon speaker speaker User Interface speaker
Top-Level UI Interface clock clock main ui video cursorX cursorY vga_red vga_red vga_green vga_green vga_blue vga_blue vga_h_sync vga_h_sync vga_v_sync vga_v_sync
vga_h_sync HSync syncgen count_xy vga_v_sync VSync video countY countX ColourChooser cursorX cursorY vga_red vga_green vga_blue Video Interface clock
Joystick pin # function 1 up 2 down 3 left 4 right 5 unused 6 button 7 unused 8 ground 9 unused 5 4 3 2 1 8 9 7 6 Female DB-9
Audio Cues • What techniques exist to ‘fool’ the ear? • Auditory mechanism isn’t capable of understanding the environment as a domain. • Headphones -- externalization of sound. • We can take advantage of these weaknesses!
Audio Cues • Ears use many cues to comprehend its surrounding. • Binaural Intensity Difference • Binaural Phase Difference • Cone of Confusion • Outer Ear Frequency Response • Reverberation
Audio Cues • Binaural Intensity Difference • BID aids the individual to localize sound by the difference in signal intensity between the two ears. • Sound appears to originate from direction of higher intensity. • 10 dB (10:1) difference required to place sound 45o to one side. • Binaural Phase Difference • Not often used - only effective for pure sinusoidal waves <1500 Hz.
Audio Cues • Cone of Confusion • The ear cannot directly perceive whether a sound is coming from ahead or behind, when source is to one side of the median plane. Any sound source along the edge of the “cone of confusion” is indistinguishable from any other sound source along the edge.
Audio Cues Up Back Left Right Front Down Cone of Confusion
Audio Cues • How does the ear tell the difference between sounds coming from ahead or behind?
Audio Cues • Outer Ear Frequency Response • Reverberation • Sounds closer to the ear have lower intensity echo signals vs. incident intensity. • Echoes and reverberation fool the brain into thinking a sound is farther away than it really is. • More echo means more diffused sound path to ear = farther away.
Sound Model • How does iAMP create the sound “location”? • Gain differences between speaker outputs. • Echo created by delayed samples kept in RAM. • Our Simplified Model • Multiplying 24-bit samples takes up too much space! • Simplified linear models applied. Not as accurate, but saves lots of space on Flex 10K20!
Dac_port.vhd Adc_port.vhd Audio_Manipulator.vhd Dac_port.vhd Ram_interface.vhd Ram_port.vhd Audio Path D/A 1 A/D D/A 1 RAM
Audio Manipulation • Sequentially capture 8 samples per sample period from RAM Sample Stack • 4 ‘incident’ delayed samples (initial wavefronts) • 4 ‘echo’ delayed samples (reflected wavefronts) • Multiply each sample by gain (1/16 to 15/16) • Provides 11.8 dB Binaural Intensity Difference (> 10 dB) • “Mix” the echo and incident samples, output to DACs for conversion out to speakers. • (Really summing…)
Audio Manipulation X_loc Y_loc System_clock Sample_clock sample_req data_ready delay Audio Controller sample_clock gain FL Speaker FR Speaker RL Speaker RR Speaker sample_in Register Bank Register Bank Register Bank
Speaker Gain vs. Cursor Location (/16) • Similar for all four speakers.
Time Delay vs. Cursor Location (t) • t is a time constant based on the speed of sound and the distance between speakers.
A/D Converter • CRYSTAL CS5360 24-Bit Stereo A/D Converter for Digital Audio • 24-bit samples -- for 16 or 8 bit sampling, we take the most significant bits (Big-Endian -- the first 16 or 8 bits.)
D/A Converter • CRYSTAL CS4390 24-Bit Stereo D/A Converter for Digital Audio • 24-bit samples -- for 16 or 8 bit sampling, we take the most significant bits (Big-Endian -- the first 16 or 8 bits.)