Lecture 6 : DIGITAL sound technology

1. Lecture 6:DIGITAL sound technology

2. Outline • Introduction to Sound • Digitising process • Multimedia System Sound • Capture & Playback of Digital Audio • Audio & MIDI file formats • File size & Quality • MIDI • Editing Digitized Sound • Speech Synthesis and Recognition

3. Introduction Sound

4. Introduction Sound (cont) • Vibrations in the air create waves of pressure that are perceived as sound • Sound comprises that spokenword, voices, musicand even noise • Sound waves vary in sound pressure level (amplitude) and in frequency(pitch)

5. Introduction Sound (cont) • A brain’s interpretation of certain impulses being sent by the inner ear through the nervous system Something vibrates in the air Waves of pressure Ear drums will translate these changes in wave Forms as sound

6. Introduction Sound (cont) • Characteristics of Sound Waves Time for one cycle Amplitude distance wavelength along wave Cycle

7. Introduction Sound (cont) • Pleasant sound • regular wave pattern • pattern is repeated over and over • Noise sound • Wave pattern are irregular • Do not have a repeated pattern

8. Introduction Sound (cont) • Continuous of Sound Wave

9. Introduction Sound (cont) • 3 Classes of Audio • Voice • Defined as talking, not singing • Does not have great fluctuations in pitch and tone • Can be captured at a much lower frequency than music • Lower frequency requires less data • Music • Created from human singing or musical instruments • Rapid changes in tone and pitch within very brief periods of time • Usually recorded and played in stereo or multiple playback tracks

10. Introduction Sound (cont) • Sound effect • Can be voice or music, but often created by natural events i.e door slamming shut or thunderclap • Vary in tone, pitch and time

11. Introduction Sound (cont) • 2 Types of audio form • Analog sound • Direct and continuous flow of sound wave i.e: switch on a microphone that will detects sound wave and transform it to electrical signal. • Detect it by measuring the pressure level at a location using a transducer to convert pressure to voltage levels

12. Introduction Sound (cont) • Digital sound • Digital sound is sound that has been converted to or created in, a discrete form (namely a set of numeric values) • Natural sound is a continuous phenomena and is converted to digital form by sampling techniques • Digitized analog sound through Analog-to-Digital Converters (ADC) • Can digitize sound from microphone, synthesizer, existing tape recording, live radio & TV broadcast & CD • After stored in a computer, information can be retrieved and edited and sent to the speakers via a Digital-to-analog Converter (DAC)

13. Introduction Sound (cont) • 3 characteristics for recording sound: • Frequency (measure of pitch) • Amplitude (measure of loudness) • Sound channels • Mono – 1 channel • Stereo – 2 channel

14. https://www.youtube.com/watch?v=qNf9nzvnd1k Introduction Sound (cont) • Frequency • Measure of how many cycles occur in one second • Measured in Hertz (abbreviation Hz) • directly corresponds to the pitch of a sound • the more frequent vibration occurs the higher the pitch of the sound • Optimally, people can hear from 20 Hz to 20,000 Hz (20 kHz) • Sounds below 20 Hz are infrasonic  humans can’t hear • sounds above 20 kHz are ultrasonic • some mammals can hear well above this. For example, bats and whales use echo location that can reach frequencies in excess of 100KHz High pitch Low pitch

15. Introduction Sound (cont)

16. 2,093 Hz 4,186 Hz note [C]… but at different frequencies

17. Low amplitude High Amplitude Introduction Sound (cont) • Amplitude • The maximum displacementof a wave from an equilibrium position • The louder a sound, the more energy it has • This means loud sounds have a large amplitude • Resolution - 8 bits for 256 levels and 16 bits for 65,536 levels • Number of bits is based on how precise the level of sound is to be measured

18. Introduction Sound (cont) • Advantages of Digital Audio (assuming the source is good and the digitization process is done properly) • Better quality than analog audio • No “hiss” sound • No disturbed by background music or sound • Difficult to lost data compared to analog • Easy to manipulate the recorded data • Enable to edit digital sound with non-linear characteristic • Disadvantages of Digital Audio • Requires big amount or RAM memory • Requires more hard disk space • Distortion

19. Digitising Process • Also known as sampling process. • There are two characteristics for recording/sampling sound. • Frequency rate • Amplitude measurement • For multimedia purpose, standard frequency rates include 11.025KHz,22.5KHz,and 44.1KHz(CD audio quality).The higher frequency is the better, and with lower frequency rates, while they sacrifice some quality, will also significantly reduce the amount of data. • Amplitude measurement is based on 8 bits(one byte) for 256 levels and 16 bits for 65,536 levels.

20. Digitising Process (cont) • Sampling rate is the number of times chosen per second (Hertz) = the number of samples per second. • E.g: A sampling rate of 22KHz would mean dividing the time axis into 22,000 equally spaced times per second

21. Digitising Process (cont) • High sampling rate

22. Digitising Process (cont) • Low sampling rate 1 second

23. Digitising Process (cont) • Eg: An amplitude measurement is based on 8 bits(one byte) for 256 levels would mean dividing the amplitude axis into 256 equally.

24. Digitising Process (cont) Low amplitude High amplitude

25. Digitising Process (cont) • Sampling of a sound wave illustrated

26. Multimedia System Sound • System sounds are assigned to various system events such as startup and warnings, among others • Macintosh – provides several system sound options such as glass, indigo, laugh • Windows – available system sounds include start.wav, chimes.wav, and chord.wav • Multimedia sound is either digitally recorded audio or MIDI (Musical Instrumental Digital Interface) music

27. Air pressure variations Digital to Analogue Converter Captured via microphone Converts back into voltage Signal is converted into binary(discrete form) 0101001101 0110101111 DAC ADC Analogue to Digital Converter Air pressure variations Capture & Playback of Digital Audio

28. Capture & Playback of Digital Audio (cont) • Recording Audio Files • Microphone • connect microphone to the microphone port and record using sound recorder

29. Capture & Playback of Digital Audio (cont) • Basic Sound Recorder Sound conversion Basic Effects

30. Capture & Playback of Digital Audio (cont) • CD-ROM Drive • Move music files from CD to hard drive or; • Play the CD and then record using the sound recorder • Line-in • pressing play on the audio source, which is connected to the computer’s audio line-in socket. Record using the sound recorder

31. Quality factors • Quality factors for digital audio file depends on: • The quality of original audio source • The quality of capture device & supporting hardware • The characteristics used for capture • The capability of the playback environment

32. Audio File Formats

33. Audio File Formats (cont) • Other audio file formats • AIFFC (compressed) –filename.aifc • RealAudio –filename.ra • MP3 (mpeg layer 3) –filename.mp3

34. MIDI File Format

35. File size & Quality • Crucial aspects of preparing digital audio files are: • Balancing the need for sound quality against available RAM and hard disk resource • Setting appropriate recording levels to get a high-quality and clean recording

36. File size & Quality (cont) • File Size vs Quality • Sampling rate: • determine frequency at which sample be drawn for recording • sampling at high rates (44.1 kHz or 22.05 kHz), more accurate capture the high-frequency • Audio resolution (Amplitude): • 8 or 16 bits • the accuracy a sound can be digitized

37. File size & Quality (cont) • File Size vs Quality (cont) • Stereo – lifelike and realistic (two ears) • Mono – fine but sound bit flat and uninteresting • Stereo require twice storage space as mono file (same length) • Formulas for determining size (bytes) for digital recording • Size of a monophonic digital recording = sampling rate xduration of recording in seconds x (bit resolution/8) x1 • Size of stereo recording = sampling rate xduration of recording in seconds x (bit resolution/8) x2

38. File size & Quality (cont) • Voice and speech: 8-bit resolution and 5-10 KHz sample rate • CD-quality music: 16-bit or higher resolution and 44 KHz • Audio file compression • Sound on the Web -- streaming audio

39. File size & Quality (cont) 44.1 KHz 16 bit Stereo 10.5 MB CD-quality standard (ISO 10149) 44.1 KHz 16 bit Mono 5.25 MB Voice or monoaural,high quality 44.1 KHz 8 bit Stereo 5.25 MB Best quality for playback on low-end PC 44.1 KHz 8 bit Mono 2.6 MB Good trade-off for voice quality 22.05 Khz 16 bit Stereo 5.25 MB Decent quality,darker sounding because of lower frequency 22.05 Khz 16 bit Mono 2.5 MB Good for voice,but the disk space requirement is high 22.05 Khz 8 bit Stereo 2.6 MB Acceptable for good quality playback in low-end PC systems 22.05 Khz 8 bit Mono 1.3 MB Thinner sound,lacks quality, television quality 11 Khz 8 bit Stereo 1.3 MB Lowest quality,dark sounding 11 Khz 8 bit Mono 650 K Lowest reasonable quality for voice

40. Calculating Audio Data Size • File Size vs Quality (cont) • 10 sec recording at 22.05 kHz with 8 bit resolution: • Mono • 22050 * 10 * (8/8)*1 = 220, 500 bytes (229.5 KB) • Stereo • 22050 * 10 * (8/8)*2 = 441, 000 bytes (441 KB) • 10 sec recording at 44.1 kHz with 16 bit resolution: • Mono • 44100 * 10 * (16/8)*1 = 882, 000 bytes (882 KB) • Stereo • 44100 * 10 * (16/8)*2 = 1, 764, 000 bytes (1764 KB)

41. MIDI Audio • Musical Instrument Digital Interface. • Quickest, easiest, and most flexible • The process of creating MIDI music is different from digitizing existing audio. • MIDI consists of music score; to make MIDI scores, developers should require: • Sequencer software (record and edit MIDI data) • Sound synthesizer • MIDI keyboard

42. MIDI Audio

43. MIDI Audio • Small compared to captured audio files • Can be used to deliver relatively high quality synthesized music for a variety of applications. • Based on the time-based approach.

44. MIDI Audio (How it works) • MIDI specifies 16 separate MIDI Channels (16 different instruments at once) • Channels is similar to the idea of television channels. • Sends a signal within a particular frequency range. • Television will receives many different ranges (or channels) at once. • User must tune up signals to set to a particular frequency range.

45. MIDI versus Digital Audio • Advantages over digital audio: • MIDI files are much more compact (200 to 1000 times smaller than digital audio) - take less RAM, spaces, and CPU resources • Can be embedded in web page load and play more quickly • Sound may better than digital audio if using high quality sound source • Completely editable, length can be changed (easy to manipulate) by varying its tempo

46. MIDI versus Digital Audio • MIDI’s disadvantages: • Playback will be accurate only if the MIDI playback device is identical to the device used for production. • MIDI cannot easily be used to play back spoken dialogue. • Why Digital Audio? • Quality of playback (consistent!) • A wider selection of application software and system support • Creating digital audio do not demand a knowledge of music theory (musical notation)

47. Digital audio - software • More advanced Digital audio editing software: • One of the most powerful and professional PC-based packages is a tool called Sound Forge • http://www.sonicfoundry.com/ • Others audio editing software: • COOL Edit Pro • Gold Wave • PROSONIQ SonicWORX • Samplitude Studio

48. Editing Digitized Sound • Digital sound editing is part of a larger field called digital signal processing • Once sound is digitized, it is in discrete numerical format • Numerical transformations on this data can be used to: • change the sound’s pitch • change the sound’s amplitude • add echoes and other special effects

49. Editing Digitized Sound (cont) • Basic sound editing operations • Multiple track- able to edit & combine multiple tracks (sound effects, voice-over, music), merge the tracks & export them in a “final mix” • Trimming – Removing “dead spots” or blank space from the front of recording & any unnecessary extra time. Menu command – Cut, Clear, Erase or Silence • Splicing & Assembly – Linking separate audio segment. Together, usually from different files

50. Editing Digitized Sound (cont) • Basic sound editing operations (cont) • Volume Adjustment – to provide a consistent volume level • Format Conversion – save files in many format, which can be read & imported by multimedia authoring system • Resampling or Downsampling - recorded & edited using 16-bit sampling rates, but lower rates & resolution – must resample or downsample ro save considerable disk space • Fade-in and Fade-out – increasing and decreasing the volume within a segment of the audio data