Computer music researches in ntuee
Download
1 / 56

Music Signal Processing PowerPoint 2000 ppt - PowerPoint PPT Presentation


  • 723 Views
  • Uploaded on

Computer Music Researches in NTUEE. Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication Engineering National Taiwan University. Research Areas of Shyh-Kang Jeng. Computer Music (JCMG) Agent Systems and Multimedia (JAMG)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Music Signal Processing PowerPoint 2000 ppt ' - Faraday


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Computer music researches in ntuee

Computer Music Researchesin NTUEE

Shyh-Kang Jeng

Department of Electrical Engineering/

Graduate Institute of Communication Engineering

National Taiwan University


Research areas of shyh kang jeng
Research Areas of Shyh-Kang Jeng

  • Computer Music (JCMG)

  • Agent Systems and Multimedia (JAMG)

  • Electromagnetic Theory and Applications (JEMG)



Outline
Outline

  • Sound Output and Separation

  • Sound Synthesis

  • Music Information Retrieval

  • Music Perception


Advantages of interactive 3 d sound system
Advantages of Interactive 3-D Sound System

  • Sound Localization

  • Spatial Impression

  • Fast Computer & DSP

  • It‘s very cheap!!!


What is hrtf
What is HRTF ?

  • HRTF(Head Related Transfer Functions) 在自由聲場下,聲音傳遞至人耳耳道的轉移函數,為達成聽覺虛擬實境最重要的關鍵.

  • 3D Sound=HRTF*(Mono Sound Source)

  • Getting HRTF -- By Measurement or Simulation



仰角

0

°

清大的

HRTF

麻省理工

HRTF

方位角

0

°和

可以辨認

:一人

可以辨認

:四人

180

°時的比較

不能辨認

:四人

不能辨認

:一人

方位角

0

°時

可以辨認

:四人

可以辨認

:四人

不能辨認

:一人

不能辨認

:一人

方位角

90

°時

均可辨認

均可辨認

方位角

180

°時

可以辨認

:四人

可以辨認

:四人

不能辨認

:一人

不能辨認

:一人

方位角

270

°時

均可辨認

均可辨認

HRTF的聽覺比較



Virtual concert hall

10

unit : m

8

6

z - a x i s

4

2

0

10

8

6

4

2

0

-2

-4

10

8

6

-6

4

2

0

-8

-2

-4

-6

-10

-8

y - a x i s

-10

x - a x i s

Virtual Concert Hall




Viess architecture
VIESS Architecture

irc2

simulation

Room

environment

HRTFL filter

Sound source

(*.wav file)

RTF filter

HRTFR filter


Viess demonstration

Weak

Strong

VIESS Demonstration

Room

environment

irc2

simulation

Sound

Source

(*.wav file)

RTF filter

Original


歌曲人聲消除

  • 原音 軟體消音 自製消音


Outline1
Outline

  • Sound Output and Separation

  • Sound Synthesis

  • Music Information Retrieval

  • Music Perception


Physical modeling synthesis
Physical Modeling Synthesis

  • Synthesize sound of a musical instrument through computation based on its sound-generation model

  • The artificial instrument can have the same parameters as the real instrument

  • Expressivity of control is unlimited







Cont d
實際燒錄(cont’d)


Outline2
Outline

  • Sound Output and Separation

  • Sound Synthesis

  • Music Information Retrieval

  • Music Perception


Music retrieval service
Music Retrieval Service

Key-Melody

Notes

Key-MelodySearching Unit

MIDI Files

The Best Match

Desired Music

MIDI Sequencer



Example of melody contour ds 1 st measure of moon river
Example of Melody Contour DS(1st Measure of “Moon River”)

<MelodyContour>

<Contour> 67 4 –1 –1 –1 –1 1</Contour>

<Meter>

<Numerator>3</Numerator>

<Denominator>4<Denominator>

<Meter>

<Beat> 1 4 5 7 8 9 10</Beat>

<MelodyContour>


Edit distance computation
Edit Distance Computation

  • p={0 0 -4 2 0 0 -3} q={0 0 0 2 2 -2}


Music query system prototype
Music Query System Prototype

Put the note into the piano bar

Choose what kinds of notes

Message Board



Outline3
Outline

  • Sound Output and Separation

  • Sound Synthesis

  • Music Information Retrieval

  • Music Perception


Integrated recognition model

Rhythm Recognition

Pitch Recognition

Timbre Recognition

Chord Recognition

Time-frequency analysis

Pattern recognition

Integrated Recognition Model

STFT

Constant-Q

Wavelet


Spectral shapes of flutes and trumpets
Spectral Shapes of Flutes and Trumpets


Slope of the attack
Slope of the Attack

Violin

Flute

AcousticPiano

Trumpet

String Ensembles

AcousticGuitar


Results of timbre recognition unit in monophonic environments
Results of Timbre Recognition Unit in Monophonic Environments

Total Recognition rate = 80.56%


Results of timbre recognition unit in polyphonic environments
Results of Timbre Recognition Unit in Polyphonic Environments

Total Recognition rate = 49.07%


Models of chord classification

Cochlea Environments

Cerebral

Cortex

Perception

Neural

Network

Wavelet

Transform

Recognized

Chords

Models of Chord Classification



Transcription example
Transcription Example Environments

Original

Transcription Result


The octave problem

2.5 Environments

2

Magnitude

1.5

1

0.5

0

0

200

400

600

800

1000

1200

1400

Frequency(Hz)

5

4.5

4

3.5

3

Magnitude

2.5

2

1.5

1

0.5

0

0

200

400

600

800

1000

1200

1400

Frequency (Hz)

The Octave Problem


Pitch detection system

Digital audio signal Environments

Constant-Q transform

Spectral peaks

64

Spectral peaks for every 64 samples

Grouping

Series G1

Series i

Series C8

sustained or silent

Onset/offset detection

off

on

Series i

Octave detector for series i

Pitch(es)

clear

Pitch memory for series i

Pitch Detection System


Octave detection example
Octave Detection Example Environments

Original

Test Results


Wav to midi
鋼琴和絃 Environmentswav to midi 轉換


Beat tracking system
Beat Tracking System Environments

  • Off-line method

  • Input signal sampling rate : 8 kHz

  • Platform : MATLAB


Variance diagram
Variance Diagram Environments


Piano music
Piano Music Environments


A pop song
A Pop Song Environments


Polyphonic music
Polyphonic Music Environments


爵士鼓節拍校正 Environments


Conductor program
Conductor Program Environments


Musical sound recognition model using som
Musical-Sound Recognition Model using SOM Environments

Front-end Processing

The best-matching neuron is fired


Experiment i musical notes in different pitches
Experiment I Environments─ Musical-notes in different pitches

  • Input Pattern:

    • Timbre: Piano

    • Pitch: C3 ~ C5, include those black-notes

      • C,C#: (Red)

      • D,D#: (Orange)

      • E: (Yellow)

      • F,F#: (Deep Green)

      • G G#: (Light Green)

      • A A#: (Blue)

      • B: (Purple)


Experiment result i learning process of som
Experiment Result (I) Environments─ Learning Process of SOM

  • Let’s see the learning process of the map while it listens to piano’s single-notes in different pitches over and over.

  • The map is in disorder at first.


Experiment result i learning process of som cont
Experiment Result (I) Environments─ Learning Process of SOM (cont.)

  • As the iteration number goes up, the map is stable and well-organized.


Experiment ii musical notes in different timbres
Experiment II Environments─ Musical-notes in different timbres

  • Input Pattern:

    • Pitch: G3 A4 C4

    • Timbre: Piano (Red), Violin (Orange), Oboe (Yellow), Clarinet (Deep Green), Trumpet (Light Green) , Trombone (Purple), French Horn (Blue)


Experiment result ii organization of notes in different timbres
Experiment Result (II) Environments─ Organization of notes in different timbres

  • For notes in different timbres and pitches, memories are mainly organized by pitches.

    • Piano (Red)

    • Violin (Orange)

    • Oboe (Yellow)

    • Clarinet (Deep Green)

    • Trumpet (Light Green)

    • Trombone (Purple)

    • French Horn (Blue)


ad