1 / 31

Student: Mike Jiang Advisor: Dr. Ras, Zbigniew W.

Student: Mike Jiang Advisor: Dr. Ras, Zbigniew W. Music Information Retrieval. Facets of Music Information. Pitch - fundamental frequency Melody Temporal- duration rhythmic Timbral * tone color. possible Applications. Aural Queries Query By Humming (QBH) systems

norman-combs
Download Presentation

Student: Mike Jiang Advisor: Dr. Ras, Zbigniew W.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Student: Mike Jiang Advisor: Dr. Ras,Zbigniew W. Music Information Retrieval

  2. Facets of Music Information • Pitch - fundamental frequency • Melody • Temporal- duration • rhythmic • Timbral* • tone color

  3. possible Applications • Aural Queries • Query By Humming (QBH) systems • Input: aural melody • matches melody, rhythm • Indexing for Aural Queries • melodies are extracted from the source • Translated into text representations of intervals, pitch • Legal • Is any passage from this piece sampled or copied from one of ours?

  4. Possible Applications • Music education • Music performance analysis • Searching music by instruments for Quintet practicing. • Music therapy • Help doctors identify efficient musical pieces. piano sonata string quartet

  5. Why not traditional kdd The nature and types of raw data

  6. Traditional Databases

  7. Signal representation • Binary File • PCM : • Sampling Rate 44K Hz 16 bits 2,646,000 int/min.

  8. Why features extraction? lower level raw data form Energy values at each sample point Object/Pattern detection Feature Extraction Feature Database Pattern Database manageable, (nearly) homogeneous subset of objects Higher level representations traditional pattern recognition classification clustering regression

  9. MusicMiner • organizing large collections of music • create MusicMaps • Automatic description of digital audio files by sound features • visualize the similarity of songs and artists • Similarity search in music collection

  10. MusicMiner- numerical measure ofperceptual music similarity Low level features extraction-400 high level features-60 feature selection Clustering

  11. notify! Whistle A query by whistling/humming system for melody retrieval A collection of approx. 2000 melodies and classical themes

  12. notify! Whistle • Note extraction process • Thresholding • Signal splitting • Fourier analysis • Quantization to MIDI-Note level

  13. notify! Whistle

  14. PlaySOM • Collection provided by user; music archives • Query by Example, Audio File • audio is indexed and feature vectors are store in vector file • interactive exploration • similarity-based search

  15. PlaySOM • Matching Description • Features(Rhythm Patterns) are passed to a self-organizing map • retrieves similar music by creating paths on the map

  16. Shazam-Industry leader in audio fingerprinting • For each audio file, generate reproducible landmarks • –Each landmark occurs at a time offset • For each landmark, generate a “fingerprint” tag that characterizes its location

  17. Shazam-Industry leader in audio fingerprinting Do same for sample Generate list of matching fingerprints timedb–timesample= Constant

  18. Shazam-no match

  19. Shazam-match

  20. C-Brahms Retrieval Engine for Melody Searching Input the melody Match the note sequence and get the answer on composer, title, notes that matched

  21. A Java-based online QBH system A Java applet records the audio signal. Then its fundamental frequency is analyzed. Adaptive preprocessing reduces the influence of background noise on the succeeding steps.

  22. GUIDO • Query by Example • probabilistic matching • probabilistic models • Clustered dataset • tree structure • match the query following the paths

  23. Midomi • Query by Humming,Query by Example • Multimodal Adaptive Recognition System • also takes into account speech and phonetic content • comparing hummed queries to other hummed querieshttp://www.midomi.com/

  24. summary • 43 MIR systems • Most are pitch estimation-based melody and rhythm match • Is there MIR system based on timbre match existed?

  25. WWW.MIR.UNCC.EDU • Auto indexing system for musical instruments • intelligence query answering system for music instruments

  26. Flow chart of MIR with sound separation Polyphonic Sound Get frame Classifier . Pitch Estimation Feature extraction FFT Get Instrument Sound separation New spectrum Power Spectrum

  27. Hierarchical Classification Bass Clarinet Oboe Bass Flute Music Wood Winds English Horn Flute Strings Brass Percussion Viola Piano Guitar Trumpet Cello Violin Harp French Horn

  28. 40ms Feature Extraction Features Classifier

  29. MIR with new strategy Polyphonic Sound Get frame Higher level Classifier . Feature extraction FFT Get Family Finish all the Frames estimation Get Instrument Candidates lower level Classifier Voting process Get Final winners

  30. Questions?

  31. Thanks.

More Related