semantic hifi browsing listening interacting sharing on future hifi systems l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
SEMANTIC HIFI Browsing, listening, interacting, sharing on future HIFI systems PowerPoint Presentation
Download Presentation
SEMANTIC HIFI Browsing, listening, interacting, sharing on future HIFI systems

Loading in 2 Seconds...

play fullscreen
1 / 21

SEMANTIC HIFI Browsing, listening, interacting, sharing on future HIFI systems - PowerPoint PPT Presentation


  • 337 Views
  • Uploaded on

SEMANTIC HIFI Browsing, listening, interacting, sharing on future HIFI systems. Music Technology Group Universitat Pompeu Fabra (UPF) Barcelona. WP5. Performance Workpackage. Interaction & Performance.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

SEMANTIC HIFI Browsing, listening, interacting, sharing on future HIFI systems


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
    Presentation Transcript
    1. SEMANTIC HIFIBrowsing, listening, interacting, sharing on future HIFI systems Music Technology Group Universitat Pompeu Fabra (UPF) Barcelona

    2. WP5. Performance Workpackage

    3. Interaction & Performance “…it becomes possible for more people to make more satisfying music, more enjoyably and easily, regardless of physical coordination or theoretical study, of keyboard skills or fluency with notation. This doesn’t imply a dilution of musical quality. On the contrary, it frees us to go further and raises the base-level at which music making begins.” (Laurie Spiegel) “Let’s develop virtual instruments that do not just play back music for people, but become increasingly adept at making new and engaging music with people, at all levels of technical proficiency.” (Robert Rowe)

    4. Interaction Has to be: • natural & intuitive • easy And yet… • allow expression • enjoyable • rewarding

    5. Input devices • Feel natural • Maximize bandwidth • Profit from users’ knowledge

    6. Input devices • Feel natural • Maximize bandwidth • Profit from users’ knowledge We propose the use of • Mouth: microphone + small video camera • Hands & arm: remote command used as a baton

    7. Input devices • Feel natural • Maximize bandwidth • Profit from users’ knowledge We propose the use of • Mouth: microphone + small video camera • Hands & arm: remote command used as a baton

    8. Mouth control information will be reinforced by the two simultaneous input modes (sound + image)

    9. Mouth • Mouth interaction will not only allow karaoke • The system will be able to detect at least 4 different mouth input modes: • Singing (karaoke) • Scat (instrumental solos) • Beat boxing (drums) • Silent mouth movements (filters & timbre changes) • Voice transformations include • Voice Excitation based Transformations (pitch change, hoarseness, whisper…) • Vocal Tract based Transformations (timbre…)

    10. Music Context • The results of each of these interaction modes, will depend on the music being played • Use of metadata will provide increasing information

    11. Music Context • The results of each of these interaction modes, will depend on the music being played • Use of metadata will provide increasing information • Example: Scatting on different musical styles

    12. Music Context • This would correspond to a simplified context • More information can be obtained: • From the type of voiced sound (voice analysis - not mere pitch-2-MIDI – should profit all timbre information) • From additional metadata

    13. Music Context • This would correspond to a simplified context • More information can be obtained: • From the type of voiced sound (voice analysis - not mere pitch-2-MIDI – should profit all timbre information) • From additional metadata

    14. Additional Metadata* Time-stamped information: Music • Composition parts (A, B, chorus…) • Harmonic & rhythmic details • Score • Program changes • …. Audio Analysis • …. *Format and contents to be defined in WP1.2

    15. Editable Metadata • Advanced users will be able to edit and enrich the Metadata (in non real time), adding value to their contribution

    16. Hands Movements Will provide complementary information • e.g. crash cymbal on beat boxing Alternate functions • e.g. baton conduction • tempo changes • dynamic changes • groove & swing modification • …… • ……

    17. Hand & Body tracking • A camera fixed to the system could be used • For better tracking resolution (spatial & temporal) an additional device seems necessary • We propose to use the same command, fitted possibly with accelerometers (and wireless communication with the system)

    18. Score Following IRCAM: Instrument Score follower (for automatic performer accompaniment) To be defined: • Options • MIDI (or synthetic) accompaniment • Time-stretched prerecorded audio • Data formats • data resulting from the audio analysis (UPF), sent to the score follower module (IRCAM) (voice2MIDI?) • position data from the score follower to the time-stretching module

    19. Performing on a simple keyboard In this part Sony CSL will implement style and performing rules in a simple keyboard able to follow and continue the user play according to simple style constraints.

    20. Deliverables

    21. MTG Participants • Xavier Serra, local manager • Sergi Jordà, technical manager • Alex Loscos, voice processing • Martin Kaltenbrunner, interfaces • 1 additional programmer