1 / 19

1. Studying the Production Process

Analysis of Realtime Processes Difficulties for Modelling Electronic Modules Produced with Graphical Open Signal Processing Environments Alain Bonardi - Ircam Intangible Heritage Workshop, april 19th, 2007. 1. Studying the Production Process.

cree
Download Presentation

1. Studying the Production Process

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Analysis of Realtime ProcessesDifficulties for Modelling Electronic ModulesProduced with Graphical Open Signal Processing EnvironmentsAlain Bonardi - IrcamIntangible Heritage Workshop, april 19th, 2007

  2. 1. Studying the Production Process

  3. Studying the Production Processexample of the String Quartett by Florence Baschet • An « augmented » string quartett • Each instrument is enhanced with sensors (motion capture, pressure) • The goal is to detect « specific events » decided by composer Florence Baschet and to trigger the execution of predefined effects

  4. Electronic Configuration of Baschet’s Quartett For each instrument : 2 sensors (motion capture and pressure) 7 different streams from sensors- three acceleration measures, - three rotation measures,- bow pressure. The problem: parameters don’t match musical concepts.

  5. Towards Musical Gesture Recognition (IMTR team) • To create relationships between • these signals, • and the concepts musicians and composers generally handle, • so that they would not just map physical data to sound generation, but instead use a set of high level musical concepts • For instance, thanks to Hidden Markov Models, the patch is able to recognize some musical gestures • credits : Frederic Bevilacqua, Norbert Schnell

  6. The production process 06 07 08 hardware setup capturing data training the HMM creation Current phase: capturing data in order to train the HMMs.

  7. Events recognition • The events videos\gesture1_violin2.avi videos\gesture1_violin1.avi videos\gesture2_violon2.avi

  8. Gesture modelling • videos\gestureModels.avi

  9. Preservation issues • Performance preservation level • Using the processes to model performance • Identification of general and specific outputs from the processes • Sets of « test data » as sets of input data, coupled with output datas, intended to validate the correct setting of hardware/software (robustness / authenticity) • Documentation provided by the composer regarding the production process itself, containing an evaluation of the problems encountered • Identification of needed tools • Model of the relationship between the data streams and the different elements – sensors, instruments • Tools for analyzing data streams • Preservation of electronic tools • Software modules used in Baschet’s Quartett • Provide information for performance modelling • But need to be preserved…

  10. 2. Preserving Electronic Tools

  11. Graphical Environmentsfor Realtime Signal Processing… • Development of interactive modules without programmation thanks to a graphical language • Growing number of softwares and communities of users • Max/MSP/Jitter, PureData, Isadora, EyesWeb, etc. • From performing arts to robotics

  12. … Provide Open Multimedia Environments • Easy interfacing with many devices(source: Jérôme Abel, http://impala.utopia.free.fr/) • Large possibilities of upgrading • Creation of new processings (Javascript) • Creation of new objects (SDK) • Creation of new libraries, implementing new paradigms • Classes in FTM library (developed by IMTR Team at Ircam)

  13. Realtime Processes • Being able from inputs to compute required outputs within a non-perceivable duration • Examples (from IMTR Team, Ircam) • Score following • Input: live sound from an instrument • Output : position in the score • Gesture acknowledgment • Input: live data from captors (for instance getting acceleration and angle informations) • Output: most probable gesture within a series of gesture models

  14. Authenticity in Preservation • Necessity of preserving processes and not only the output data • But these processes depend on hardware and software conditions • How can we guarantee reperformance respecting minimal authenticity ? • Technical incompatibilities: processors, operating systems, proprietary formats, software versions, etc. • Preservation requires adaptations • Example: Migration from one technical environment to another.Lichtung II by Emmanuel Nunes, from jMax to Max

  15. Patch Abstract Description/Virtualization • Being able to describe in an abstract way a software electronic module (patch) • Requires extraction of artistic and technical knowledge from patches and documents

  16. Towards AutomaticPatch Virtualization ? • Strong difficulties due to the lack of formalism of these open environments • The same processing, for instance a reverberation, may be implemented in different ways • Exchanges between modules or objects may be described in very different ways • Connections, send/receive objects, references (FTM), etc. • A global and comprehensive approach is impossible • Specific approaches • Design simple but helpful modules for musical assistants and realtime process developers ?

  17. Tools for Patch Abstract Description • Patch parsing • Getting patch structure and synthetic information about its content • Patch comparison • Extracting differential information from two patches • Domain ontology • Building a thesaurus to set patch classes

  18. Realtime Process Ontology • Towards an organology of realtime processes ? • Double structure • Mathematical description(ex. Faust system, by Grame) • Musical intention description

  19. Conclusion • The introduction of electronic device and especially realtime softwares has provided • New possibilities of creation • Tools to model performance • But imposed the necessity to preserve them • Preserving electronic modules is quite difficult • Directions of work • Patch parsing • Patch comparison • Domain thesaurus

More Related