Developing a virtual piano playing environment
This presentation is the property of its rightful owner.
Sponsored Links
1 / 23

Developing a Virtual Piano Playing Environment PowerPoint PPT Presentation


  • 88 Views
  • Uploaded on
  • Presentation posted in: General

Developing a Virtual Piano Playing Environment. By combining distributed functionality among independent Agents. A.Broersen. Overview. Introduction to the project Backgrounds of the concepts Design of a multi-agent platform Playing with the system Conclusions and remarks

Download Presentation

Developing a Virtual Piano Playing Environment

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Developing a virtual piano playing environment

Developing a VirtualPiano Playing Environment

By combining distributed functionality among independent Agents

A.Broersen


Overview

Overview

  • Introduction to the project

  • Backgrounds of the concepts

  • Design of a multi-agent platform

  • Playing with the system

  • Conclusions and remarks

  • Work in progress towards the aid for a real piano teacher


Introduction parlevink

Introduction: Parlevink

(http://parlevink.cs.utwente.nl)

A group on Language, Knowledge and Interaction doing research on:

  • Human-Computer Interaction

  • Dialogue systems

  • Autonomous agents

  • Virtual worlds

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration


Introduction field

Introduction: Field

Exploring possibilities usingVR and Human-Computer Interactionin distributed learning environments.

Case: Piano Playing

  • interactions using a real keyboard, VR, dialogues and dataglove

  • playing with more people at the same time

  • creating musical tools and a teacher in a dynamic learning environment

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration


Introduction adri

Introduction: ADRI

(Artificial Didactic Recital Instructor)

GoalBuilding a framework for a piano teacher in a virtual piano playing environment

With

  • a real keyboard connected to

  • a piano in a 3D-world with

  • tools for music- and piano-education

  • in a distributed architecture

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration


Introduction requirements

Introduction: Requirements

ADRI should contain:

  • different independent processes

  • server functionality

  • Java3D

  • Midi for sound

  • using Windows and Unix

  • general knowledge/lesson structure

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration


Design agents

Design: Agents

For flexibility different ‘Agents’ are used

What are Agents in ADRI?

  • Different functions different skilled agents

  • When can you call the processes ‘Agents’?

  • When are agents ‘Intelligent’?

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration


Design multi agent platform

Design: Multi-Agent Platform

Connecting the agents with each other in a platform:

  • Simple communication (messages) between agents

  • On each machine one agent to control the ADRI-agents

  • One agent is manually controlled to handle the available agents and their different connections

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration


Design communication

Design: Communication

Exchanging XML-messages:

  • Communication messages

    using a simple protocol

  • Midi messages

    as quickly as possible

  • ADRI messages

    exchange of knowledge (graphs)using multicast functionality

    ‘Is someone listening?’

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration


Playing the agents

Playing: The Agents

  • Midiator

    playing and capturing Midi sounds

  • Visualizer

    user dialogue in a 3D World with a visualization of a piano and notes

  • Supervisor

    (remotely) control the system

  • Communicator

    handle agent functionalityon one Java Virtual Machine

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration


Playing more agents

Playing: More Agents

  • Expert

    provide songs and chord recognition

  • Analyser

    analyses the notes in a song

  • Observer

    tracks the users actions and progress

  • Instructor

    gives assignments to a learner

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration


Playing user interface

Playing: User Interface

In the Supervisor:

  • Dynamic visualization and control of the complete agent network in a graph

  • An agent dialogue frame with the specific options in the Agent

  • Sending messages to agents

    Different combinations of agents give different environments

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration


Playing the visualizer

Playing: The Visualizer


Conclusions agents

Conclusions: Agents

In the resulting prototype:

  • the agents are kept ‘simple’

  • the graphical Supervisor provides a good control and overview of the system

  • the message-parsing is fast enough for real-time exchange of Midi

  • there is no good structure (yet) for knowledge and lesson structure

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration


Conclusions communication

Conclusions: Communication

Using no standard package:

  • The communication layer is still under development according to different needs in the application

  • Quick exchange of sound and graphical events using Midi

  • Midi capture is a problemusing only Java

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration


Work in progress building

Work in progress: Building...

  • Creating more demo Agents:

    • Navigator

    • Translator

    • Corrector

  • Using advise and remarks of human experts and users

  • Developing plans for lessons:Using graphs for state-transition diagrams

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration


Work in progress where

Work in progress: Where?

For (more) information or questions:

  • Parlevink, Dept. of Computer Science

  • University of Twente, the Netherlands

  • http://parlevink.cs.utwente.nl

  • [email protected]

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration


Demonstration playing

Demonstration: Playing


Demonstration dataglove

Demonstration: Dataglove


Playing the supervisor

Playing: The Supervisor


Demonstration screen shots

Demonstration: Screen-shots

Host address

Remote Host

Route to Supervisor

Other Agent

ADRI Agent


Demonstration screen shots1

Demonstration: Screen-shots

Terminator

Multicast Member

Multicast Group

Communication Group


Playing the midiator

Playing: The Midiator


  • Login