developing a virtual piano playing environment
Download
Skip this Video
Download Presentation
Developing a Virtual Piano Playing Environment

Loading in 2 Seconds...

play fullscreen
1 / 23

Developing a Virtual Piano Playing Environment - PowerPoint PPT Presentation


  • 123 Views
  • Uploaded on

Developing a Virtual Piano Playing Environment. By combining distributed functionality among independent Agents. A.Broersen. Overview. Introduction to the project Backgrounds of the concepts Design of a multi-agent platform Playing with the system Conclusions and remarks

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Developing a Virtual Piano Playing Environment' - ricky


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
developing a virtual piano playing environment

Developing a VirtualPiano Playing Environment

By combining distributed functionality among independent Agents

A.Broersen

overview
Overview
  • Introduction to the project
  • Backgrounds of the concepts
  • Design of a multi-agent platform
  • Playing with the system
  • Conclusions and remarks
  • Work in progress towards the aid for a real piano teacher
introduction parlevink
Introduction: Parlevink

(http://parlevink.cs.utwente.nl)

A group on Language, Knowledge and Interaction doing research on:

  • Human-Computer Interaction
  • Dialogue systems
  • Autonomous agents
  • Virtual worlds

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration

introduction field
Introduction: Field

Exploring possibilities usingVR and Human-Computer Interactionin distributed learning environments.

Case: Piano Playing

  • interactions using a real keyboard, VR, dialogues and dataglove
  • playing with more people at the same time
  • creating musical tools and a teacher in a dynamic learning environment

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration

introduction adri
Introduction: ADRI

(Artificial Didactic Recital Instructor)

GoalBuilding a framework for a piano teacher in a virtual piano playing environment

With

  • a real keyboard connected to
  • a piano in a 3D-world with
  • tools for music- and piano-education
  • in a distributed architecture

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration

introduction requirements
Introduction: Requirements

ADRI should contain:

  • different independent processes
  • server functionality
  • Java3D
  • Midi for sound
  • using Windows and Unix
  • general knowledge/lesson structure

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration

design agents
Design: Agents

For flexibility different ‘Agents’ are used

What are Agents in ADRI?

  • Different functions different skilled agents
  • When can you call the processes ‘Agents’?
  • When are agents ‘Intelligent’?

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration

design multi agent platform
Design: Multi-Agent Platform

Connecting the agents with each other in a platform:

  • Simple communication (messages) between agents
  • On each machine one agent to control the ADRI-agents
  • One agent is manually controlled to handle the available agents and their different connections

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration

design communication
Design: Communication

Exchanging XML-messages:

  • Communication messages

using a simple protocol

  • Midi messages

as quickly as possible

  • ADRI messages

exchange of knowledge (graphs)using multicast functionality

‘Is someone listening?’

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration

playing the agents
Playing: The Agents
  • Midiator

playing and capturing Midi sounds

  • Visualizer

user dialogue in a 3D World with a visualization of a piano and notes

  • Supervisor

(remotely) control the system

  • Communicator

handle agent functionalityon one Java Virtual Machine

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration

playing more agents
Playing: More Agents
  • Expert

provide songs and chord recognition

  • Analyser

analyses the notes in a song

  • Observer

tracks the users actions and progress

  • Instructor

gives assignments to a learner

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration

playing user interface
Playing: User Interface

In the Supervisor:

  • Dynamic visualization and control of the complete agent network in a graph
  • An agent dialogue frame with the specific options in the Agent
  • Sending messages to agents

Different combinations of agents give different environments

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration

conclusions agents
Conclusions: Agents

In the resulting prototype:

  • the agents are kept ‘simple’
  • the graphical Supervisor provides a good control and overview of the system
  • the message-parsing is fast enough for real-time exchange of Midi
  • there is no good structure (yet) for knowledge and lesson structure

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration

conclusions communication
Conclusions: Communication

Using no standard package:

  • The communication layer is still under development according to different needs in the application
  • Quick exchange of sound and graphical events using Midi
  • Midi capture is a problemusing only Java

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration

work in progress building
Work in progress: Building...
  • Creating more demo Agents:
    • Navigator
    • Translator
    • Corrector
  • Using advise and remarks of human experts and users
  • Developing plans for lessons:Using graphs for state-transition diagrams

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration

work in progress where
Work in progress: Where?

For (more) information or questions:

  • Parlevink, Dept. of Computer Science
  • University of Twente, the Netherlands
  • http://parlevink.cs.utwente.nl
  • [email protected]

Overview

Introduction

Design

Playing

Conclusions

Work in progress

Demonstration

demonstration screen shots
Demonstration: Screen-shots

Host address

Remote Host

Route to Supervisor

Other Agent

ADRI Agent

demonstration screen shots1
Demonstration: Screen-shots

Terminator

Multicast Member

Multicast Group

Communication Group

ad