1 / 24

Jacob: Virtual Reality Instruction Agent - Pilot Project of VR Valley Twente

This document details the development of "Jacob," a virtual reality instruction agent within the VR Valley Twente pilot project. Jacob's tasks include instruction, navigation, and object manipulation in a virtual world. The approach focuses on using existing knowledge and theories, with an emphasis on software engineering principles for maintainability and adaptability. The technology utilizes Java and VRML standards, allowing Jacob to run on various platforms. Current features include teaching construction tasks like Towers of Hanoi, object manipulation, and simulation of physical processes. Next steps involve enhancing natural language dialogue, intelligence, and multi-user interactions. The architecture includes layers for tasks, 3D world abstraction, and physics protocols, enabling Jacob to interact intelligently in the virtual environment. The system's design emphasizes modular schemas and instructional methods tailored to user actions. The ultimate goal is to create a versatile and intelligent virtual assistant with broad applicability.

rupali
Download Presentation

Jacob: Virtual Reality Instruction Agent - Pilot Project of VR Valley Twente

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Jacoban instruction agent in VR Marc Evers (evers@cs.utwente.nl) 6 March 2000

  2. Contents • context & project description • approach & technology • current state & next steps • architecture of the system • design details • conclusions

  3. Context • pilot project of VR Valley Twente • objective: establishing a regional knowledge centre on VR • organisations: UT (INF, WB), HE, HSA, Carp, MST, ... • 6 pilot projects • http://www.vr-valley.com/

  4. Jacob Project • Jacob as assistant • tasks: instruction, navigation, etc. • Jacob shows intelligent behaviour • multimodal interaction • interaction between multiple agents • manipulation of objects in the virtual world

  5. Approach • use existing knowledge and theories • Intelligent Tutoring Systems, Computer Graphics, AI, … • software engineering focus • how to build such a system in a maintainable, adaptable, …, way • use OO techniques & software architecture knowledge

  6. Technology • runs on any PC or Workstation with web browser+VRML browser+3D graphics card • programming languages • Java 1.1; VRML 97 • standards • H-Anim for Jacob’s body • under consideration: Java 3D, JavaSpeech

  7. Current state • Jacob teaches construction task • Towers of Hanoi • manipulation of objects in virtual world • by Jacob and by user • Jacob has explicit model of the task • simulation of physical processes • gravity, collisions • simple animation

  8. Next steps • natural language dialogue • ‘intelligence’ for Jacob • multi user Jacob (DeepMatrix) • integration in VMC, in 2 steps: • first, Jacob will play isolated role in VMC • second, Jacob interacts with VMC

  9. Basic architecture ‘task’ layer (?) abstract 3D World: agents, cubes, tables, ... abstraction of 3D platform 3D platform: VRML/EAI; Java 3D; ...

  10. World View

  11. Class VWorld • simulation of physical aspects • gravity, collisions • other physical constraints • management of VE objects • agents & objects • keeps track of the state of VE objects • observations

  12. Class VEObject • defines general behaviour of VE objects • bounding box, position, size • gravity, (b)locking • relations like isOnTopOf

  13. Subclasses of VEObject • Cube • detects picking, dropping, dragging by user • Avatar • knows position and orientation of user • Jacob • Jacob’s manifestation in the world • lookAt, moveObject, walktTo, shakeHead

  14. : DraggingSensor : VWorld c : Cube : GravityThread : World NodeProxy Listener 1: drop performed 2: agentMovesObject(c) 3: enforceWorldConstraints( ) 4: startGravity( ) 5: start( ) 6: setPosition( ) 7: update( ) 8: worldStateChanged() ‘Physics’ protocol

  15. ‘Physics’ protocol

  16. c : Cube : VWorld : VEObject b1 : Bounding b2 : Bounding Box Box 1: setPosition(v) 2: avoidCollision(c,v) 3: avoidCollision(c,v) 4: canCollide( ) 5: getBoundingBox( ) 6: getBoundingBox( ) 7: avoidCollision(b2, v) 8: get... Collision avoidance protocol

  17. Collision avoidance protocol

  18. Instruction View

  19. Class Task • knows how to create the ‘scene’ • has knowledge on performing the task • Schema: hierarchical structure • Composite design pattern • action, explanation, pre/postcondition, invariant • schema interface is generic • SchemaIterator determines current task state

  20. Class InstructionModel • InstructionRule • rules that fire other rules or start instruction methods based on conditions • InstructionMethod • executed until goal or state is reached • example: • while user does nothing: wait, give hint, wait, show move

  21. : VWorld t : Task : Schema : Schema : Instruction Iterator Model 1: worldStateChanged( ) 2: updateCurrent( ) 3: precondition(t) 4: postcondition(t) 5: invariant(t) 6: getSubSchemas( ) 7: taskStateChanged( ) 8: currentSchema( ) Task update protocol

  22. Task update protocol

  23. Concluding remarks • current results can be generalised • Jacob based on existing knowledge and theories • instruction agent concept is very general • instruction, assistance, design guidance… • demo & documentation • http://www.cs.utwente.nl/~evers/jacob/

  24. Jacob Screenshot

More Related