1 / 41

Exploring large data spaces in Virtual Reality

Exploring large data spaces in Virtual Reality. Robert G. Belleman Section Computational Science University of Amsterdam robbel@science.uva.nl. Overview. Large data spaces Interactive exploration environments Interaction techniques in Virtual Reality A test case

gamma
Download Presentation

Exploring large data spaces in Virtual Reality

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Exploring large data spaces in Virtual Reality Robert G. Belleman Section Computational Science University of Amsterdam robbel@science.uva.nl

  2. Overview • Large data spaces • Interactive exploration environments • Interaction techniques in Virtual Reality • A test case • vascular reconstruction in a virtual environment R.G. Belleman

  3. High Performance Computing • High Performance Computing: • computing power increases (Moore’s “Law”) • storage capacity increases Result: “data spaces” get increasingly large and complex, multi-dimensional, time dependent. R.G. Belleman

  4. What are “data spaces”? Roughly speaking: • Data sets • the “files” or “states” that are generated • Parameter spaces • the number of “variables” and their allowed “freedom” (range, resolution) in a program R.G. Belleman

  5. Examples of large data sets • Medical images (i.e. CT, (f)MRI, PET) • 512 x 512 at 16 bits slices in common use • tens to hundreds slices: 5Mb to5Gb per scan • 1024 x 1024 at 16 bit in very nearfuture: 20Mb to 20Gb per scan • time variant scans R.G. Belleman

  6. … more examples ... • Simulation experiment results • FEM, MD, lattice Boltzmann • often dimensionality > 3(e.g. time variant) • multiparameter data fields • gigabytes of data per “run” R.G. Belleman

  7. … more examples ... • Measurements • high-speed data acquisition devices: particle accelerators, microbeam scanners, DNA scanners, CLSM • Financial data, etc. Terabytes of data per experiment is no longer an exception! R.G. Belleman

  8. Parameter spaces • Simulation of complex systems • intractable: a certain timestep in a simulation can only be reached by starting at t=0 • NP complete: time and space requirements grow exponentially with problem size • Explicit simulation by a guided search through parameter space required (non-deterministic algorithms: SA, CA, NN, LBM, etc.) R.G. Belleman

  9. Examples of parameter spaces • Molecular dynamics • picosecond timeresolution • docking involves searchthrough large problemspaces R.G. Belleman

  10. … more examples … • Finite Element Methods (FEM) • large (hierarchically) structured meshes • Lattice Boltzmann methods (LBM) • large 3D (hierarchically) structured grids • large parameter spaces • “Optimization” problems in general R.G. Belleman

  11. From data to knowledge • Analysis of data spaces is often difficult • no analysis methods known, or ill-posed • size of data sets too large or too complex Often data analysis or simulation runs can take days, sometimes weeks! R.G. Belleman

  12. Bring in the expert • Presentation is often the only way to obtain insight (note: not limited to visualization) Is it possible to make short cuts? E.g. by putting “an expert” in the loop? R.G. Belleman

  13. HITL • Human In The Loop • a.k.a. “interactive exploration” • a.k.a. “exploratory analysis” • a.k.a. “computational steering” • a.k.a. “problem solving environments” • a.k.a. “virtual laboratory” R.G. Belleman

  14. Interactive Exploration Environments Goal: providing an interactive environment that allows for the exploration of large data spaces. Distinction between static and dynamic environments. R.G. Belleman

  15. Interactive Static Exploration Environments (ISEE) • Exploring large time-invariant datasets • Multi-modal data representation • visualization • sonification • haptification? R.G. Belleman

  16. Interactive Dynamic Exploration Environments (IDEE) • Exploring dynamically changing data from “living” simulations • Changing parameters: What if...? • Requires time management: R.G. Belleman

  17. Time management Synchronous (lockstep) Asynchronous R.G. Belleman

  18. Prerequisites for an IEE • Why Virtual Reality? • Quality presentation • Informative, avoid clutter • Rapid update rate: for continuous perception: • > 10 fps for vision • > 20 cps for sound • > 1000 cps for haptics R.G. Belleman

  19. Prerequisites for an IEE • Intuitive interaction • increased functionality requires a well considered user interface • Real-time feedback: < 0.1 sec delay These often conflict one another. R.G. Belleman

  20. VR interaction techniques XiVE: X in Virtual Environments There is no “WIMP” for VEs. • XiVE swallows GUIs into a VE • allows existing applicationsto be used in VEs with nochanges R.G. Belleman

  21. VR interaction techniques Context Sensitive Speech Recognition Interaction with visual constructs can be hard in a VE. • Speech is a different modality • Adding context decreases WER (?) • Fast, intuitive interaction • “come here”, “make blue”, “increase size by 200%” R.G. Belleman

  22. VR interaction techniques SCAVI: Speech, CAVE and Vtk Interaction • Direct interaction with Vtk “actors” using pointer or voice • select, drag, scale, rotate, copy, paste, etc. • event handlers; when in focus, when dragged, when selected, when spoken to, etc. R.G. Belleman

  23. VR interaction techniques GEOPROVE: Geometric Probes for VEs • Measurements in VR • Uses probes consisting of markers: R.G. Belleman

  24. So how does all this work?Let’s look at a test case...

  25. Simulated vascular reconstructionin a virtual operating theatre

  26. Overview • Interactive virtual environments for the exploration of • Multi-dimensional datasets • Multi-parameter spaces (computational steering) • Visualization and interaction in Virtual Reality (VR) • Applied to a test case:simulated vascular reconstruction in VR R.G. Belleman

  27. VRE • The Virtual Radiology Explorer(VRE): • Static exploration of 3D medical datasets • Virtual Reality (VR) interface • CAVE at SARA, Amsterdam • Portable ImmersaDesk • Surface/volume rendering • Virtual endoscopy • PACS data and computinginterface • Data storage and processingon parallel system (IBM SP2) R.G. Belleman

  28. Vascular disease • Stenosis:Treatment: thrombolysis, balloon angioplasty, stent placement, endarterectomy, bypass • Aneurysm:Treatment: shunt, bypass R.G. Belleman

  29. The problem • Best treatment often not obvious • read: the parameter space • Human body is a complex structure • read: the data space • A treatment is not always best under all situations • read: combination of both R.G. Belleman

  30. Pre-operative planning R.G. Belleman

  31. Traditional treatment ofvascular disease R.G. Belleman

  32. Interactive simulated vascular surgery R.G. Belleman

  33. The Virtual Laboratory • Shared use of distributed computing resources:high performance computers, scanners, algorithms, etc. • Connected via high performance networks • Common infrastructure: the Virtual Laboratory • Multi-disciplinary scientific experimentation • Problem solving environments (PSE) • Time/location independent scientific experimentation • Collaborative scientific research For additional information... DutchGrid initiative: http://vlabwww.nikhef.nl/ R.G. Belleman

  34. Simulated Vascular Reconstruction • Simulated vascular reconstruction • Patient specific angiographydata • Fluid flow simulationsoftware • Simulation of reconstructivesurgical procedure in VR • Interactive visualization ofsimulation results in VR • Pre-operative planning • Explore multiple reconstructionprocedures R.G. Belleman

  35. Preprocessing • Segmentation of patient specific MRA/CTA scan • Isolates region of interest • Lattice Boltzmann grid generation • Defines solid and fluid nodes, inlet and outlet conditions R.G. Belleman

  36. Fluid flow simulation • Lattice Boltzmann Method (LBM) • Lattice based particle method • Regular lattice, similar to CT or MRI datasets • Spatial and temporal locality • Ideal for parallel computing • Allows irregular 3D geometry • Validated with experimentsand FE simulations • Non-compressiblehomogeneous fluid,laminar flow • Velocity, pressure and shearstress calculated fromparticle densities R.G. Belleman

  37. Interactive exploration in VR • Visualize simulation results • Flow field, pressure, shear stress • Real time • Interactive exploration • VR interaction to locateregions of interest • Interactive grid editing • Simulate vascularreconstruction procedure R.G. Belleman

  38. Interactive exploration in VR • Quantification in VR: GEOPROVEGeometric Probes for Virtual Environments R.G. Belleman

  39. Interactive vascular surgery R.G. Belleman

  40. Summary • Test case shows example of a Problem Solving Environment (PSE) • Shared use of distributed resources • Time/location independent collaborative experimentation • PSEs open new possibilities for collaborative scientific research • Grid initiatives (Globus) • Virtual Environments provide intuitive interface for the exploration of multi-dimensional datasets and parameter spaces R.G. Belleman

  41. Questions? For more info: http://www.science.uva.nl/~robbel/ or email robbel@science.uva.nl

More Related