1 / 45

Seeing and Sensing: displays, sensors, and systems for medical interaction

Seeing and Sensing: displays, sensors, and systems for medical interaction. Michael Halle, PhD Surgical Planning Lab Department of Radiology, BWH mhalle@bwh.harvard.edu. Acknowledgements. S. Pieper M. Shenton F. Talos A. Tannenbaum C. Tempany S. Warfield W. Wells C.F. Westin

glenna
Download Presentation

Seeing and Sensing: displays, sensors, and systems for medical interaction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Seeing and Sensing: displays, sensors, and systems for medical interaction Michael Halle, PhD Surgical Planning LabDepartment of Radiology, BWH mhalle@bwh.harvard.edu

  2. Acknowledgements S. Pieper M. Shenton F. Talos A. Tannenbaum C. Tempany S. Warfield W. Wells C.F. Westin and the rest of the SPL… N. Aucoin P. McL Black K. ChinzeiE. Grimson S. Haker K. Hynynen F.A Jolesz R. Kikinis L. O’Donnell NCRR, NCI, NLM, NSF, DOD et al. http://www.spl.harvard.edu

  3. Computer hardware and software is becoming more advanced, but getting dynamic information in, and meaningful information out, is still hard. Observation:

  4. The I/O problem

  5. Result • poor integration of important data + information may not completely and accurately get to physician = doctor cannot make most informed decisions & innovations aren’t easy

  6. What we’ll do about it • new display technologies to understand complex data • software systems that simplify data exchange between different systems • collaborative software tools

  7. Who am I? • MIT Computer Science (BS) • World’s most advanced 3D displays at MIT Media Lab (MS, PhD) • Systems and graphics design • Long collaboration with SPL • Started full-time 1995

  8. Holography and 3D

  9. Graphics first • For the last ten years, better graphics has been more important than 3D displays • Now, we can start to build systems that integrate the best of 2D and 3D displays

  10. Several candidate technologies • Volumetric displays(eg, Actuality Systems) • Space-filling • Collaborative • No opacity Provided by Nakajima, Atsumi et al.

  11. Female pelvis on 3D display (Hoyte)

  12. Stereo glasses / head-mounted display • Can be cheap • High quality imagery • Can’t register with real world easily • Only one user

  13. Lenticular and microlens displays • Multiple viewers & walkaround • High image quality • Can be difficult to produce imagery Lenticular Microlens

  14. Display approach • None of these displays are perfect for all tasks • Computer and monitor still central • Use each, appropriately • Maintain mental context when switching

  15. Maintaining context Provided by Leventon et al.

  16. Input devices: data fusion Provided by Leventon et al.

  17. Input devices • Good displays need good data • Input means interfacing • Interfacing usually means translation • Analogy: language and society • Power of common language

  18. Communication community Central Switchboard translator translator 2D vis. probe Effective path

  19. Communication community Central Switchboard translator translator 3D vis. probe 2D vis. translator other contributors

  20. Advantages • New “contributors” are simple • Experimentation is easier • More robust • Easy to monitor system • Richness of interaction & computation

  21. Extension: collaborative interface • Interface communication • Shared data sources • “Shared mental context” • (Early version a decade ago!)

  22. Status • Communication mechanism prototype • Proof-of-concept device interface • Short term: integrate with Slicer • 3-4 months: prototype collaborative interface • 1-2 years: 3D display hardware on site • Other collaborative interfaces

  23. Conclusion • For each tool, its task • Technology with purpose • Community communication • The future, piece by piece

  24. Contact information Michael HalleSurgical Planning LabDept. of RadiologyBWHmhalle@bwh.harvard.edu

  25. Intraoperative Changes Craniotomy Initial Provided by Warfield et al.

  26. Visualization • Data Acquisition • Data Analysis • Rendering too little too much just right

  27. Core technologies • Data acquisition & preprocessing • Segmentation • Registration • Analysis • Model building • Visualization • Interaction not a linear pipeline!

  28. Data acquisition example: diffusion tensor imaging Provided by Westin, Meier, et al.

  29. Diffusion Tensor MRI Tractography Provided by Westin, Mamata, et al.

  30. Preprocess: feature enhancement Example: 3D adaptive filtering of vessels Original Filtered Courtesy CF. Westin

  31. Segmentation • cannot be avoided • based on many criteria • must cope with imperfect signal

  32. EM-MRF Segmentation EM EM-MRF [Kapur98] T. Kapur, W.E.L. Grimson, W.M. Wells III, R. Kikinis. Enhanced Spatial Prior for segmentation of Magnetic Resonance Imagery, MICCAI98, Cambridge, MA, October 1998. [Kapur99] Tina Kapur. Model based three dimensional Medical Image Segmentation Ph.D. Thesis, MIT EECS Dept, 1999.

  33. Registration • Multiple input channels • Different timepoints • Different patients • Patient to atlas

  34. ATM Classification Combine statistical classification and registration of a digital anatomical atlas Brain atlas Registration Template Distance Transforms prototypes Statistical Classification Segmented images Grey value images

  35. Atlas Mapping atlas knowledge into patient data sets Provided by Kaus, Nabavi, Warfield

  36. Cartilage Thickness Mapping Courtesy Warfield, Winalski

  37. Mapping of all information: Brain Morphology Vessel Morphology Brain Function (GM, WM) Pathology External Information Not automatic! Integrated Visualization

  38. Interaction • Bring the results to the physician • People are interface too • Changing the look of medicine

  39. Intra-operative MRI • Precise targeting of • lesion margins • surrounding anatomical structures • image updates as needed - (“brain shift”) • accurate location of residual tumor • Immediate detection of complications (hemorrhage, swelling, ischemia)

  40. Volumetric FE Deformation Provided by Warfield, Ferrant et al.

  41. 3D-Navigation Provided by F. Talos

  42. Robot & Prostate Focused Ultrasound (FUS) Novel Interventional Activities

  43. FUS Setup Provided by Hynynen et al.

  44. 3D Slicer • Surgical simulation and navigation software • Displays multi-modality images in 2D and 3D

More Related