330 likes | 432 Views
OpenIGTLink Update. NA-MIC Summer Project Week 2010 Junichi Tokuda, Ph.D. Department of Radiology Brigham and Women’s Hospital. NCIGT National Center for Image Guided Therapy http:// ncigt.org. Backgrounds. Hardware/software in IGT Position tracking sensors MR/CT/Ultrasound scanners
E N D
OpenIGTLink Update NA-MIC Summer Project Week 2010 Junichi Tokuda, Ph.D. Department of Radiology Brigham and Women’s Hospital NCIGT National Center for Image Guided Therapy http://ncigt.org
Backgrounds • Hardware/software in IGT • Position tracking sensors • MR/CT/Ultrasound scanners • Robotic devices • Planning and navigation software • Data types • Images • Positions / Transforms • Software / hardware status, etc… • Communication standard for IGT?
Goals of OpenIGTLink • Available • Research and commercial • Simple • From embedded system to HPC • Extensible • Variety of data types • Reliable • Used clinically!
Protocol − Overview • Message-based Protocol • No session / messages are independent • Allows application-specific message types Header Body IMAGE Type Body TRANSFORM Type Body COMMAND Type Body …. Body Size
Protocol − Body • Position / Transform x y z ox oy oz w position orientation (quaternion) tx ty tz sx sy sz nx ny nz x y z
Protocol − Body • Image • 2D/3D • Integer / floating point • Scalar / vector • Partial Image update Generic info size transform partial volume index/size Pixel data
The OpenIGTLink Library • Multi-platform C/C++ Library • Windows, Linux, Mac OS X, Solaris • Used in QNX, iPhone OS, vxWorks etc.. Application C++ socket & thread class C++ message class C message structure
The OpenIGTLink Library • Code snippet (9 lines to send position)
The OpenIGTLink Library • Various “hello world” examples • Tracker simulator • Imager simulator • Data receive server • etc.
3D Slicer OpenIGTLink I/F • Import to / export from MRML scene • Image (vtkMRMLScalarVolumeNode) • Transform (vtkMRMLLinearTransformNode) Ethernet Tracker 3D Slicer MRML Scene OpenIGTLink Module 1 Connector node 1 LinearTransform Imager Module 2 Image OpenIGTLink Connector node 2 … LinearTransform Ext. software Connector node N Image Module N OpenIGTLink
3D Slicer OpenIGTLink I/F • Import / export
3D Slicer + BrainLAB Elhawary H, Norton I, Patel P, Rigolo L, Papademetris X, Hata N, Golby AJ (BWH / Yale / BrainLAB)
3D Slicer + BrainLAB • Research prototype (Slicer) + FDA-approved IGS system (BrainLAB) IGS system Research Prototype BrainLAB VectorVision 3D Slicer VVLink OpenIGTLink Bridge Software BioImage Suit VVLink OpenIGTL Image / Tracking data (VVLink) Image / Tracking data (OpenIGTLink)
MRg Prostate Biopsy Robot Fischer G, Tokuda J, Song S, Iordachita I, DiMaio S, Cho N, Csoma C, Bleivns J, Burdette EC, Fichtinger G, Hata N, Tempany C (Johns Hopkins, BWH, Queen’s, Acoustic MedSystems)
MRg Prostate Biopsy Robot • Integration of Slicer, robot and scanner 3D Slicer Robot Proxy Controller Command OpenIGTLink OpenIGTLink Pipe Pipe Current position Scanner Proxy MRI Scanner Real-time image OpenIGTLink Proprietary Proprietary Imaging slice position
Neurosurgical Manipulator Arata J, Tada Y, Yamada A, Tanaka Y, Fujimoto H (Nagoya Institute of Technology, Japan)
Neurosurgical Manipulator • System diagram
Motion Correction for MRgFUS • MR Image Motion Correction by US-based motion tracking MRI Scanner No Motion w/ motion Uncorrected w/ motion Corrected Scanner Hardware OpenIGTLink Schwartz BM, McDannold N Brigham and Women’s Hospital Imaging slice position
Cardiac EP Catheter + 3D Slicer Jagadeesan J, Hata N Brigham and Women’s Hospital
Cardiac EP Catheter + 3D Slicer • Matab/Simulink in backstage…
Next Steps • OpenIGTLink Protocol Version 2 (draft) • Backward compatibility with version 1 • Query support • Better IGS system support • Other message types?
Query Support • “Get” message to request data • “Start”, “stop” and “pause” messages to control data streaming • “Status” message to respond to query
IGS Systems Support • Remote Image List • Display a list of images available in the remote software • Tracking data • Multi-channel, device name • Fiducial, trajectory • Share surgical planning information
Other Message Types? • Remote Execution • Run Tcl/Python on 3D Slicer from remote • Remote procedure call • XML data / Associative array • General data types • Versatility but compromised performance
MATLAB Interface • MATLAB – 3D Slicer communication
OpenIGTLink + ZeroMQ • A layer between OpenIGTLink and TCP/IP • Support different message patterns • Point-to-point • Publish-subscribe • Request-reply • Parallelized pipeline • Proposed in CTK OpenIGTLink ZeroMQ TCP IP Ethernet http://www.commontk.org/index.php/Documentation/Messaging by Patrick Cheng (Georgetown)
Acknowledgements • BWH. Nobuhiko Hata, Ron Kikinis, Clare Tempany, Alexandra Golby, Haiying Liu, Tina Kapur, Jayender Jagadeesan, Haytham Elhawary, Benjamin M. Schwartz, … • Isomics, Inc. Steve Pieper • Kitware, Inc. Luis Ibanez • JHU. Sam S. Song, Nathan Cho, Iulian Iordachita, Peter Kazanzides, … • Queen’s. Univ. Gabor Fichitinger • Georgetown Univ. Ziv Yaniv, Patrick Cheng, Kevin Cleary • Acoustic MedSystems Inc. Jack Blevins, E. Clif Burdette • Worcester Polytechnic Institute. Gregory Fischer • AIST, Japan. Kiyoyuki Chinzei • Nagoya Inst of Tech. Junpei Arata, Atsushi Yamada, Hiroaki Kozuka, Hideo Fujimoto • Nagoya Univ. Yuichiro Hayashi
Grant Support • This work is supported by 1R01CA111288, 5U41RR019703, 5P01CA067165, 1R01CA124377, 5P41RR013218, 5U54EB005149, 5R01CA109246, R01EB006494 and R21EB007770 from the NIH. This study is also in part supported by NSF 9731748, CIMIT, Intelligent Surgical Instruments Project of METI (Japan).
Publications • Tokuda J., Fischer G.S., Papademetris X., Yaniv Z., Ibanez L., Cheng P., Liu H., Blevins J., Arata J., Golby A.J., Kapur T., Pieper S., Burdette E.C., Fichtinger G., Tempany C.M., Hata N. OpenIGTLink: an open network protocol for image-guided therapy environment. Int J Med Robot. 2009 Dec;5(4):423-34. PMID: 19621334. PMCID: PMC2811069. • Tokuda J., Fischer G.S., DiMaio S.P., Gobbi D.G., Csoma C., Mewes P.W., Fichtinger G., Tempany C.M., Hata N. Integrated navigation and control software system for MRI-guided robotic prostate interventions. Comput Med Imaging Graph. 2010 Jan;34(1):3-8. PMID: 19699057. PMCID: PMC2815337. • Arata J., Kozuka H., Kim H.W., Takesue N., Vladimirov B., Sakaguchi M., Tokuda J., Hata N., Chinzei K., Fujimoto H. Open core control software for surgical robots. Int J Comput Assist Radiol Surg. 2010 May;5(3):211-20. PMID: 20033506.
For more information… OpenIGTLink ML: openigtlink@bwh.harvard.edu
History • Dec 2007: Discussed in IGT Week • Jan 2008: Defined in All Hands Meeting • May2009 : Supported in 3D Slicer 3.4 • Jun 2010: Protocol Extension
3D Slicer OpenIGTLink I/F • Performance Optimization by Steve Before After