CPSC 875

1. CPSC 875 John D. McGregor C 8 More Design

2. Modes

3. Modes • Each mode can lead to a very different internal path • The “state” pattern encapsulates the logic for a given state in a module and swaps out the state module with each change of mode • The module that encapsulates the entire state machine is a higher level

10. Timing • For example, the control period of the hapticdevices is 1ms to keep the smooth control response for the user, and the control period of the error detection is the slowest. The control rate for the manipulator motion control is 30Hz, which is limited by the maximum communication frequency (about 35Hz at 19200bps RS-232 baud rate) between the computer and the motor driven units. The control frequencies of the different software modules are follows: • Manipulator motion control: 30Hz • Error detection module: 1Hz • I/O control module: 10Hz • Haptic device control: 1000Hz • Surgical simulation: 200Hz

11. Component model • Physical, component model

12. Component model • A cisst component contains lists of provided interfaces, required interfaces, outputs, and inputs, as shown in Figure 1. • Each provided interface can have multiple command objects which encapsulate the available services, as well as event generators that broadcast events with or without payloads. • Each required interface has multiple function objects that are bound to command objects to use the services that the connected component provides. It may also have event handlers to respond to events generated by the connected component. • When two interfaces are connected to each other, all function objects in the required interface are bound to the corresponding command objects in the provided interface, and event handlers in the required interface become observers of the events generated by the provided interface. • The output and input interfaces provide real-time data streams; typically, these are used for image data (e.g., video, ultrasound).

13. Local/Global Component Manager

15. Creates a pipeline

19. Endoscope • Gaze contingent endoscope control is one of the various options offered as part of the main module. The center of the endoscopic camera image gets automatically aligned with the surgeon’s fixation point on the 3D screen, as long as a foot pedal is pressed. Consequently, two hardware components act as input (writer modules): • The foot pedal module reads the current state of the four pedals (pressed/released). • The eye tracker module processes the gaze position, obtained by the eye tracker glasses.

20. Endoscope - 2 • The endoscope is tracked automatically as long as the length of the vector is greater than a certain threshold. The main module directly talks via UDP to the robot’s hardware controller. The robots have a clock cycle of 6.5ms, which means that in every interval at least one set of joint positions needs to arrive at the hardware controller. This timing could not be met by a separate module that reads values from the blackboard and sends them to the robotic hardware, as the calculation of the trocar kinematics already takes about half of the cycle time. Nevertheless, joint values are written to the blackboard for further consumption, e.g., by the visualization module.

21. Eye tracker The hardware will be interfaced and read out in accordance with the device-specific timings. The obtained values are then published to the central storage instance, the blackboard. The eye tracker [3] is connected via FireWire to a Mac; the foot pedals are connected to the parallel port of the main PC, which is running a standard Linux. All necessary pre-processing steps, e.g., a recursive time-series filtering [6] to smooth the approximately 400 values/sec obtained by the eye tracker, are performed outside and thus relieves the main module.

22. Blackboard • Besides the already mentioned data, the blackboard holds also calibration data of the surgical instruments, spatial calibration data of the robot bases, and the joint angles of each robot.

23. Display • The scenario involves two modules that act as readers: • The 3D display module acquires two video streams from the endoscopic camera. After de-interlacing, correction of brightness and size, the images are displayed at 25fps on the stereo screen. It’s running on a Windows machine and also reads the current (smoothed) gaze point to visualize its position on the screen. The visual feedback to the operator improves operability. • The visualization module shows a 3D environment of the scene, including robot and instrument movements, the operating table, and the master console. To update the joint angles of the models, this module reads the calibration data and the joints from the blackboard at 25Hz.

24. Latency • The main module directly talks via UDP to the robot’s hardware controller. The robots have a clock cycle of 6.5ms, which means that in every interval at least one set of joint positions needs to arrive at the hardware controller. This timing could not be met by a separate module that reads values from the blackboard and sends them to the robotic hardware, as the calculation of the trocar kinematics already takes about half of the cycle time. Nevertheless, joint values are written to the blackboard for further consumption, e.g., by the visualization module.

25. Yet another architecture

26. Service-oriented • http://msdn.microsoft.com/en-us/library/aa480021.aspx • Service • A Component capable of performing a task. A WSDL service: A collection of end points (W3C). • A type of capability described using WSDL (CBDI). • A Service Definition • A vehicle by which a consumer's need or want is satisfied according to a negotiated contract (implied or explicit) which includes Service Agreement, Function Offered and so on (CBDI).

27. Web service • A software system designed to support interoperable machine-to-machine interaction over a network. It has an interface described in a format that machines can process (specifically WSDL). Other systems interact with the Web service in a manner prescribed by its description using SOAP messages, typically conveyed using HTTP with XML serialization in conjunction with other Web-related standards (W3C). • A programmatic interface to a capability that is in conformance with WSnn protocols (CBDI).

28. Service oriented architecture • A set of components which can be invoked, and whose interface descriptions can be published and discovered (W3C). • The policies, practices, frameworks that enable application functionality to be provided and consumed as sets of services published at a granularity relevant to the service consumer. Services can be invoked, published and discovered, and are abstracted away from the implementation using a single, standards-based form of interface. (CBDI)

31. Qualities • Enabled by Web services • Technology neutral Endpoint platform independence.   • Standardized Standards-based protocols.   • Consumable Enabling automated discovery and usage.

32. Qualities • Enabled by SOA • Reusable Use of Service, not reuse by copying of code/implementation. • Abstracted Service is abstracted from the implementation.   • Published Precise, published specification functionality of service interface, not implementation.   • Formal Formalcontract between endpoints places obligations on provider and consumer.   • Relevant Functionality presented at a granularity recognized by the user as a meaningful service.

33. Benefits • There is real synchronization between the business and IT implementation perspective. • A well formed service provides us with a unit of management that relates to business usage. • When the service is abstracted from the implementation it is possible to consider various alternative options for delivery and collaboration models.

34. Preliminary services • <Move/Absolute/Joint space>: Moving an MM to a desired position in its joint space. • <Move/Relative/Joint space>: Moving an MM to a desired position in its joint space with respect to its current position. • <Detect tool tip>: Detecting the position of a tool tip in the microscope image. • <Undo Motion/Fast>: Returning the specified MM to its previous known position. • <Register/Coarse>: Registering each MM to the microscope coordinates using the stereo tracking system (sub-milimeter accuracy).

35. Intermediate services – <Autofocus/Passive>: Automatic focusing on an object or a region of interest (ROI) in the image by moving the objective up/down. – <Autofocus/Active>: Bringing the tool tip to focus, automatically. – <Track tool tip>: Tracking the tool tip(s) in real-time in microscope image. – <Calibrate/Camera>: Calibrating the microscope images, i.e. finding pixels sizes, skewness and image rotation. It can be automatic, semi-automatic or manual. – <Register/Fine>: Registering each MM to the microscope coordinates (sub-micron accuracy). – <Move/Absolute/Cartesian space>: Moving an MM to a desired position in the reference coordinate system. – <Move/Relative/Cartesian space>: Moving an MM to a desired position with respect to its current position in the reference coordinate system.

36. Advanced services – <Coordinated Motion>: All of the tools are moved in the same direction along one of the axes (X,Y or Z), while the objective tracks the same path to keep all the tool tips in focus. – <Click & Locate>: Locating the tip of a tool at a point specified by the user on the microscope image using calibration/registration information and/or visual servoing. – <Move/Safe>: Moving while avoiding collisions, following a diagonal path (to avoid breaking micropipettes). – <Undo Motion/Safe>: Undoing the last motion, avoiding collisions, following a diagonal path (to avoid breaking micropipettes). – <Refine calibration/registration>: Refining the calibration/registration information using the tool tip tracking data or user clicks. – <Touch water surface>: Moving the objective down, stopping when water/lens contact is detected [13]. – <Retract Tool>: Retracting tool from the field of view. – <Change Tool>: Fully retracting the tool and micromanipulator, in order to change the tool. – <Change Objective>: Changing the objective lens, focusing on the same spot. – <Master-Slave Control>: The selected MM follows the master position.

37. State machine

38. Next Steps • Develop an AADL model of one additional architectural pattern. Include information that shows the qualities enhanced/degraded by the pattern. • Add to the AADL model of the surgical robot.