---

Draft Report for Healthcare Robotics

(Okamura, Henriksen, Mataric, eds.) - final version to be presented to congress in 2009 contains several passages that speak to the importance (and current lack) of haptics in medical robotics.

• CCC-medical-healthcare-v7.pdf

Several sections of the report speak to the need for haptic information in robotic surgery.

Section 3.2.6:

Another example is the use of virtual fixtures in surgery. The term ``virtual fixture'' refers to a general class of guidance modes, implemented in software and executed by a robotic device, that help a human- machine collaborative system perform a task by limiting movement into restricted regions and/or influencing movement along desired paths. Virtual fixtures can enhance robot-assisted minimally invasive surgery by ensuring that the manipulator inside the patient does not enter forbidden areas of the workspace, such as organ surfaces that should not be cut and delicate tissue structureable to override the virtual fixture if desired.

Section 3.2.7:

One of the most useful forms of imaging is magnetic resonance imaging (MRI). The design of MRI-compatible robots is especially challenging because MRI relies on a strong magnetic field and radio frequency (RF) pulses, and so it is not possible to use components that can interfere with, or be susceptible to, these physical effects. This rules out most components used for typical robots, such as electric motors and ferromagnetic materials. In addition, surgery or interventional radiology inside an imager places severe constraints on robot size and geometry, as well as the nature of the clinician-robot interaction. Novel materials, actuation mechanisms, and sensors are required to create robots that can be seamlessly integrated into the interventional suite.

Section 4.5, Robust, High-Fidelity Sensors

We focus here on two types of sensing especially important for medicine and health care: biocompatible/implantable sensors and force/tactile sensing. These sensors, along with perception algorithms, are often necessary to give state of a caregiver/physician, the patient, and (in some cases) the environment.
Biocompatible/implantable sensors would be a great catalyst to major advancements in this field. The close physical interaction between robots and patients requires systems that will not harm biological tissues or cease to function when in contact with them. In surgery, mechanisms must be designed that will not unintentionally damage tissues, and sensors need to be able to function appropriately in an environment with wetness, debris, and variable temperature. For prosthetics, sensors and probes must access muscles, neurons, and brain tissue and maintain functionality over long periods without performance degradation. These sensors and devices must be designed with medical and health robotics applications in mind, in order to define performance requirements.
When robots work in unstructured environments, especially around and in contact with humans, using the sense of touch is crucial to accurate, efficient, and safe operations. Tactile, force, and contact data is required for informed manipulation of soft materials, from human organs to blankets and other objects in the household. It is particularly challenging to acquire and interpret spatially distributed touch information, due to the large area and high resolution required of the sensors. Current sensors are limited in robustness, resolution, deformability, and size.

Section 4.8 Physical Human-Robot Interaction

When clinicians or patients use robots to interact with environments that are remote in distance or scale, the operator needs to have a natural interface that makes the robot seem "transparent". That is, the operator of a surgical robot, prosthesis, or rehabilitation robot should feel as if he or she is directly manipulating a real environment rather than interacting with a robot. Haptic (force and tactile) displays give feedback to the user that is akin to what he or she feels in the real world. This haptic feedback can improve performance in terms of accuracy, efficiency, and comfort.

Networked Telerobotics (Springer Handbook)

D. Song, K. Goldberg, N.Y. Chong, "Networked Telerobots," Cha. 32 in Springer Handbook of Robotics, B. Siciliano and O. Khatib eds., Springer 2008.

Augmented reality for dynamic operations

Song et al. review the state of the art in networked telerobotics, including a couple of cases (e.g. [Belusov05]) in which dynamic grasping operations were performed using a combination of vision and dynamic models for motion prediction in an augmented-reality display.

[Belusov05] I. Belousov, S. Chebukov, V. Sazonov: Web-based teleoperation of the robot interacting with fast moving objects, IEEE Int. Conf. Robot. Autom. (ICRA) (2005) pp. 685–690.

Haptics (Springer Handbook)

B. Hanaford and A. Okamura, Haptics, Cha. 30 in in Springer Handbook of Robotics, B. Siciliano and O. Khatib eds., Springer 2008.

Hanaford and Okamura review many issues in haptics including key results from Psychophysics (receiver operating curve ROC, just noticeable difference, JND) exploratory procedures (EP), etc.

Simulators and haptics in medical applications

"Simulators have proven proven highly effective in developing minimally invasive surgery skills [30.14, 30.15]."

30.14 C. Basdogan, S. De, J. Kim, M. Muniyandi, M.A. Srinivasan: Haptics in minimally invasive surgical simulation and training, IEEE Comput. Graphics Appl. 24(2), 56–64 (2004)
30.15 P. Strom, L. Hedman, L. Sarna, A. Kjellin, T. Wredmark, L. Fellander-Tsai: Early exposure to haptic feedback enhances performance in surgical simulator training: a prospective randomized crossover study in surgical

"While most conventional linear and nonlinear FE algorithms cannot be run in real time, methods such as preprocessing can allow them to run at haptic rates [30.49]... Major challenges in this field include the modeling of connective tissue supporting the organ, friction between instruments and tissues, and topological changes occurring during invasive surgical procedures"

30.49 S.P. DiMaio, S.E. Salcudean: Needle insertion modeling and simulation, IEEE Trans. Robot. Autom. 19(5), 864–875 (2003)

Vibration feedback can also be used to provide information about patterned textures, roughness, and other phenomena that have clear vibratory signals. ... Kontarinis and Howe [30.61] showed that damaged ball bearings could be identified through vibration feedback, and Dennerlein et al. [30.62] demonstrated that vibration feedback improved performance over no haptic feedback, for a telemanipulation task inspired by undersea field robotics. ... In virtual environments, Okamura et al. [30.63] displayed vibrations modeled based on textures and puncture of a membrane. Similar to the event-based haptics above, the vibration waveforms were modeled based on earlier experiments and played open loop during interaction with the virtual environment.

Medical Robotics and Computer Integrated Surgery (Springer Handbook)

R.H. Taylor, A. Menciassi, G. Fichtinger, P. Dario Cha 52, in Springer Handbook of Robotics, B. Siciliano and O. Khatib eds., Springer 2008

52.2.3 Virtual Fixtures and Human–Machine Cooperative Systems

...the fact that a computer is actually controlling the robot’s motion creates many [new] possibilities. The simplest is a safety barrier or no-fly zone, in which the robot’s tool is constrained from entering certain portions of its workspace. More sophisticated versions include virtual springs, dampers, or complex kinematic constraints that help a surgeon align a tool, maintain a desired force, or maintain a desired anatomical relationship. The Acrobot system shown in Figs. 52.5c,d represents a successful clinical application of the concept, which has many names, of which *virtual fixtures* seems to be the most popular [52.35, 36].

As the ability of computers to model and follow along surgical tasks improves, these modes will become more and more important in surgical assistant applications. Figure 52.6 illustrates the overall concept of human–machine cooperative systems in surgery, and Fig. 52.7 illustrates the use of registered anatomical models to generate constraint-based virtual fixtures. These approaches are equally valid whether the surgeon interacts with the system through classical teleoperation or through hands-on compliant control.

52.35 M. Li, M. Ishii, R.H. Taylor: Spatial motion constraints in medical robot using virtual fixtures generated by anatomy, IEEE Trans. Robot. 23, 4–19 (2007)
52.36 S. Park, R.D. Howe, D.F. Torchiana: Virtual fixtures for robotic cardiac surgery, Fourth Int. Conf. Med. Image Comput. Computer-Assisted Intervention (2001)

Teleoperation (Springer Handbook)

G. Niemeyer, C. Preusche, G. Hirzinger, Cha. 31. in Springer Handbook of Robotics, B. Siciliano and O. Khatib eds., Springer 2008

A reminder that the "first transcontinental telerobotic surgery" was in 2001 [31.21] but did not include force feedback.

Comparisons of direct, shared and supervisory control for various applications with more or less time delay. Predictive simulation and sensor fusion.

Virtual fixtures in teleoperation

"A special application of shared control is the use of virtual fixtures [31.44]. Virtual elements, such as virtual surfaces, guide tubes, or other appropriate objects, are superimposed into the visual and/or haptic scene for the user... Capitalizing on the accuracy of robotic systems, while sharing control with the operator, telerobotic systems with virtual fixtures can achieve safer and faster operation [31.45]"

[31.21] J. Marescaux, J. Leroy, F. Rubino, M. Vix, M. Simone, D. Mutter: Transcontinental robot assisted remote telesurgery: Feasibility and potential applications, Ann. Surg. 235, 487–492 (2002) 31.44 L. Rosenberg: Virtual fixtures: Perceptual tools for telerobotic manipulation, Proc. IEEE Virtual Real. Int. Symp., New York (Seattle 1993) pp. 76–82
31.45 J.J. Abbott, P. Marayong, A.M. Okamura: Haptic virtual fixtures for robot-assisted manipulation, Proc. 12th Int. Symp. Robot. Res., Vol. 28 (2007) pp. 49–64

-- Started MarkCutkosky - 26 Dec 2008