On this page... (hide)
1. Vision for Mobile Manipulation
A robotic hand reaches into a bag and gropes about, and after a moment it emerges with a set of keys. While holding the ring firmly with two fingers, the others are used to separate and grip one key in particular. The robot then begins to insert the key into a lock. The lock is old, and the key worn, and the hand feels the key begin to slip in its grasp. After tightening the grip and changing the key's angle in the lock, a satisfying sound and tell-tale vibration indicates the key can be turned and the door opened.
Our projects in mobile grasping and manipulation rely on biologically inspired solutions to hand design and sensor integration and interpretation challenges. Robotic hands need to be strong and lightweight, and allow for variable compliance tailored to the task at hand. At the same time "mechanical intelligence" which allows the hand to passively conform to objects using a small set of actively controlled grasp types relieves a large portion of the control burden. Tactile sensing suites on hands and arms, in contrast to physically separated vision systems, are still relatively uncommon, fragile, and limited in scope and coverage. As we improve the sensors themselves and cover more of our robots' hands and arms, we need to explore how to effectively use that information to improve grasping and manipulation.
2. Projects
A number of advances will be needed to realize the goals of robust grasping and manipulation on mobile robotic systems. Some of our current work towards those ends is outlined below:
- ReachBot project with Stanford ASL & NASA to locomote via manipulation in rough terrain (starting in 2021)
- TRI Household Robot
- SLIM is an ambidexterous hand and wrist developed under the TRI project. It has a very low profile for reaching in clutter, very low fingertip inertia, and a wide range of available grasp types.
- AdhesionAssistedGrasping
- Tactile sensors and artificial skins are being designed with sensing capabilities inspired by that of humans. Low frequency normal and shear pressure sensing are being combined with high frequency dynamic tactile sensing to produce a single skin which can detect light touch, high pressure, and slip.
- Underwater Manipulation -- Underwater manipulation can be challenging because of water currents and lubrication. We are developing an underwater robotic hand for an humanoid diver, in collaboration with the C.S. Dept, as well as investigating suction flow as a way to improve acquisition and retention of objects underwater.
- Under-actuated compliant hands are being developed for a couple of applications, including underwater exploration and mobile manipulation (DARPA Arm-H). Common themes include the ability to selectively lock or release joints and to vary the stiffness of joints to increase the range of poses that the hands can assume.
- MultiLimbedClimbing A multi-limbed robot climbing on a challenging rocky wall is essentially manipulating its own body and grasping the surface with its hands and feet. We are developing hand/foot end-effectors in a collaborative project with UC Santa Barbara and Duke.
- Arm-H (private page) A selectively compliant, under-actuated hand is being designed in collaboration with SRI and Meka Robotics. The selectively compliant mechanism should allow the hand to be quite dexterous despite being under-actuated, and allows for simple control of how the hand passively adapts to and grasps objects. The latest in our work on tactile sensing is being incorporated to give the hand the ability to feel what it interacts with and respond accordingly.
- Tactile event sensing for grasping and manipulation. We have developed a suite of normal pressure and dynamic tactile sensors for an underactuated hand mounted on a typical industrial robot arm. This setup can identify and characterize tactile events, such as contact and slip. A particular goal is to distinguish between hand/object and object/world slippage. One application of this work is for a Robot Drilling system.
3. Publications
(from MobileManipulationPubs)
Michael Lin, Hao Li, Chengi Xing, Mark Cutkosky, “Navigation and 3D Surface Reconstruction from Passive, Whisker Sensing,” International Journal of Robotics Research, in review.
Brouwer, D., Citron, J., and Cutkosky M., "Leveraging Non-Prehensile Tactile Data for Object Retraction in Constrained Clutter Using Imitation Learning," IEEE ICRA@40 Extended Abstract (preprint)
Brouwer, D., Citron, J., Choi, H., Lepert, M., Lin, M., Bohg, J., and Cutkosky M., "Tactile-Informed Action Primitives Mitigate Jamming in Dense Clutter," IEEE ICRA 2024 DOI: 10.1109/ICRA57147.2024.10610224
Chen, T. G., Newdick, S., Di, J., Bosio, C., Ongole, N., Lapôtre, M., Pavone, M., Cutkosky, M., Locomotion as manipulation with ReachBot. Sci. Robot. 9, eadi9762 (2024). DOI:10.1126/scirobotics.adi9762
Choi, H., Cutkosky, M. R., Stanley, A. A., "Integrated Pneumatic Sensing and Actuation for Soft Haptic Devices", IEEE Robotics and Automation Letters, 2023, Accepted
Lin, M. A., Reyes, E., Bohg, J., & Cutkosky, M. R. (2022, October). Whisker-Inspired Tactile Sensing for Contact Localization on Robot Manipulators. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 7817-7824). IEEE.
S. Newdick, N. Ongole, T.G. Chen, E. Schmerling, M.R. Cutkosky, M. Pavone, "Motion Planning for a Climbing Robot with Stochastic Grasps," Proc. IEEE ICRA 2023.
T. G. Chen, B. Miller, S. Schneider-Newdick, A. Bylard, M. Pavone, and M. R. Cutkosky, “ReachBot: A Small Robot with Exceptional Reach for Rough Terrain,” in Proc. IEEE Conf. on Robotics and Automation, 2022.
S. Schneider, A. Bylard, T. G. Chen, P. Wang, M. R. Cutkosky, and M. Pavone, “ReachBot: A Small Robot for Large Mobile Manipulation Tasks,” (arxiv) in IEEE Aerospace Conference, Big Sky, Montana, 2022.
Tony G. Chen, Abhishek Cauligi, Srinivasan Arul Suresh, Marco Pavone, Mark Cutkosky, Testing Gecko-Inspired Adhesives With Astrobee Aboard the International Space Station: Readying the Technology for Space, IEEE Robotics and Automation Magazine (Sept. 2022). IEEE RAM Best Paper Awardee, May 2023
- Related news article and video: Gecko gripper tested aboard the ISS.
Ruotolo, W., Brouwer, D. and Cutkosky, M.R., From grasping to manipulation with gecko-inspired adhesives on a multifinger gripper." Science Robotics 6.61 (2021): eabi9773.
Ruotolo, W., "Design Principles for Manipulation with Astrictive Contact", PhD thesis, Stanford University, August 2021
Lin, M.A.*, Thomasson, R.*, Uribe, G., Choi, H., and Cutkosky M.R. "Exploratory Hand: Leveraging Safe Contact to Facilitate Manipulation in Cluttered Spaces". IEEE RA Letters., 2021. https://doi.org/10.1109/LRA.2021.3068941
Frishman, S., Di, J., Karachiwalla, Z., Black, R. J., Moslehi, K., Smith, T., ... & Cutkosky, M. R. (2021, September). A Multi-Axis FBG-Based Tactile Sensor for Gripping in Space. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 1794-1799). IEEE.
Ruotolo, W., Thomasson, R., Herrera, J., Gruebele, A., and Cutkosky, M.R., "Distal Hyperextension is Handy: High Range of Motion in Cluttered Environments."IEEE RA-Letters", 2020. https://doi.org/10.1109/LRA.2020.2965914
Huh, T.M., Choi, H., Willcox, S., Moon, S. and Cutkosky, M.R., "Dynamically Reconfigurable Tactile Sensor for Robotic Manipulation". "IEEE RA Letters", 2020. https://doi.org/10.1109/LRA.2020.2972881, (preprint)
Gruebele, A., Roberge, J.P., Zerbe, A., Ruotolo, W., Huh, T.M., Cutkosky, M.R. "A Stretchable Capacitive Sensory Skin for Manipulation in Cluttered Environments". "IEEE RA-Letters", 2020.
Wu, X.A.*, Huh, T.M.*, Sabin, A., Suresh, S.A. and Cutkosky, M.R. "Tactile Sensing and Terrain-Based Gait Control for Small Legged Robots". IEEE Transactions on Robotics, 2019. https://doi.org/10.1109/TRO.2019.2935336, (preprint)
Ruotolo, W., Roig, Frances S. and Mark R. Cutkosky, "Load-Sharing in Soft and Spiny Paws for a Large Climbing Robot," accepted for IEEE RA-Letters, 2019. (preprint)
Wang, S., Jiang., H., Huh, T.M., Sun, D., Ruotolo, W., Miller, M., Roderick, W.R.T., Stuart, H., and Cutkosky, M.R., "SpinyHand: Contact Load Sharing for a Human-Scale Climbing Robot," accepted for ASME Journal of Mechanisms and Robotics (2019). (preprint)
Hashizume, J., Huh, T-M, Suresh, S., and Cutkosky, M.R., "Capacitive Sensing for a Gripper with Gecko-Inspired Adhesive Film," accepted for IEEE RA-Letters, 2019. (preprint)
Hauser, K., Wang, S. Cutkosky, M.R., "Efficient equilibrium testing under adhesion and anisotropy using empirical contact force models." IEEE Transactions on Robotics 99 (2018): 1-13. https://doi.org/10.1109/TRO.2018.2831722
Jiang, H., Wang, S. and Cutkosky, M.R., "Stochastic models of compliant spine arrays for rough surface grasping" The International Journal of Robotics Research June, 2018: https://doi.org/10.1177/0278364918778350
Huh, T. M., Liu, C., Hashizume, J., Chen, T. G., Suresh, S., Chang, F. K., and Cutkosky, M.R. "Active Sensing for Measuring Contact of Thin Film Gecko-Inspired Adhesives," Robotics and Automation Letters, 2018. DOI: 10.1109/LRA.2018.2851757, (preprint)
Wang, S., Jiang, H. and Cutkosky, M.R., "Design and modeling of linearly-constrained compliant spines for human-scale locomotion on rocky surfaces," The International Journal of Robotics Research, Vol 36, Issue 9, pp. 985 - 999, Published online July 2017. DOI: 10.1177/0278364917720019
Wang, S., Chung, S., Khatib, O., Cutkosky, M.R. SupraPed: Smart Staff Design and Terrain Characterization, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015.
Stuart, H., Wang, S., Khatib, O., Cutkosky, M. (2017). The Ocean One hands: An adaptive design for robust marine manipulation. The International Journal of Robotics Research. IJRR 2017
Khatib, O., Yeh, X., Brantner, G., Soe, B., Kim, B., Ganguly, S., Stuart, H., Wang, S., Cutkosky, M., Edsinger, A., Mullins, P., Barham, M., Voolstra, C., Salama, K., L’Hour, M., Creuze, V. (2016). “Ocean One: A Robotic Avatar for Oceanic Discovery.” IEEE Robotics & Automation Magazine. RAS 2016
Wu, X.A., Christensen, D.L., Suresh, S.A., Jiang, H., Roderick, W.R.T., and Cutkosky, M.R. "Incipient Slip Detection and Recovery for Controllable Gecko-Inspired Adhesion," Robotics and Automation Letters, 2017. DOI: 10.1109/LRA.2016.2636881.
Wang, S., Jiang, H., and Cutkosky, M.R. “A Palm for Rock Climbing Based on Dense Arrays of Micro-spines,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016. Best Paper Award on Safety, Security, and Rescue Robotics in Memory of MotohiroKisoi
Estrada, M., Hockman, B., Bylard, A., Hawkes, E.W., Cutkosky, M.R., and Pavone, M. Free-Flyer Acquisition of Spinning Objects with Gecko-Inspired Adhesives, Proc. 2016 IEEE Int’l Conf. Robotics and Automation. DOI: 10.1109/ICRA.2016.7487696
Wu, X.A., Huh, T.M., Mukherjee, R., Cutkosky, M.R. "Integrated Ground Reaction Force Sensing and Terrain Classification for Small Legged Robots," IEEE Robotics and Automation Letters, 2016. DOI: 10.1109/LRA.2016.2524073 IEEE RA-L 2016
Wu, X.A., Suresh, S.A., Jiang, H., Ulmen, J.V., Hawkes, E.W., Christensen, D.L., and Cutkosky, M.R. "Tactile Sensing for Gecko-Inspired Adhesion," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015. IEEE/RSJ IROS 2015 Best Conference Paper Award
Wang, S., Chung, S., Khatib, O., Cutkosky, M.R. "SupraPed: Smart Staff Design and Terrain Characterization," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015. (Attach:IROS2015_SupraPed Δ)
Stuart, H.S, Bagheri, M., Wang, S., Barnard, H., Sheng, A.L., Jenkins, M., Cutkosky, M.R. "Suction Helps in a Pinch: Improving Underwater Manipulation with Gentle Suction Flow," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015. (preprint Δ)
Heyneman, B. and Cutkosky, M.R., "Slip Classification for Dynamic Tactile Array Sensors," accepted for the International Journal of Robotics Research, 2015. (preprint)
Cutkosky, M.R. and Ulmen, J., Dynamic tactile sensing, in The Human Hand: A Source of Inspiration for Robotic Hands (R. Balasubrmanian and V. Santos, eds.), ch. 18, Berlin, Heidelberg: Springer Verlag, 2014.
Stuart, H.S., Wang, S., Gardineer, B., Christensen, D.L., Aukes, D. and Cutkosky, M.R., "A Compliant Underactuated Hand with Suction Flow for Underwater Mobile Manipulation," IEEE/ICRA 2014 (preprint).
Wu, X.A., Burkhard, N., Heynemann, B., Valen, R., and Cutkosky, M.R. "Contact Event Detection for Robotic Oil Drilling," IEEE/ICRA 2014, June 2014.
Heynemann, Barrett and Cutkosky, M.R., "Slip Interface Classification through Tactile Signal Coherence," IEEE/RSJ IROS 2013 DOI 10.1109/IROS.2013.6696443 (preprint).
D.L. Christensen, E.W. Hawkes, A. Wong-Foy, R.E. Pelrine, M.R. Cutkosky, "Incremental Inspection for Microrobotic Quality Assurance," Proc. ASME 2013 IDETC/CIE 2013, Portland, OR (preprint).
D. M. Aukes and M. R. Cutkosky, “Simulation-Based Tools For Evaluating Underactuated Hand Designs,” in 2013 IEEE International Conference on Robotics and Automation, 2013. (preprint)
D. M. Aukes, M. R. Cutkosky, S. Kim, J. Ulmen, P. Garcia, H. Stuart, and A. Edsinger, “Design and Testing of a Selectively Compliant Underactuated Hand,” International Journal of Robotics Research, April 2014, v. 33, pp. 721-725.
Aukes, D., Kim, S., Garcia, P., Edsinger, A., & Cutkosky, M. R. (2012). Selectively Compliant Underactuated Hand for Mobile Manipulation. 2012 IEEE International Conference on Robotics and Automation.
Aukes, D.A., Heyneman, B., Duchaine, V., and Cutkosky, M.R. "Varying spring preloads to select grasp strategies in an adaptive hand" IEEE IROS 2011, San Francisco, Sept. 2011. (slides)
Ulmen, J., Cutkosky, M. "A robust, low-cost and low-noise artificial skin for human-friendly robots,", 2010 IEEE International Conference on Robotics and Automation (ICRA), 3-7 May 2010