Biomimetics and Dextrous Manipulation Lab
Main /

KatGregoriousBlog

SummerBlogs

Kat's Summer Blog

Week 9 Overview

  • Membrane puncture pilot tests completed

17.Aug.17

Goals

  • Successful user studies! (and hopefully more than just 1!)
Today is the big day! It will be the first time we are testing with experienced users—at least for this round of testing—and especially people outside of engineering, which will be very valuable. It has been hard to recruit users because our target audience includes radiology residents, who are extremely busy. We're hoping that even though only one person signed up in advance, we will be able to get at least a couple more. And worst case we can see if some of the doctors we have met with are willing to try the device.
Despite some issues in the morning with the circuit—mainly fixed by the classic shut down the whole system and make sure nothing is shorting—everything was working well by the time our first user came. The trial went very smoothly, and the user was able to give us a lot of helpful feedback. Not only that, but it was also nice to see how someone with no engineering background interacted with the setup.

16.Aug.17

Goals

  • More pilot tests for membrane puncture
  • Come away from Neuro-interventional Radiology meeting with concrete future directions
  • Finish CAD for HoloNeedle Kinect stand
I started working on the CAD for the Kinect stand when I got in, but it turns out Michael and I miscommunicated, and I didn't need to make such a complex stand. That turned out to be good for me because things are really ramping up for the membrane puncture user studies.
Before our meeting, Amy and I went over the user study script as well as the consent forms. First realizing that we needed a script, and then that we should focus on minimizing the time it takes to read were important. I cut the script down from 4 minutes to about 1-2 minutes. I know from doing previous work that users tend to get disinterested really quickly, and often don't even listen. And because our users are very busy physicians, we really need to stay attuned to how attentive they are.
After lunch Amy and I met with a Bruce and a couple doctors in neuro-interventional radiology. I came out of the meeting not only with a lot of insight, but also pretty excited. Although the only procedures that truly require MR guidance—at least for now—are breast biopsies, which don't benefit much from haptic feedback, there are many other applications we can put our efforts into. For instance, one of the doctors mentioned that for interventional radiologists working in war zones, having haptic feedback for ballon insertion in amputation cases could be life-saving. Knowing what I'm working on could make this kind of a difference really keeps me thinking I am so lucky to be where I am, and doing the work I'm doing! Furthermore, the doctors said a very justifiable use case for the device would be in fluoro- or CT-guided procedures—when doctors perform these procedures they are exposed to radiation for a considerable amount of time. Therefore, if they can perform these procedures via teleoperation, it would make a significant difference over time in terms of their health.
After the meeting we spent the rest of the afternoon finalizing the script, and testing out the setup to work out last-minute bugs. Among temperature compensation at the base, EAP actuation, slow-blow fuses and motor smoking, as well as software issues, it feels as though we've encountered most issues that might come up in the user studies. Although disheartening in the moment, I'm really glad we've encountered this issues because we'll be much better prepared for the user studies.
In the evening we moved everything to Lucas and got the setup working there. We finally put a threshold on the voltage input to the EAP, which has been making a difference. This way, users are much less likely to stop due to noise before they hit the membrane.

15.Aug.17

Goals

  • Complete pilot tests for membrane puncture
  • Begin new Bio-X poster
  • Begin CAD for HoloNeedle Kinect stand
I realized today that I completely forgot to fill in my blog entries for all of last week—things got pretty hectic trying to get everything done for both Amy and Michael's pilot studies, as well as balancing studying for the GRE. So instead of backtracking and trying to remember the specifics of each day, I put the major events in the Week 8 Overview section.
Amy and I spent the morning conducting our first pilot tests, which, compared to our motor smoking last week, went very smoothly. As I am well aware of from my PD days, it's very helpful to get feedback that we never would have realized otherwise—the big picture can get lost when you've been working on the details for so long.
I also spent a good deal of time working on the new Bio-X poster, which needs to be submitted for printing in 2 short days!

14.Aug.17

Goals

  • Destroy the GRE
  • Membrane puncture user study finishing touches
As indicated in my goals, I spent almost the better half of the day reviewing for and taking the GRE. I didn't get to the lab until around 6pm when Amy and I spent about an hour finalizing the user study setup. Everything was good to go when we tested, and we brainstormed some ways to shorten the study so our users 1) won't lose interest, and 2) will be more likely to actually participate in the study. One way we already shortened it is to decrease the distance between the home and starting positions, as well as the end limit for the needle past the membrane. According to our trials, each user should need about 10 minutes, but we have a built-in 5 minute buffer to account for questions from and clarifications for the users, as well as possible setup malfunctions.

Week 8 Overview

  • After multiple attempts and issues resulting from membrane puncture pilot studies, one of the motors went up in smoke (probably near flames given how hot it was)
  • I learned a lot after the malfunction, both about the potential issues with the setup, circuitry, and motor itself–also learned the utility of a slow-blow fuse (which we may or may not use in the future)
  • Manufactured many more membranes and phantoms for the user tests
  • Finished manufacturing the HoloNeedle user study box, inserting beads, and making the phantom
  • Round 1 of phantom manufacturing for the HoloNeedle was a slight dumpster fire—it took almost 3 hours of waiting and stirring to find out that we poured too much phantom into a glass beaker that was too tall for the heat from the hot plate to evenly distribute and produce a transparent phantom
  • We then realized we were too tired to come up with the genius idea of a lid over the beaker, but did decide to get a wider and more shallow pan
  • By the end of the week we made a successful phantom, which will cure over the weekend and be tested next week
  • Unfortunately at the end of the week—and after a non-trivial amount of time working on it—I found out that I would not be able to present the FBG work at Bio-X due to the desire to keep the project under wraps before a paper is published
  • While it's unfortunate I won't be able to present on this topic—which I am very excited about and can see a clear path to future applications—I will be able to present on the equally fascinating active needle
  • In any case, it will be a great experience for me to present a poster at a conference, even though it is internal, since I have never done that before

7.Aug.17

Goals

  • Bio-X poster outline
  • HoloNeedle CAD

Week 7 Overview

  • Bio-X poster name and structure
  • Phantom manufacturing for both membrane puncture and HoloNeedle user studies
  • Membrane puncture pilot testing

4.Aug.17

Goals

  • Finish HoloNeedle marker CAD
  • Assemble HoloNeedle box
  • Edit CAD for membrane puncture tests
Apart from our weekly lab meeting, I spent almost the entire day in Solidworks. I did finish the CAD for the HoloNeedle phantom box, and subsequently assembled it. I also edited the CAD for all sizes of the membrane puncture boxes because I didn't correctly dimension one of the inner boxes according to our new setup.

3.Aug.17

Goals

  • Finish box for HoloNeedle tests
  • Make membranes for needle puncture test
  • Pour phantom for both tests
  • CAD for new HoloNeedle box
I spent the morning assembling the HoloNeedle box, which included placing the printed grids, stringing the small strings with colored beads and fixing them at the correct distance. This process turned out to be much more tedious than anticipated—it was difficult to get the beads and strings exactly straight, and because the box itself did not have tabs, it was not perfectly aligned as we had hoped. Furthermore, the coloring on the beads bled off when mixed with Loctite—this is concerning as it may also bleed when the phantom is poured in. Overall, the grid stickers did not prove to be as useful nor as accurate as we anticipated.
After brainstorming, I suggested etching a ruler on the side of the acrylic box so we can process the images in imageJ, thus eliminating the need for the grid. I will incorporate this, as well as the tracker mount into the next CAD.
After lunch Amy and I quickly made some more membranes, and while I started on the new CAD for the HoloNeedle phantom box, she assembled the boxes I CADed yesterday.

2.Aug.17

Goals

  • Print HoloNeedle box grids at FedEx
  • Make phantom for HoloNeedle user tests
The sticker paper material we bought, in combination with many dysfunctional printers, led to an entire morning of failed attempts to print the grids on the stickers. Michael was finally able to get his home printer to print them.
I spent most of the afternoon CADing the various boxes we would need for the membrane puncture pilot tests. Amy and I thought of a new method for better securing the membranes during the study—wrap the membrane around a smaller inside box, tape the kimwipe to the inside box, then place the inner box inside the outer box, which will compress the membrane since it's only larger than the inner box by the membrane thickness.
At the end of the day I spent a while talking with Amy and Michael about planning for the next few days as it's crunch time for both projects, and I only have a few weeks left!

1.Aug.17

Goals

  • Finish box + phantom for haptic feedback user study
  • Fit box for HoloNeedle study with strings and plan target placement
  • Print grid stickers for HoloNeedle box
In the morning, Amy and I finished putting together the phantom box for the pilot test and poured gelatin between the membrane. It turned out well enough for our pilot study, which we will conduct once Jung Hwa returns from her internship in the evening.
I also made grids of various pitch lengths to stick onto the box for our user study. Unfortunately we couldn't get the printer to take the glossy paper we have, so I'll go to FedEx in the morning to print them.
The afternoon mainly consisted of editing Michael's HoloNeedle paper, as well as doing more prep for the Bio-X poster (which the cardio group will hopefully approve).
Late evening will be pilot testing with the haptic feedback system and phantoms we manufactured.

31.July.17

Goals

  • Meeting with Amy and Jung Hwa to determine next steps for user study
  • Make phantom for Michael and Amy's user study
Today ended up being a whirlwind of random things coming up, mostly related to applying to PhD programs. I spent a good chunk of the morning discussing with Amy and Michael about what their application experience was like and how I might strengthen my app. I then met with Mark at lunch to get his input as well.
In the afternoon I talked with Amy about what our user study setup will be, and—after finally getting access to a laser cutter than can actually cut—we made the pieces for the phantom box.

Week 6 Overview

  • Analyzed cardio experiment data

28.July.17

Goals

  • Make more phantom membranes for haptic feedback user test
  • Help edit Michael's paper
  • Figure out next steps for cardio collaboration after meeting and resolve testing issues
The morning consisted mostly of general paper editing, followed by some discussion about how to move forward with user testing. We'll have to set up the test a few days early and make sure we can get everything working in the med school, as it will be a big move from the lab. We also need to figure out a way to reduce test time to 5-10 minutes max.
In the afternoon I spent some time figuring out how to make the phantom membranes much thinner than our previous round. I ended up taping a kim wipe to the flat piece of ABS we used before; however, this time I spread the Dragonskin FX directly on the kim wipe, and smoothed it over with a flat piece of aluminum, only over the painter's tape. Thus, the membrane would not have the thickness of the thin film we used previously. They turned out pretty well, and Jung Hwa will test them over the weekend to make sure they have similar properties to pig liver.
We spent the remainder of the afternoon in the cardio meeting, where Mike's lab and one of the undergrads in BioE were present as well. Mike first presented a completely new setup, which he modeled after a professional left heart system—this, however, was incredibly expensive which is why he's building his own. Speaking of expenses, though, we found out that by using the med school's prototyping equipment, Mike is spending around 10x what it would cost us to make or even outsource!! My mind was completely blown that they had to spend that much—obviously, we immediately started thinking of ways we could help them outsource or even make some of the parts ourselves. We then discussed the issue of slack in the FBGs after they are sutured in place of the chordae. In the end we settled on using two very large sutures to encapsulate the smaller Gortex suture, which will be tied instead of glued to reduce complications of switching sutures during the test. In addition, we talked about simplifying the Stewart platform to a 3 DOF platform—the final word on this will be after Mike has the MRI data and we can determine the bare minimum we need.

27.July.17

Goals

  • Learn more cool things at Living Machines!
  • Solidify direction for haptic feedback user study after meeting with Roger
An update after our meeting with Roger:
"His overall idea is that it is not possible to find one specific procedure that is a perfect fit for this application—i.e. one that both requires MRI and in which haptic feedback is necessary. However, he told us that there are two scenarios we should focus on: first, in very confined procedures in which MRI resolution (2-3mm) is not adequate, and second, when there are significant artifacts interfering with MRI and the needle itself such as air, water, and implants. Furthermore, he believes our best use case will be in neuro-radiology—we are meeting with them mid-August."
"Next, in terms of the phantom manufacturing, he told us about the extensive testing he did in his PhD to attempt to create a realistic membrane. His conclusion was that it is not only very difficult to mimic a biological membrane, but that it is not worth our time and efforts due to the high variability of mechanical properties among different membranes, both in one body and among different people. Additionally, the needle itself—diameter, angle, tip geometry, and sharpness will all greatly affect the force sensed by the tip of the needle. Given this variance in properties of different needles that would be used in various procedures, he believes that justifies using the previous phantom. Therefore, we will use membranes similar to those used in Natalie and Santhi's paper so that we can refer to that paper for justification."
"Moving forward, we will first focus on recruiting residents for our experienced user study. Roger told us there is a resident-wide meeting every Thursday, and he can get us in contact with these people. We are thinking we can rent a conference room in Lucas for a few days, and invite residents to stop by at a convenient time—hopefully we will catch quite a few around the time of the meeting. Additionally, we are going to talk through the user study, and minimize the time. Roger told us that he thinks doctors won't spend more than 5 minutes, so we want to brainstorm ways to shave off some time. We can also work on this during the pilot studies."
Overall, it was a really productive meeting and allowed to us to step back and revisit the whole picture, which has been getting lost in the nitty gritty details of phantom manufacturing.
In the afternoon, we went to Rebecca Kramer's talk on soft robotics and her current work in that field. I was really inspired by the photo she showed of 3D-printed conductive material on the surface of a glove. I think there are a lot of fascinating possibilities for the interface of soft robotics and rehabilitation robotics; specifically, I think that can make rehabilitation devices more accessible and user-friendly, while reducing the need for constant physician supervision. This again got me more excited about doing a PhD, where I could spend years developing meaningful technologies similar to these.

26.July.17

Goals

  • Learn some cool things at Living Machines!
Today is the big start of Living Machines—the conference we are hosting. I helped set up the morning talks, and roamed around helping with random aspects of the conference throughout the day. Since my next to-do item is to make phantoms and targets in TLTL, I needed to save that for a day when I have a large block of free time. I decided it would be a better use of my time to go learn about soft robotics since I probably won't have a conference literally next door for quite a while!
Tomorrow and Friday will be a bit more back to normal as I have a meeting with Amy and Roger from the med school, as well as a recap/brainstorming meeting with the cardio group.

25.July.17

Goals

  • Finish cardio experiment data analysis
  • Pinpoint errors in data and possible reasons
  • Test slides for Living Machines
  • Make more membranes in TLTL with Amy
  • Meet with Michael to decide phantom mfg next steps
I spent the morning finishing the data analysis from our experiment with the cardio group. This process helped us pinpoint weaknesses in our setup and data collection. For instance, although we initially recorded the colors associated with each FBG and which chordae they correspond to, we did not keep track of which FBG corresponded to which port number in the optical interrogator. This led to confusion in data analysis—even after using process of elimination to determine the correct FBGs, we still may have had some error. Furthermore, two of the normal test plots show about one third of the measured λ values going to zero directly following a peak. We believe this is due to the FBG undergoing more strain than it can handle, which may be due excessive slack in the FBG suture placement.
Furthermore, we think there are a few other sources of error, which we plan to address at our meeting with Mike on Friday to improve future iterations of the experiment. One significant issue we need to address is how to suture the FBGs to the chordae without excessive slack or tension. Currently, it is very difficult for Mike to reach into the chamber with the heart already in place—one idea is to suture outside of the chamber, and another is to open up the chamber on the side, or even extend it out farther. Second, we need to ensure each FBG is equally tensioned, and devise a system with which we can accomplish this. An idea spurred from Hannah's suggestion is a small pulley—3D printed to minimize the additional strain or torque—that can differentially tension the FBG to a set magnitude. Of course, in the next iteration we need to keep track much more closely of which FBGs are in which port.
In the afternoon I met with Amy, who had an amazing find—she came across a company that manufactures silicone vessels to exactly mimic biological tissues, both in normal and diseased states. They can also custom manufacture, which would be ideal for us, saving us lots of time trying to experiment and eventually land on the desired tissue properties.
Following that, I met with Michael to discuss next steps for phantom manufacturing process. We settled on making small containers for each user test; each container will have multiple small, spherical targets. The point of making the targets small is so that instead of getting a binary yes/no of hitting the target, we can more precisely determine how far away from the target the user was. Previously, marshmallow-like pieces were used as targets, and these seeped into the phantom material, making it difficult to determine how far away from the target users were. Furthermore, we plan to cut out mesh grid stickers to paste on 2 sides of the acrylic cubes with phantoms in them. That way, we can take photos from 2 angles of the grids, therefore giving us the exact location of the needle with respect to the targets.

24.July.17

Goals

  • Process cardio experiment data in MATLAB
  • Coordinate meetings with Bruce's contacts
  • Lay out next steps for HoloNeedle and membrane puncture user studies
Living Machines is starting tomorrow, so I spent the first part of my morning helping Capella stuff and alphabetize name tags. I had a nice conversation with Capella and Hannah about pursuing a PhD, which I am seriously considering now after working in the BDML. Before this summer, I had never given a passing thought to that, but it's something I can definitely see myself doing. After the craziness of Living Machines has died down I am definitely going to make it a point to talk to more of my lab mates and Mark about the pros and cons of a PhD and whether it may be the right path for me.
Later in the morning, Amy and I went through the emails from Bruce's contacts who replied to us over the weekend. Bruce has been really awesome about helping us—and always very quickly—with introducing us to people, giving us insight from the medical side of the project, and suggesting procedures that would be helpful for us to view. It seems as though the neuro-endovascular team breast imaging teams are best to talk with. Due to some doctors on vacation, we'll meet with the neuro team at the end of August. In the meantime, Bruce said we can come observe some breast biopsy procedures when he starts them again in August. As I've said before, I feel incredibly lucky to be in this lab! I'm able work on really incredible projects where I can clearly see how they will impact both patients' and doctors' lives.
After lunch and for the rest of the day I finished processing the data from the cardio experiment. I'm getting a lot faster at and gaining more intuition for MATLAB, which will obviously be really helpful for the future.

Week 5 Overview

  • Background reading on TIPS procedure, and mechanical properties of relevant tissues
  • Learned phantom manufacturing
  • Lab meeting presentation on cardio collaboration
  • Completed cardio experiment

21.July.17

Goals

  • Successful lab meeting!
  • Become better oriented with cardio experiment, and contribute useful suggestions
The presentation in lab meeting went really well, and I think I was able to give a good overview of the cardio project and future directions. Not many people had even heard about this collaboration, so it was exciting to introduce something new. I gave some background on why diseases of the mitral valve are so significant—patient prognosis is usually grim, as most patients die within 5 years without treatment. So there is a large need to determine which of the hundreds of procedures will most benefit patients. I also gave some background on the anatomy of the heart as well as the cardiac cycle, since most people in our lab are not familiar with those. I then reviewed the experiment setup, and how we instrument the chordae with FBGs. Hannah had a great idea of using a small pulley system to more accurately tension the FBGs since right now it's very hard to make sure there isn't slack in the fiber, and that it isn't under too much initial strain. Lastly, I addressed Mike's vision to make the papillary muscle attachments move as they would in the human body. His idea, which a couple undergrads are working on, is to use a Stewart platform. However, after talking to Mark, we realized Stewart platforms are not only very difficult to miniaturize, but they might also be overkill for this application. Depending on the trajectory of the papillary muscles, we may even be able to get away with a 1 DOF system that utilizes a linkage.
In the afternoon we headed over to the cardio lab to begin the experiment—little did we know we would be there until 10pm!

20.July.17

Goals

  • Cast Ecoflex and Plastisol membranes
  • Determine future directions for haptic feedback in MR-guided procedures
  • Prepare slides and talk for lab meeting
In the morning Amy, Michael, and I went to TLTL to cast some of the membranes. We attempted to make some of the membranes as thin as possible, because our target membrane size is likely no thicker than 1mm. However, we realized after attempting to de-gas the mixture in the vacuum, that in the future we would need to separate the membranes based on thickness. We tried to de-gas all membranes of different thicknesses in the same chamber; however, the thinner membranes don't need as much de-gassing, and they all shriveled up by the time the others were done. Now that we know how to manufacture the phantoms, we can at least experiment with different thicknesses and materials. Once the membranes cure, we will measure their elastic modulus and try to determine how similar they are to biological membranes, hopefully with the help of Bruce.
We were lucky to hear back from Bruce very shortly, who was willing to come by the lab to talk to us about potential MR-guided procedures. He also agreed with Mark's suggestion—that while some procedures may not be MR-guided now, they likely could be in the future. However, he said the most promising applications—which also currently use MRI—will come from neuro radiology. Bruce introduced us via email to a few of these doctors, so we're waiting to schedule meetings with them.
For the remainder of the day, I worked on finalizing the presentation and talk for lab meeting tomorrow.

19.July.17

Goals

  • Understand cardio setup for FBG experiment
  • Talk to Mark and Amy about TIPS alternative
I spent the first few hours of the day at the cardio lab talking to Mike about the FBG experiments, which was really fascinating. He ran us through the overall goal of the experiment, some anatomy, and his setup. It was clear he spent a ton of time on his setup, which was pretty impressive and complex mechanically, especially for someone with a medical background.
In the afternoon I went through the photos, videos, and surgical textbooks that Mike sent us. I had never seen the setup with the actual pig heart in it before, so it was a bit hard to visualize—however, after some reading I felt up to speed. I also spent some time brushing up on anatomy from my pre-med days so I would be prepared to present the background slides on this project at lab meeting on Friday.
Later I met with Amy and Mark to talk about an alternative to TIPS, and Mark brought up a great point—while a procedure may not be MR-guided right now, it may be something doctors are working toward in the future. Especially with X-ray and CT-guided procedures, MR-guidance may be desirable due to the lack of radiation on the patient. With this in mind we will meet with Bruce tomorrow to discuss more in-depth which procedures we can tackle, and see whether we are moving in the right direction.

18.July.17

Goals

  • Email Bruce to get a ballpark estimate of the pressure in the HPV
  • Read paper on phantom manufacturing to determine how to get desired elastic modulus
  • Plan out next steps for HoloLens project and presentation
Today again involved lots of reading to determine the appropriate elastic modulus at which we should manufacture the phantoms. After reading through more literature and corresponding with Bruce, we realized there is quite a large range in possible elastic moduli, heavily depending on disease state: from 10kPa to 25 kPa. Keeping in mind that we want user studies as short as possible, we decided it would be best to make two phantoms, each at the lowest and highest elastic modulus. Thus, we would be able to test the difference in force felt by the user as well as the difference in haptic feedback.
In the afternoon we had our weekly meeting, which was really helpful for us. Mark pointed out, after watching a YouTube animation of the TIPS procedure, that the needle might actually be inserted at an angle and not perpendicular to the tissue as we previously modeled. Furthermore, we need to consider a layered model in order to best mimic the actual conditions: puncturing the portal vein, traversing the liver, and puncturing just one membrane of the HPV.
Given the insight we got from the meeting, I started researching the elastic moduli of the liver and portal vein so we could appropriately model those when we manufacture our phantoms. However, upon further research of the TIPS procedure, I realized it is actually X-ray guided via fluoroscopy rather than MR-guided, which is a bummer given how much time we spent researching it.

17.July.17

Goals

  • Finish reading 'Force Modeling for Needle Insertion Into Soft Tissue'
  • Determine which tissue properties we need to accurately model tissues involved in prostate biopsy and TIPS
  • Research materials with similar properties/how to make our own material

Papers read:

  • 'Force Modeling for Needle Insertion Into Soft Tissue' [Okamura et al., 2004]
  • 'Robotic Needle Insertion: Effects of Friction and Needle Geometry' [O'Leary et al., 2003]
  • 'Mechanics of Dynamic Needle Insertion into a Biological Material' [Mahvash et al., 2009]
  • 'Biomechanical Study of Hepatic Portal Vein in Humans and Pigs and Its Value in Liver Transplantation' [Wang et al., 2009]
  • 'The Effects of Testing Environment on the Viscoelastic Properties of Soft Tissues' [Ottensmeyer et al., 2004]
  • 'Needle Insertion Modeling and Simulation' [DiMaio et al., 2003]
Today was all background reading on mechanical properties of the HPV (hepatic portal vein), which is involved in TIPS. I initially thought it would be a quick search for the elastic modulus of the HPV, but with every article I came across I discovered more factors that we might need to take into consideration. First, the elastic modulus of the HPV is dependent upon the pressure that vein is subjected to [Wang et al., 2009]. Second, that pressure will vary slightly person-to-person, as well as among different disease states. Therefore, in order for our phantom tissue to be representative of what a surgeon will feel during this procedure, we likely need to manufacture multiple phantoms each with a different elastic modulus. Third, there is the question of how close these elastic moduli from the study—using post mortem in vitro HPVs—are to in vivo HPV elastic moduli. This question arose after reading Ottensmeyer et al., 2004, which claims in vivo vs. ex vivo elastic moduli can vary up to > 50%, while in vivo vs. in vitro perfused can result in differences of 17%. Thus, it may be useful to us to vary the elastic moduli by plus/minus 17% from the selected pressure(s). Lastly, and less directly related to phatom manufacturing, is the needle insertion velocity. According to Mahvash et al., 2009, there exists a saturation velocity at which the needle can be inserted into biological tissues to minimize the magnitude of the fracture. While this may justify the constant velocity at which we keep the needle during user studies, it likely will not be implemented in the OR and may not be worth our time to sort through and adapt the model for our use. Instead, what we want to focus on is maximizing the force felt by the surgeon at membrane puncture, and to stop the need as soon as possible thereafter to avoid puncturing the other wall of the vein.

Week 4 Overview

  • FBG temperature-dependent and force calibration
  • IRB training
  • Research surgeries that would benefit from active needle/haptic feedback
  • Research material properties of tissues involved in relevant surgeries

14.July.17

Goals

  • Research mechanical properties of veins and find material with equivalent properties
  • Research surgeries that would benefit from an active needle and haptic feedback (i.e. those that require puncture of one membrane and for the needle to stop immediately after puncture)
I spent most of the day doing background research on surgeries that could benefit from haptic feedback following membrane puncture. However, after a few hours of looking into these procedures, I realized many of them didn't necessarily require MRI compatibility, which is our main goal. After talking with Amy, we decided the two procedures that would benefit most from our application are TIPS and prostate biopsy. Given this information, we would need to determine which properties of these tissues are necessary to consider, specifically for a needle puncture application. Mark pointed us to a paper by Allison Okamura, 'Force Modeling for Needle Insertion Into Soft Tissue,' which we started reading at the end of the day.

13.July.17

Goals

  • Finish IRB training
After a morning full of more IRB training, I needed something to break up the monotonous scrolling, reading, and quizzing. I talked with Amy to see if there is anything I can research in the meantime to help with the haptics side of the project. The next steps are to address something they were asked when they last presented their paper—how would changing the tissue change the success of the membrane puncture? In their last study they used a film as the membrane in a phantom; however, in reality the tissue properties may be quite different, thus affecting the success of the membrane puncture. Therefore, it will be necessary to focus in on a specific operation that is both image-guided (to remain under the MR-compatible scope of the project), and is difficult using just imaging (so the project has a meaningful end goal). Amy told me given these considerations, the general procedure they want to tackle next is Portal Vein Bypass Surgery, and more specifically Transjugular Intrahepatic Portosystemic Shunt (TIPS). For the rest of the day I'll familiarize myself with the procedure, tissue properties of the membranes involved, and current procedure protocol.
At the end of the day I FINALLY finished IRB training!! Although it wasn't the most thrilling day, I know the information I learned will be useful for designing and implementing the experienced user studies, as well as for general best practices and guidelines in research moving forward. And for the time left today I'll get started on TIPS background research.

12.July.17

Goals

  • Finish FBG force calibration
  • Begin CITI training
I met briefly with Jung Hwa and Amy this morning to first, make sure I'm on track with FBG calibration for testing tomorrow, and second, to plan out what I can do with Jung Hwa and Amy moving forward. Everything looks good with the FBG calibration, so all I have to do is suture the 5 FBGs we're using, finish force tests, and edit the temperature data to exclude the outliers. In order to help Jung Hwa and Amy with the experienced user studies, I'll need to complete IRB training through CITI—yay more online training fun!
By the end of the day I finished all force tests, and have all coefficients of the linear fit line from the force and temperature calibrations for the five FBGs we plan to use. We got an email at the end of the day from the cardio group asking to postpone the experiment until next week—this is probably for the best as Michael will be back and we can discuss steps going forward as a group.
I spend the remainder of the day doing the first third of IRB training.

11.July.17

MRC: Hi Kat, I talked briefly with Jung-Hwa who came in this evening and she clarified a couple things (which I'm adding gratuitously to your blog, for posterity):

  • Although you are doing a force calibration and there are both thermal and mechanical strains, the mechanical stresses and strains are much higher than for the force-sensing needle. Therefore the thermal strains are much less significant as a percentage of the total measurement. So temperature calibration is less critical than I was thinking.
  • It should suffice to calibrate at a couple of temperatures and then we can easily interpolate for whatever the exact body temperature is. And even if we're off by a bit, it won't matter.
  • The fibers are unjacketed, but they do have a thin protective coating which makes them less fragile than plain fibers. The coating is thin and it would be a good idea to avoid scraping the fibers on metal tools or parts, which could scratch the coating.

With this info. I'm confident we can easily do a "good enough" calibration tomorrow.

Goals

  • Finish FBG temperature-dependent calibration
Unfortunately I wasn't able to finish the temperature-dependent calibration yesterday due to how long it takes to stabilize the temperature within 1/10th of a degree. I learned this the hard way when I looked at the λ vs. temperature plots, and realized half of the data I collected did not fit the linear trend it should have for 37C. I thought it strange that only 37C had an error, so I brainstormed possible sources of error: how long I let the temperature stabilize before taking a data reading, thermocouple position and/or thermocouple itself, nichrome wire, unstable power source voltage (not much I can do here since this is the power source itself). Of the aforementioned sources of error, the easiest would be to change out the nichrome wire, so that was step one. Not much change, so I started looking into getting new K-type thermocouples from the physics stockroom. In the end I decided that adjusting the voltage, and therefore temperature, more slowly—would be my best option for the time being as that's likely the most significant source of error since 1/10th of a degree produces a large δλ. Looks like that mostly fixed my problem.
The sleeves finally came in today, which means I can start force calibration as soon as I finish temperature-dependent calibration.
Afternoon update: still had the same strange issue with about half of the FBGs. At ONLY 37C, about half of the FBGs had λ readings that were very off from the expected value that would match the linear fit from 36C to 38C. I redid the readings for those fibers multiple times, still with that issue. At the needle meeting we brainstormed using water instead of just air to heat the fiber to the correct temperature. I talked to Jung Hwa and she said she had the same issue with calibration, but always used just air. Because we only need 5 FBGs for Thursday, and because I've already spent so much time on this, we'll just go ahead with force tests for those 5 and troubleshoot the others later on.
Jung Hwa will come by later on to put on the sleeves, so I should be able to finish the force tests tomorrow.

10.July.17

Goals

  • Finish FBG temperature-dependent calibration
First of all, how is it week 4 already?! That means it's almost week 5, which means SURI is almost halfway over! I spent the first few weeks bouncing around among different parts of the needle project, and lately have been taking the lead on the cardio project. I've already learned a TON about different systems, which is valuable not only for general knowledge purposes and better understanding the needle project, but also for potential future research. That said, I would like to focus on one subsystem for the remainder of the summer so I can develop specific, in-depth knowledge.
Luckily, Amy returned today, and she spent the first part of the morning explaining to me the haptic feedback system for the needle project. She's focusing on the feedback for membrane puncture, although there will of course be additional uses for haptic feedback in the future through this project. Since I've never done anything with haptics, it was really cool to get an overview of what produces this kind of feedback. It's integrated with so much of our technological world—phones, headphones, etc.—so it was exciting to finally learn how a lot of those things work.
It looks like once I'm done calibrating the FBGs I'll get to work with Amy on making the sensors themselves, as well as setting up an experienced user study with the end goal of a paper. Pretty excited about all of that!
In the meantime, the rest of the day will be spent calibrating the FBGs and processing the data.

Week 3 Overview

  • FBG temperature-dependent and force calibration setup
  • MATLAB script for FBG calibration
  • Calibration data analysis

7.July.17

Goals

  • Finish FBG calibration script in MATLAB
  • Continue FBG location determination and temperature-dependent tests
Almost the entire day today was filled with meetings, so I wasn't able to get much further with the FBG calibration. I did, however, very early in the day finish the MATLAB script for force and temperature sensing, as well as set up the excel file with the equations using linear fit coefficients generated from MATLAB.
First meeting was with Jung Hwa—she ran me through a couple more details on FBG sensing, then took me over to TLTL so I could get an overview of the phantom manufacturing process. The FBGs will take first priority in the most immediate future, so I will go back to TLTL with her late next week to run through the entire process. I do have an idea of how to improve the current phantoms, which are transparent gel with colored targets of a stiffer gel. Currently, for the needle insertion study, we can only tell whether the needle has hit the target, but not how far away from the target the needle is in the case it misses the target. I plan to laser cut a few rings–of desired distance from each other—so that we can pour into each ring a different colored stiff gel to determine how far away from the target the needle is. More on that late next week post-FBG calibration.
The second meeting was our lab-wide weekly meeting, with a focus this week on the ICRA debrief from Sam and Matt. I was most fascinated and impressed by the magnetically controlled capsules that can go down the human esophagus and into the stomach to locate, probe, and extract a sample from a tumor. The capsule is controlled with a device outside and above the stomach, which moves the capsule by creating a magnetic moment and traveling in parallel the desired route. Engineering is crazy!!
Meeting number 3 was an update on Living Machines and the logistics that need to be taken care of.
And the final meeting of the day was the rescheduled needle meeting. I explained to the rest of the group the work I'm doing with the FBGs, as well as the overall vision of the cardio project. It seems not many people in the lab are familiar with this project, as I believe I'm the first person who has been able to dedicate the majority of their time to it. Alex made a lot of progress on casting the diaphragm more effectively, and Sam had some interesting ideas about making a curved diaphragm, which is apparently very hard and has yet to be done successfully. Overall, really exciting stuff going on!
The rest of the afternoon was spent doing more FBG sensor localization.

6.July.17

Goals

  • Determine exact location of FBG sensors and begin temperature-dependent tests
  • Familiarize myself with current MATLAB script and edit for FBG calibration
  • Get additional sutures from Mike
  • Contact Cecilia about Living Machines
Unfortunately the day started quite late today due to a doctor appointment; however, I was able to contact Cecilia and go over to Mike's lab to get an overview of the cardio project in the morning.
The overall goal of the cardio project is to be able to determine which of the 30-40 procedures for mitral valve repair/replacement is most effective. At the moment, surgeons do not know which of these procedures is best—therefore, we want to measure the forces, ex vivo, in a diseased heart before and after various procedures, in order to determine which method is most effective.
Right now, Mike's group just needs the calibrated FBGs in order to present the force data in the first paper. Later on, they may need more help with flow data, etc. That should all be determined in a couple meetings—one before they start testing (likely late next week), and the second when everyone is back in town in a couple weeks.
Now that I have data from one of the FBGs, I need to figure out how to post-process it to get the λ that corresponds with a ΔF = 1N or ΔT = 1 degree C. What I need from my MATLAB script is to read in the data files of wavelengths for one test (because I did three runs of each), take the appropriate wavelength column, average over that column, then put that averaged value into a vector that will hold the averaged value for each test of a different weight (or temperature). I will use a scatter plot to plot those averaged wavelengths against either force or temperature. Finally, I will use the polyfit function with a first-degree polynomial for the data, taking the two coefficients to generate an equation that will give λ(F) or λ(T).
I started out trying to edit a script that had been used for FBG calibration for the needle; however, after a few hours of trying to make the file-reading syntax work, I realized I didn't need anything that complicated. This was actually great for me because it's another opportunity to improve my MATLAB skills! After getting started on this code, I realized I understand MATLAB a lot more now that I'm writing a script for my own application—this will be pretty helpful both for future classes and for future research use.

5.July.17

Goals

  • Set up and complete temperature-dependent FBG calibration
Mission complete! I successfully set up and completed the temperature-dependent calibration for the practice FBG. As planned, I used an aluminum tube that spanned the length of the sensor, and wrapped nichrome wire around the tubing. I secured the tubing with kapton tape to avoid the wire shorting. While I spent time calculating the input voltage in order to get exactly the temperatures I wanted—36, 37, and 38 degrees C—the resistance of the nichrome wire after cutting it to 0.5 ft was significantly off from the calculated values according to data sheets. I ended up varying input voltage according to the thermocouple reading. As shown in the photo below, I inserted the thermocouple into the tubing so it was just above the middle of the tube, but not touching the FBG so there wouldn't be any measured strain. I also taped kapton to the top and the bottom of the tube in an attempt to avoid air flow through those ends.
Now that I have a reliable temperature-testing setup, on to the real FBGs.
Turns out the real FBGs have about 10 feet of unprotected FBG length—in other words, they don't have a sleeve protecting the very fragile wire. Therefore, we need tubing over that part of the FBG up until right before the sensor itself; otherwise, the FBGs will almost surely break once in a beating heart. We ordered some tubing from McMaster last week; however, there is so much friction in the tubing that even when I taped the tubing completely flat on a table, I could only get about 5 inches of the 10 feet through. We're ordering some more tubing that will hopefully be in ASAP.
While I wait for the tubing to come in, I'm going to pinpoint the exact location of the FBG sensors and do the temperature tests for as many FBGs as possible. That's about all we can do without the tubing because we need the sutures for the force test, and the sutures won't fit through the tubing so they have to go on after. And still, I'm a bit worried about doing the temperature tests without the tubing because of how fragile the FBGs are, along with my track record of breaking pretty much everything I touch.

3.July.17

Goals

  • Solidify a list of things that need to be done while Michael is gone in order of importance
After beginning the FBG calibration process last week, we realized we were missing one important test condition: that the FBGs are kept at a temperature equivalent to that inside the human body. FBGs are not only strain-dependent, but also temperature dependent. Therefore, the FBGs must be kept at constant temperature, as they will be while in use, in order to accurately calibrate them.
Thinking about different ways to heat the wire to constant temperature, using nichrome wire seemed to be the fastest and easiest to implement. I plan to cut a small piece of aluminum tube just longer than the part of the FBG with the sensor such that the sensor will be in the middle of the heated tube in order to minimize heat lost to the environment at the sensor. Using the nichrome wire specs—resistance/foot and current needed to heat the wire to a certain temperature—I can calculate how long the wire should be. After wrapping the nichrome wire around the tube and insulating it with kapton tape, I can slowly increase the current through the wire until the thermocouple reads 37 degrees C.
After picking up more sutures from the cardio group, I spend the rest of the afternoon discussing potential figures for Michael's paper on the HoloLens side of the needle project.
List of things to do, in order of importance, while Michael is gone:
1) meet with Mike from cardio group to plan future experiments
2) calibrate FBGs in temperature-controlled environment
3) schedule experiment with Jung Hwa and run experiments with Mike
4) meet with Jung Hwa to learn phantom manufacturing for experienced user study
5) prototype method for measuring needle placement accuracy
6) set up pilot study with MRI-only condition

Week 2 Overview

  • Learned how to code in Unity and publish programs
  • Learned FBG calibration process

30.June.17

Goals

  • Be able to do FBG calibration and analysis on my own
We had our weekly lab meeting this morning, and this week we started lab presentations with updates on what various sub-groups have been working on. We saw what the SuperSCAMP group is doing, and it seems as though they have made a ton of progress recently on the controls side. The basic idea is to be able to take off, fly, perch, climb, self-rescue if needed, and fly away—it's pretty incredible that they already have the basics of this down and I'm excited to see what they can accomplish by the end of the summer. The other presentation was by Matt, who gave us an overview of what he did over the past year at EPFL—also incredible work! He and his team worked on a quadrotor team that could enter small spaces, such as for search-and-rescue applications, as well as open doors. I especially appreciated the insight he gave us into deciding what question to answer in his research, which included a lot of research about similar bio-inspired mechanisms in nature. I think we can learn a lot from bio-inspired mechanisms, and maybe this is a product of working in a biomimetics lab, but I think there are a lot of applications—medical especially—that can learn from this approach.
The rest of the day involved learning how to calibrate the FBGs and understand what the data outputs correspond to. Due to lack of materials—sutures and fiber jackets—I'll continue the remainder of the calibrations (a lot!!) on Monday.
I also started reading up on current medical applications for FBGs, focusing specifically on cardio.

29.June.17

Goals

  • Publish an app for HoloLens from Unity (crossing my fingers for minimal software/installation issues)
I ended up spending over 6 hours today learning and troubleshooting the process for CNCing the gecko adhesive molds. Although this isn't my main focus for the summer, I've been asked to help out with the [long and tedious] process given I have prior CNC experience. Although I can't say it's the most exciting machining I've ever done—more than 5 hours of set-up for very specialized machining, that, in the end, you can barely tell made a mark in the part—it is good to see how one of main products of the lab is made.
And with the little time I had left in the day I went back to the HoloLens tutorial in an attempt to actually get something up and running. However, after talking to Michael and trying to plan for where I can be most useful during the two weeks he's gone, we agreed it would be a better use of my time to switch gears to the FBG calibration. This is great, because as soon as the FBGs are calibrated, we can meet with the cardio surgeons. The prospect of having a project that is somewhat 'my own' is pretty exciting.

28.June.17

Goals

  • Publish an app for HoloLens from Unity (crossing my fingers for minimal software/installation issues)
  • Come away from the day with a better understanding of the overarching goals and aspirations for the needle project after the mixed reality surgery team meeting
Once again, the day has unfortunately been filled almost exclusively with debugging Unity and Visual Studio, and crossing my fingers every time I download a new version or toolkit.
Apart from the downloading, I went to the Mixed Reality Surgery meeting. I met pretty much everyone involved in the project, including a few others in ME, surgeons, BioE students, and a few post-docs in the med school. What was really incredible was to see that the work we are doing is directly impacting patients, right now—the surgeon is already what has been developed thus far in a surgery tomorrow. You can really tell the people from the med school are excited about this, and they've told us their patients are equally excited about this. Even though I haven't been on this project very long I can tell it'll be a really rewarding one.

27.June.17

Goals

  • Publish an app for HoloLens from Unity
  • Contain my espresso addiction
Due to construction in the lab, I spent the morning watching Microsoft's HoloLens: Holograms 101E course videos so I can be more efficient and prepared once I can get back in the lab where we have the software and equipment I need. I got about halfway through the app build when I realized the computer I was working on didn't have a few of the app packages I needed, so I had to spend the rest of the afternoon waiting for these to install :(
In more exciting news, we had our weekly needle meeting and brainstormed on how to solve some of the HoloLens calibration issues. Mark brought up a cool thought for the future: the HoloLens can do a user-based calibration as the surgeon sets up and looks around the room. This would utilize a method similar to SPAAM, and wouldn't necessitate any extra steps for our users, which is of course the most ideal scenario. For the time being, we will use optical triangulation from the OptiTrack cameras mounted to the ceiling of the OR along with the HoloLens's RGB camera to get our calibration data.
During the needle meeting we somehow landed on a tangent about a previous discussion with some cardio surgeons at the med school. The details of what they want to do remain vague, but essentially they want to insert FBG sensors into the heart in order to have real-time, accurate measures of the forces felt by the heart muscles. I believe the goal is order track certain disease states of the heart to get a better understanding of how little-understood diseases propagate. Apparently we will meet with this group again in a week or two, and I'm pretty keen on learning more about the project. It sounds like right now no one else in the lab has time to work on this, so if I can, I would be pretty excited about spearheading a project like this.
Thinking about the potential cardio project, as well as the other medical stuff I'm working on, made me realize that I'm finally coming full-circle to where I started at Stanford. I started off as a pre-med, dead set on becoming a neurosurgeon–after a couple of years here I realized I wanted something where I could be creative, yet still immersed in the medical field. This research feels pretty perfect!

26.June.17

My entire day was consumed by Unity tutorials, but by the end of the day I coded up a functioning "Roll a Ball" game. The game itself is very simple; however, there is definitely a learning curve to Unity, both in terms of learning how Unity organizes scene objects, user inputs, interactions, and so on, as well as in C# syntax. Overall, I really enjoyed getting started with Unity both because it'll be useful to help with our HoloLens project, and because the nerd in me loves that it incorporates physics and dynamics with coding!

Week 1 Overview

  • LOTS of background reading to get up to speed
  • Determine which projects I'm most interested in, and where I can be most helpful
  • Give insight/suggestions to co-researchers based on HoloLens background reading

23.June.17

Finished reading the articles listed yesterday, and began Unity tutorial.
More notably, I realized that my short stint in PD—after pre-med/IR and before ME—is coming in handy because we're building something that directly interfaces with our users, the surgeons. I feel I have a strong understanding of the design process from classes and previous projects, and that has been really helpful while thinking about how we can make the calibration process as seamless as possible for the surgeons, while still retaining the accuracy we need. Because I've been at Stanford for five years and have had drastically different majors and intended career paths, it can often feel as though my time here has been disjointed and may not all contribute to the work I do, or will do in the future. However, even after these first few days I can see everything coming together which is really rewarding.

22.June.17

Getting versed in current knowledge and methods for HoloLens calibration was today's focus. I read the following papers:
1) "Comprehensive Tracker Based Display Calibration for Holographic Optical See-Through Head-Mounted Display" (2017)
2) "Interaction-Free Calibration for Optical See-Through Head-Mounted Displays Based on 3D Eye Localization" (2012)
3) "Evaluation of User-Centric Optical See-Through Head-Mounted Display Calibration Using a Leap Motion Controller" (2016)
4) "Corneal-Imaging Calibration for Optical See-Through Head-Mounted Displays" (2015)
5) "Interactive Visual Calibration of Volumetric Head-Tracked 3D Displays" (2017)
Main takeaways from the papers:
*While HoloLens calibration with respect to the environment is integrated into the HoloLens and its programs, calibration of the HoloLens with respect to individual users remains an issue.
**Even if a user performs an initial calibration of the HoloLens, if the HoloLens itself moves on the user's head, or if the user takes it on/off, his or her perception may shift based on how the HoloLens is put back on
*SPAAM (Single Point Active Alignment Method) is the most common method used to calibrate the HoloLens between uses (i.e. whenever the HoloLens may shift with respect to the user's head, including when the same user takes it on/off
**SPAAM requires the user to align a rendered fiducial marker with physical fiducial markers on a tracked object, in order to acquire the relative position of several coordinate systems: the object, the user's eyes, and the HoloLens RBG camera
**Once these three positions are detected, and the user runs through several trials of aligning the rendered and physical fiducial markers, the HoloLens is calibrated
***At the most basic level, the math behind the calibration requires matrix transformations
*Because SPAAM is tedious and requires somewhat extensive user interaction every time the HoloLens is put on, researchers have been looking into alternative calibrations that do not require user interaction; these methods are still in development
**Corneal tracking is one of the most popular alternative methods that continuously recalibrates without user interaction
***This method estimates the center of the user's eye on a frame-by-frame basis by reflecting an image onto the cornea and comparing those point positions with those of the points displayed on the screen
***While corneal tracking is more accurate than using the SPAAM method without re-calibrating between uses or users, it is not more accurate than using SPAAM every time the HoloLens is put back on
*The pattern of the fiducial marker matters—spheres with a pattern that is a combination of circles and lines was most accurate out of a sphere, cube, and box, as well as circles, a grid, and a combination, respectively.

21.June.17

Today we all pitched in on a deep-clean of the lab, which was an all-day affair. And anything that involves organization is right up my alley.

20.June.17

I devoted more time today getting more explanations of the projects going on in the lab. The rest of the day was spent on reading papers that came from these projects, mainly the needle—although I really want to start some hands-on work like all MEs, both my past experience and other lab members' past experience indicates I should get totally up-to-speed on the theory and current work before I dive in.

19.June.17

General Summer Goals

  • Gain a better understanding of engineering applications and possibilities in the medical sphere
  • Develop more in-depth skills in robotics
  • Learn at least one area of the medical robotics projects in-depth
  • Contribute at least one impactful thing to the medical robotics projects
We hit the ground running with a guest lecture by Ravi B. on "Robotics-Inspired Implantable Passive Mechanisms to Re-Engineer the Human Body." He outlined how he wants to transform tendon-transfer surgery—instead of resulting in a 1:1 coupling from the direct suture-to-tendon, he uses passive differential mechanisms to give the patient improved differential finger movement. This was not only a really intriguing talk that got me even more excited about the future of engineering in medicine, but also a great reminder of the incredible events and speakers we have on campus. During the year it's easy to get caught up in class work and miss these sorts of events, so having a flexible schedule over the summer while I'm in the lab is something I'm looking forward to!
Back in the lab, I got a whirl-wind overview of all of the BDML current projects. As a pre-med-turned-engineer, I totally geek out over medical applications in engineering—so, the MR-compatible active needle project was right up my alley. After a brief run-down of the needle project, I found out there are a few different sub-projects: the AR/HoloLens integration/calibration, the needle sensing, experienced user testing, haptic feedback, and the needle actuation system. Although I initially thought I would be most interested in the more mechanical side, such as the needle actuation system, I found the AR application and haptic feedback aspects to be really fascinating. Because the people working on things other than the HoloLens are gone this week, I'll focus on that application for the rest of the week.
Edit - History - Print - Recent Changes - Search
Page last modified on August 17, 2017, at 04:48 PM