Summer Blog - 2026
Week 1
Monday
SURI Orientation was today! Besides learning about the logistics of the orientation and visiting the lab, I also went on a scav hunt with Taylor where we went around campus and took pictures of cool items, one being a wooden crocodile. I'm feeling quite excited about this program, and I am excited to learn more about robots and engineering.
Tuesday
Today was a lighter day. I completed all the safety trainings, and finished reading a paper about the GIRAF arm and space robot.
Wednesday
I completed the in-person lab safety training and took a tour of the SRC, which is where I'll be working this summer. I also discussed research expectations and deliverables with Venny and Stanley, which will be a fully working teleoperation control system for GIRAF that they will be test. I also read a bunch of papers on current control systems for robots and brainstormed some ideas. Here are some ideas I had along with their key advantages and disadvantages.
1) Xbox
- Ads: already has an existing framework
- Disads: unintuitive, doesn't capture all movement possibilities
2) Virtual Reality
- Ads: overall positive user studies
- Disads: motion sickness
Sidenote, I really liked this paper: https://www.mdpi.com/2076-3417/10/24/9051#B37-applsci-10-09051
3) Leader follower architect
- Ads: overall positive user studies
- Disads: would probably not work for this robot as arm is 10 feet
4) BCI
- Ads: would be the most intuitive as user doesn't have to do anything but think
- Disads: would probably be pretty hard to implement & would not have as much resources to implement
5) Something similar to the Da Vinci robot
6) Something similar to an arcade machine
7) Motion capture suit
Thursday
Today was a fun day because I got to operate some robots myself (the anymal robot and the Da Vinci, which is used for surgical procedures). Some key takeaways I had were that 1) there needs to be a way for the user to assume control of the robot at the beginning and 2) there needs to be an emergency stop for the robot.
Something I noticed about the anymal robot which is what the GIRAF arm will be attached to is that coordinate axes are not intuitive because I always needed to readjust the ipad to align with the robot body. It did not help that the robot light was in the back and not the front. There was also some trial and error to get familiar with the control system which is not good in high stakes situations.
Something I noticed about the Da Vinci robot was that I really would have appreciated a haptic sensor because I could not tell how hard the robot was gripping the object or how hard I was pressing.
After reviewing these two robots, I read more papers and am thinking of creating something with virtual reality and direct manipulation interface.
Friday
Today was a national holiday, so I just finished up some papers from my dorm.
Week 2
Monday
This week, my goal is to flesh out all the details of the controller system I am going to build. Today, I completed the PRL safety training and also attended a SURI fireside chat with a current PhD student about her journey. Then, I continued reading a bunch of papers on controller systems. The following below are my notes.
UI principles
- Good if user can see battery life
- Know when an arrow happens or a limitation in the robot movement has been reached, for example, maybe when we can no longer extend or something
- Robots might be able to twist in a way that the human arm cannot twist → VR limitations
- User should be aware of what's around the robot
- Must communicate available actions to user
- Option where users can zoom in and out of existing view?
- Interface should provide hints about next moves
- User should be able to switch between views
- The information provided by the UI should prevent user errors, and if a user makes a mistake, the UI should allow for its rectification. In contrast with undoing a “Cut” operation in a word processor, a “Cut” command to prune a tree through a teleoperated agri-robot cannot be undone.
Trends
- Clear shift toward multimodal hybrids (such as haptic-immersive fusions)
communication latency remains the main obstacle, with delays over 200 ms undermining stability and safety regardless of interface complexity.
- Haptic devices are among the most active areas of teleoperation,
- Trends towards autonomy and haptic devices → maybe the person just presses a button and the thing just goes there and then you start rotating
Some other ideas
- Control a robot by moving a hologram, and use Unity, ROS, motion planning, torque simulation to safely translate that into real robot motion.
- compute joint torques from the robot’s planned motion and send that back to the AR headset as visual feedback (Manual vs automatic mode)
- Control the robot by manipulating a virtual version of it in your real environment
https://ieeexplore.ieee.org/abstract/document/6977975
https://ieeexplore.ieee.org/abstract/document/11450088
https://ieeexplore.ieee.org/abstract/document/8957294
Tuesday
Today, I learned how to collect some data for GIRAF and got to met the other person who will also be working in BDML. I then brainstormed some ideas for the controller. The following are my notes.
Main view = first-person (robot camera) Side panel / overlay = third-person view
- Able to switch views, zoom in/zoom out → maybe make the ways these are seen like customizable (like the size of the windows)
- Maybe just like the controllers connected to the computer → I mean like ideally we could operate robot without having to actually see it
- Rh lh orientation?
- Undoing stuff like rotation back once you originally rotate it
- Moving on keyboard/xbox controller but commands on screen/button
- Grip (autonomous)
- I feel like xbox controller is the most intuitive but maybe i should switch that out for vr → potential for bimanual???? Also might need to think about actually moving the robot.
- Adding red dots in view for correction, red lines for how close the robot should stand
- Maybe like another screen with the controller information so user knows robot movement limitations
Wednesday
In the morning, I mapped out some diagrams for the teleoperation; something I have to consider is bimanual control. In the afternoon, I had lunch with the rest of BDML and engaged in the annual lab clean up. Who would've thought there was a difference between imperial and metric screws! After that, I assisted in data collection for the GIRAF bot.
Thursday
Today, I continued fleshing out the details of my teleoperation system. We also had a meeting with Mark to discuss the compliance matrix. Below are my notes.
Friday
Today was BDML kickoff! We went to Angel Island, ate lunch, went on a hike, and talked about our goals.
Week 3
Monday
Today was a lighter work at home day. I went to the SURI fireside talk and learned more about how to have a stronger digital presence for recruiters and companies. Since I felt inspired, I worked a little bit on my portfolio. Then, I fleshed out more details of my teleoperation system since I got to try out the old control system on Thursday. I also did a lab training at Lab 64 for their laser cutter. Looking ahead, I am going to research more about human ergonomics, game hardware design, and how to actually program a VR controller/system.
Tuesday
Today, I looked into programming a VR controller system by learning the basics of how computer code controls a VR control system and how I could edit it to fit my system. I also finished my first draft of how the VR system will work. Attached are my notes.
Wednesday
Today, I learned more about how to connect a controller system to a robot. Currently, I am trying to sync up a joystick and a camera so that when I move the joystick in a certain direction, the camera moves in the corresponding direction in the corresponding amount.
Thursday
Today, I connected the robot motor to my computer and was able to get it to rotate to a certain position. We also had a crash course on robotics which I found very interesting.
Friday
Today, I completed the robot arm and visualization. Below are some demonstrations. The first video shows that when the user moves the robot arm, the position of the robot arm is projected as a red dot on the plot. The second video is shows that someone clicks a position on the sphere, the robot arm moves to the corresponding position. Some challenges I faced making the second demo was the direction and conversion of the angle: if the user clicked a position that was 350 degrees, the robot might make a weird movement since it also corresponds to 10 degrees. With the help of coding agents, I fixed it by making the robot arm go to closest equivalent angle. Another challenge I faced was the fact that the orientation kept flipping: for example sometimes clicking left of the sphere meant the robot arm would go to the left, but if I somehow flipped the orientation, the right of the sphere would now mean go to the left and the left of the sphere now meant go to the right. Another challenge I faced was that instead of taking the shortest possible distance to get to the new position, the robot arm would always reorient itself and then go to the new position. I fixed these two problems by asking the program to calculate the shortest possible distance and then take that distance. I am excited to apply these lessons to the actual teleoperation system I am trying to build.
Trace position demo Command posistion demo
Week 3
Monday
Today, we had a SURI meeting, and I listened to some lightning talks and how to schedule informational interviews. After that, Venny helped me fix some orientation consistencies in my robot arm/sphere controller, and then I got to work making a similar control model for the actual robot wrist that will be on the GIRAF robot. I also downloaded ROS, which was a little bit of a pain since I have a Mac, and it only works on Linux. Thus, I had to use a virtual environment to download it, specifically Parallel Desktop, which I already had on my computer due to having to use Ansys in another lab I am working in.
Tuesday
Today I started on the code for the wrist robot arm and then helped show a visiting professor anymal. I also refined my VR controller a little bit. Here are some more ideas I generated:
- Let go of the grip -> robot freezes -> user gets chance to reorient hand to avoid awk positions
- Maybe instead of clicking the joystick down we hold down button A for up/dpwn
- Or just clicking joystick to toggle between modes
Wednesday
Today I connected the robot wrist's camera so that it was able to be streamed on my laptop. I thought the AI
vision of feature on OAK viewer was cool because it identified which objects were being streamed. However, it always thought that its own robot arm was a chair, so I thought that was funny because it couldn't recognize itself in a way. I also tried to connect the robot wrist to my computer, but for some reason, Dynamixel wouldn't scan the motor IDs. After some help from Venny, we realized we were missing a wire connection, which was funny because it was something so simple yet detrimental (story of technology). In the morning I also had to drive to Research Park near the Stanford School of Medicine to pick up some materials, which I thought was relaxing since I enjoy driving but I don't have that much experience.
Thursday
Today I got the robot wrist demonstration to work, meaning that when I move the wrist its position is projected onto my computer and when I click a position on the sphere the robot wrist moves to the corresponding position. Venny also made me some fake VR controllers, so I could get started on proof of concept while waiting for the real VR controllers to arrive. I also connected my computer to the Optitrack system which will act as the headset (tracking the position of the VR controllers).
Friday
Page last modified on July 10, 2026, at 11:04 am