A set of mixed-reality apps that allow surgeons to anatomically align and interact with CT scan data through Microsoft Hololens. This is one part of my MS research work so far. In this public repo I post video demostrations of the system. For now, I am not making the codes public. I have used Unity3D, MRTK, Hololens 1 and 2 for this project. Here is a news article summarizing part of our work so far. Surgeon and researcher innovate with mixed reality and AI for safer surgeries
The mission is to augment every surgery. Compared to a pilot flying a plane or even a regular google-map user on his way to work/home, surgeons today have their instruments clustered behind them hanging in the wall. The Google Maps user or the pilot gets constant real-time updates regarding where they are, what to do next, and other vital data that helps them make split seconds decisions. They don't have to plan the trip for days or memorize every turn and detail of every landmark along the way. They just do it. On the other hand, surgeons today have to do rigorous surgical planning, memorize the specifics of each unique case, and all the necessary steps to ensure the safest possible surgery. Then they engage in complex procedures for several hours, with no targeting or homing devices or head-mounted displays to assist them. They have to feel their way to their objective and hope everything goes as they planned. Through our research, we aim to change this process forever. We are making the "google-maps" for surgery.
Following is a video demostrations of the IntraOpVSP system. The following youtube video demostrates how IntraOpVSP can be a game-changer when it comes to surgical planning visualization.
In this demostration, a CT scan data has been perfectly aligned with respect to a 3D printed mannequin head. The detailed surigical planning has been aligned with the head. The Hololens-2 creates an illusion for the user that he/she can now look through the skin and inside the subject's head. I also demostrate different visualization styles I have implented. Manual manipulation of the CT scan data has also been presented.
Automatic head 3D model alignment result.
