Page added to clipboard.

3-D in the OR

Imagine this scenario: A patient lies on a surgical table while a doctor guides specialized instruments through the blood vessels in her brain. But one vessel has an unusual bend and twist that’s not showing up well on the X-ray image he’s using for guidance. So the doctor calls up a 3-D image of the patient’s blood vessel system, displayed directly on his special set of glasses. It hovers like a hologram in his field of vision. With voice commands and hand gestures, he rotates and magnifies the image, investigating the shape of the bend in all three dimensions. Then he tells the image to disappear and continues the procedure, navigating his instruments through the tricky section with ease.

That might sound like science fiction. But it is now science fact.

A new tool allows surgeons to refer to a 3 dimensional “map” of a patient’s blood vessel system during surgery. Dr. Gabrielle Loeb (Columbia Vagelos Physicians & Surgeons ’18) developed the tool when she was a medical student. Experts in the fields of neurosurgery, radiology, 3-D imaging experts, augmented reality engineers, and machine learning professionals are working together on the project. Our own Dr. Philip Meyers served as the Principal Investigator for the neurosurgical aspects.

The tool consists of a headset made up of a tiny, powerful computer and a clear lens. It looks a little like a heavy-duty set of safety goggles. But turn the computer on, and it can project images onto the lens for the wearer to see. It’s part of a technology trend called “augmented reality,” in which images can be overlaid onto a regular view of the world. (That’s different from “virtual reality,” in which the view of the real world is totally shut out, and only the virtual images are displayed.) Using augmented reality, the surgeon can see the actual world to conduct the procedure as usual, and can also call up extra information at will.

Dr. Meyers was happy to support the student’s effort and be one of the first surgeons to use this new tool. Dr. Meyers is an endovascular neurointerventionalist, which means he treats brain conditions from inside blood vessels. He inserts a catheter (tube) into a large artery—usually near the hip—then guides the catheter through the vessel system to the target area of the brain. He then uses tiny, specialized instruments from inside the catheter to perform the surgery. Continuous X-rays allow him to see what’s happening inside the body. This real-time, two-dimensional X-ray “movie” is called fluoroscopy. In this way, endovascular neurosurgeons conduct entire procedures on the brain without ever needing to make an incision in the skin or bone of a patient’s head.

But recently, endovascular surgical teams at Columbia did something new. They used an augmented reality system to assist during 10 specially selected endovascular procedures. The procedures are called cerebral angiograms.

“We found the device could be extremely valuable in planning and performing a cerebral angiogram on a patient with difficult anatomy,” says Dr. Sean Lavine.

When it comes to blood vessel anatomy, the trick is in the details. Like faces, most blood vessel systems follow a predictable anatomical plan: eyes here, nose there, mouth there. But, also like faces, no two individual vessel systems are the same: How wide is each eye? How far apart are the eyes? How deeply set? Small differences add up to completely unique faces, and completely unique vessel systems. Differences in vessel anatomy—throughout the body and in all three dimensions—are of enormous importance during an endovascular procedure.

So Dr. Meyers and Dr. Lavine used the hologram to review their patients’ actual, 3-D vessel anatomy just prior to a procedure. During the procedure, the surgeons made use of the normal fluoroscopy “movie,” and also had the option of calling up the “hologram” at will. And there can be as many “holograms” as there are headsets: Dr. Jason Carroll, a Columbia Neurosurgery fellow undergoing specialty training in endovascular neurosurgery, was also able to use both image displays to assist in the procedure.

Dr. Sean Lavine, Clinical Co-Director of Neuroendovascular Services at NYPH, and Dr. Jason Carroll, Endovascular Surgical Neuroradiology fellow. Dr. Carroll wears the augmented-reality headgear and uses it in real time to assist in performing a cerebral angiogram. The team also used normal fluoroscopy.


The “hologram” isn’t a live, real-time image of the procedure. It is constructed from the patient’s normal preoperative scanning—the MRI or CT scans that many patients undergo prior to an angiogram. The 3-D information from these scans is typically displayed on a computer screen before the procedure. The “floating” image is a new way to have the information at a surgeon’s fingertips right there in the operating room.

And the benefits of the new system don’t have to stop there. The headset could potentially display other information for the surgeon, such as the patient’s vital signs. It could allow an expert surgeon to provide assistance in a procedure happening in another physical location. And Dr. Meyers believes it may be extremely useful for training residents and fellows like Dr. Carroll.

The team believes that having a 3-D hologram available during endovascular procedures like angiograms have the potential to make those procedures faster, safer and more effective.

“Dr. Loeb and her team are to be congratulated on bringing a brilliant concept to reality. It was very exciting for me and the entire neuroendovascular team to be the first to use this augmented-reality system with a patient,” says Dr. Lavine. “This is an absolute breakthrough in the future of angiographic procedures. I am proud to have taken part in this groundbreaking case.”

patient journey

Use this button to save pages to your clipboard for future use.

OK. Got it.