The U.S. Department of Defense teamed up with Google to create an AI-powered microscope that could help doctors identify cancer. The tool, called the Augmented Reality Microscope, typically costs between $90,000 and $100,000. Experts believe that augmented reality microscopy will help support doctors in small labs as they face labor shortages and increasing workloads.

In his office at the Seattle VA Hospital, Dr. Nadeem Zafar had an argument to resolve.

Zafar is a pathologist, a doctor who performs clinical tests on body fluids and tissues to diagnose diseases such as cancer. It’s a profession that often works behind the scenes, but is an important pillar of health care.

Late last year, Zafar's colleagues consulted him about a prostate cancer case. The patient clearly had cancer, but the two doctors disagreed on its severity. Zafar believes the cancer is more aggressive than that of his colleagues.

Zafar turned to his microscope - a classic favorite tool in pathology that doctors rely on to help make diagnoses. But this device is no ordinary microscope. It is an artificial intelligence microscope jointly built by Google and the US Department of Defense.

The pair used the special microscope to examine the case, and Zafar was right. Within seconds, the AI ​​accurately pinpointed the site of the tumor that Zafar believed was more aggressive. After the machine backed up his argument, Zafar said his colleagues were convinced.

"He had a smile on his face and he agreed. That's the beauty of this technology, it's like an arbiter of sorts," Zafar told CNBC.

This artificial intelligence-driven tool is called the Augmented Reality Microscope (ARM), and Google and the Department of Defense have been quietly working on it for years. The technology is still in its early stages and not yet actively used to help diagnose patients, but preliminary studies are promising and officials say it could prove to be a useful tool for pathologists who don't have easy access to second opinions.

Miter is a non-profit that partners with government agencies to solve big problems involving technology. Researchers there are working with ARM to identify vulnerabilities that could cause problems for clinical pathologists.

At first glance, the ARM looks a lot like a microscope in a high school biology classroom. The device is beige and has a large eyepiece and a tray for examining traditional glass slides, but it's also connected to a square computer tower that houses the AI ​​model.

When a glass slide is prepared and mounted under a microscope, the AI ​​can map out the location of the cancer. The pathologist can see this bright green line through the eyepiece and on a separate monitor. The AI ​​can also show the severity of the cancer and generate a black-and-white heat map on the display showing the cancer's boundaries in pixels.

Because the AI ​​is superimposed directly on the microscope's field of view, it doesn't interrupt the pathologist's established workflow, said Patrick Minot, senior autonomous systems engineer at Miter.

This convenient practicality is an intentional design choice. Pathologists have struggled with workforce shortages in recent years, like many other areas of health care. But as the population ages, the workload of pathologists continues to increase.

This is a dangerous combination for the pathology profession. If a pathologist's workload is overloaded and something is missed, it can have serious consequences for the patient.

Some institutions have been trying to digitize pathologists' workflows as a way to improve efficiency, but digital pathology comes with its own set of challenges. Digitizing a single slide can require more than a gigabyte of storage space, so the infrastructure and costs associated with large-scale data collection can quickly balloon. For many small health systems, digitization is not yet worth the trouble.


ARM isn't meant to replace digital pathology systems, but Minot says it could help medical institutions bypass the need for them. For example, a pathologist could choose to use ARM's software to take screen captures of slides, which would cost far less to store.

ARM can ensure that physical microscopes, not just computers, remain an integral part of pathologists' work processes.