Augmented reality (AR) is one of the latest innovations taking route into various markets which include gaming, medicine, automotive, retail amongst others. AR technology has been in development since the 1960’s. According to a recent forecast, the AR device market is expected to reach $659.98 million by the end of 2018. AR technology lets users see the real world and projects digital information onto the existing environment. In many ways, AR is a mixture of VR, imposed into real life. These virtual contents are typically in the form of digital imagery or sound, usually applied in 3D models or videos. AR works via the use of a range of sensors such as a camera, computer components or a display device.

AR Opportunities in Healthcare

The application of augmented reality technology is opening up new opportunities in the healthcare industry. It’s expected that by the year 2020, the global market will reach a value of $1.5B. New AR innovations can help enhance doctors and surgeons ability to diagnose, treat, and perform surgery on their patients more accurately by giving them access to real-time data and patient information faster, and more precisely than ever before. AR can also bring huge value to practicing medicine and education by allowing students and trainee physicians to better visualize health issues and scenarios that they one day will be treating. The benefit that AR can bring to the healthcare industry can be groundbreaking and we are just witnessing the beginning of what is to come from AR in the field of medicine.

Augmented Surgery

As data access technologies are already very advanced, the next step is to provide real-time, life-saving patient information to surgeons which they can use during simple or complex procedures. Augmented reality will allow surgeons to precisely study their patients’ anatomy by entering their MRI data and CT scans into an AR headset and overlay specific patient anatomy on top of their body before actually going into surgery. Surgeons will be able to visualize bones, muscles, and internal organs without even having to cut open a body. This could also help them determine exactly where to make injections and incisions and it could be used to display life-saving information for paramedics and first responders during a medical emergency. AR can not only be used to perform accurate and low-risk surgeries, but it can also help surgeons save time in the case of an emergency surgery. Instead of searching among papers or through electronic medical records, surgeons can have access to all of that information on their AR screen within seconds.

At hospitals like Imperial College and St. Mary’s Hospital in London, surgeons and doctors have already begun using Microsoft’s HoloLens AR glasses during reconstructive surgery on patients who have suffered severe injuries. Before using AR, these surgeons had to use a handheld scanner to locate major blood vessels near the wound, but now their augmented reality system helps them find those major blood vessels directly and accurately by displaying them in a three-dimensional virtual image. This is just one of the many examples of how AR can assist surgeons during procedures. Whether surgeons are conducting a minimal procedure or a taking on a life tumor during surgery, AR can help save the lives of many and we have only begun to see just how valuable this technology is to surgeons.

Augmented Surgery Startups to Watch

1. BioFlightVR: (PnP Spring 2017 Batch) Bioflight offers a wide range of medical virtual reality and augmented reality services. Bioflight has a VR/AR doctor training and 360° enhanced videos to help physicians and surgeons learn about new products and procedures within their field. The company has also developed an AR/VR a medical training module to help students and doctors refine their learn and increase their retention.
2. EchoPixel: EchoPixel develops medical imaging devices that enable doctors to use CT images of a patient’s abdomen and display a 3D model. Their products allow doctors to render patient-specific anatomy leading to increased clinical knowledge, faster operations, and better care. Their EchoPixel True 3D AR product uses a wide variety of current medical image datasets to enable radiologists, cardiologists, pediatric cardiologists, and more to see patient specific anatomy in an open 3D space. The system works by using four cameras to track the user’s head movements, glasses to turn images into 3D visuals, and a stylus to let users move and interact with objects in real time.
3. Proximie: Proximie is a revolutionary application transforming the way surgeons work and teach. Proximie is a device and platform agnostic, cloud-based SaaS product. It can pair with most computers, tablets or smartphones with cameras to connect surgeons & students, in real time, anywhere in the world. Proximie layers digitally created content on a live video stream, to provide “hands-on” virtual assistance. It effectively enables a remote surgeon to virtually “scrub-in”. 
4. MedicalRealities: Medical Realities is an innovative group offering medical training products, specializing in virtual reality, augmented reality and serious games. Medical procedures are made viewable on consumer VR devices like the Oculus Rift. Through medical training content available, the company is able to reach a much wider audience making it low-cost and easily accessible for learners. Medical students could have access to practice procedures much earlier in their training careers than they would normally do, creating a new and unique learning opportunity.
5. Help Lightning - Vipaar: Vipaar is an AR video support solution. In 2013, the company partnered with the University of Alabama, Birmingham on an orthopedic surgery using Google Glass. Surgeons are able to remotely project their hands into the display of a surgeon on site wearing the AR technology and point and guide to what needs to be done during the procedure.

Augmented Diagnosis 

Augmented reality also makes it possible for doctors to better determine their patients’ symptoms and accurately diagnose them. Often times patients struggle to accurately describe their symptoms to doctors, but with AR, patients can describe their symptoms better. For example, when patients come to the doctor’s for a basic shot, nurses can use AR to find veins easier. AccuVein is an AR startup that uses a handheld scanner which then projects over skin and helps nurses determine where veins are.

Augmented Diagnosis Startups to Watch

1. EyeDecide by Orca Health: EyeDecide is one of its kind medical app, which uses the camera display for simulating the impact of specific conditions on a person’s vision. Using apps like EyeDecide, doctors can show a simulation of the vision of a patient suffering from a specific condition.
2. AccuVein: AccuVein is using AR technology to make both nurses and patients lives easier. 40% of IVs (intravenous injections) miss the vein on the first stick, with the numbers getting worse for children and the elderly. AccuVein uses augmented reality by using a handheld scanner that projects over skin and shows nurses and doctors where veins are in the patients bodies. AccuVein has been used on about 10 million patients, making finding a vein on the first stick 3.5x more likely.
3. Augmedix: The San Francisco-based company aims to harness the power of the Google Glass to make healthcare more patient-centric and cut down on administration. Augmedix provides a technology-enabled documentation service for doctors and health systems, so physicians do not have to check their computers during patient visits, while medical notes are still generated in real time.
4. SentiAR: SentiAR is digital health, software device company adding a new dimension to clinical practice in interventional procedures. Spun out of Washington University, they are developing the first 3D visualization platform using real-time holography of the patient's anatomy and catheter location with a goal to provide the clinician and patient a faster, safer delivery of care. SentiAR’s mission is to transform the experience for both patient and clinician in interventional procedures.
5. Atheer: Atheer is the pioneer of AiR (Augmented interactive Reality) computing. AiR Smartglasses connects their AR glasses to an android-based computer. AiR Smartglasses integrates hand-tracking and gesture control with their see-through display. AiR Glasses enable users to view critical work information right in their field-of-view and interact with it using familiar gestures, voice commands, and motion tracking. Beyond data access and bringing life-saving information into doctors field of vision, the ability to interact with that information through gestures, voice commands, and motion tracking is a game-changing capability.
6. Meta: Meta offers users a total augmented reality (AR) experience. Meta makes an AR headset/glasses of the same name, as well as software to run on it. The Meta headset and software can be used for a variety of different purposes, including in the healthcare industry to assist doctors and surgeons with patient information and data.
   

Augmented Practice

The benefits that AR can bring to the field of medicine and education are revolutionary. Medical institutions are beginning to implement AR into their curriculum to provide students with a valuable hands-on learning experiences. Essentially, the idea for using AR in education is to simulate patient and surgical encounters for students to make all of their mistakes on AR rather than in a dissection lab or worse, in a real-life procedure. Students will use AR so they can accurately learn about diagnosing patients with health conditions or take part in an AR surgical procedure. AR technologies will also allow medical professionals to continuously observe and give feedback to students during their practice.

Another advantage of implementing AR into education is that training can now be made more systematic. Through an AR training program, doctors in training can practice on anything and everything that may come up in a real-life medical situation rather than randomly training with what’s given in a dissection lab. Medical students have always based medicine on theory and proven evidence, and now AR technologies actually allow them to visualize and practice those theories during their training. An example of this are AR apps that can be used to overlay anatomy data on a 3D human skeleton, giving them a better understanding of how the human body works. 

Augmented Practice Startups to Watch

1. ImmersiveTouch: ImmersiveTouch provides cutting-edge VR and AR training and education solutions to medical professionals, students, and the healthcare industry. ImmersiveTouch combines their innovative technologies with strong academic partnerships to promote world-class medical education and patient safety. Their company’s comprehensive education solutions include next generation surgical simulators and learning management systems. These can be found in leading medical centers around the world, including Johns Hopkins, the University of Calgary, and the University of Chicago.
   
2. Touch Surgery: Touch Surgery offers an app that allows users to practice surgery at any place and at any time. They use an interactive mobile surgical simulator that guides you step-by-step through every part of an operation, and every decision that’s made along the way. Using sophisticated technology, Touch Surgery is creating accurate and valuable surgical content, disseminating the best techniques and procedures to improve the quality of surgery worldwide.
   

High tech advances have clearly improved outcomes in medicine. Still, AR technology has a way to evolve for all this to be realized and implemented into the daily routine of medicine.  AR Hardware needs to fit comfortably and securely on the practitioner’s head. AR images will need to be projected as clear and accurately as possible. Doctors must also begin to buy into the concept that AR can bring vast improvements to their practice. Through the use of AR to assist in surgery, during the training of medical students, and even during regular doctor appointments, we will begin to see AR incorporated more into medicine throughout the coming years. There are a variety of technical and even societal challenges by incorporating AR into medicine, but it’s certainly doable and will be commonly seen within the next few years.