Surgeons set to have a healthy dose of AR

“Do I want to have my console with my GP and the guy is wearing Google Glass? I’m not sure I do.”

Virtual and augmented reality (VR and AR respectively) are long lost brothers; both of them only started developing properly in recent years but until now VR has always been the big brother that has firmly stolen the attention of the mass market. However, AR is coming into its own and has made friends with companies like Touch Surgery, that are harnessing and planning to implement it in real life scenarios that are going to save lives. 

Touch Surgery made its name with the surgical simulation mobile application it released in 2013 that tries to solve one of the biggest problems in surgery. That is, when someone is conducting a procedure in the operating room, they can either be performing the procedure for the first time or they haven’t done it for a while and they’re refreshing themselves, both on patients that are alive and need care. The app helps surgeons memorise the steps to a plethora of procedures and Touch Surgery has seen incredible successes with the software according to CEO Jean Nehme: 

“We’ve got about 1.5 million registered users and of those about 40% are medical…I guess the success of the app is a combination of things right. We’re very lucky to work with great surgeons, and people in the surgical space and being receptive to this type of technology right. They have encouraged us, supported us and been involved in building procedures…Then we’re very lucky to have a great team, we’ve been able to take a concept of surgery and with mainly not being surgeons, you know, build the graphics and the detail and the procedures and the simulations and the systems and the data and mobile component to it to get it working and operating.” 

At CES 2017 Touch Surgery showed off the next step of its work; the company is bringing its surgical simulation application to AR with the vision that one day a surgeon will perform an entire operation wearing an AR headset that tells him what steps to take in the correct order. According to Nehme, the company decided a long time ago that it wanted to make something a lot more immersive: 

“We decided that like three years ago, when there weren’t many AR headsets. We’ve always been interested in augmented reality but for us, you know, the technology has been a little bit late to come…we see this industry moving fast, and we think augmented reality is going to be a really interesting technology. But it’s always, we’re a problem solution company and our focus is how do we help surgery.” 

Walking into the Touch Surgery offices showed the company is committed to bringing the much hailed application to AR by the seemingly all hands on deck approach by its employees. The company has partnered with DAQRI, an AR headset manufacturer based in California, and Microsoft in order to get two augmented reality platforms to start working with. Whilst visiting the company headquarters, a representative for the former was there and was keen to demonstrate the capabilities of the headset. 

After trying both the DAQRI headset as well as Microsoft’s Hololens it’s clear that the technology and software by Touch Surgery is in its early stages, but even in an embryonic state, the technology has incredible potential. Sure, the voice commands don’t always work and the field of view is limited, but you’re not thinking about that, you’re too busy thinking about the near future where a surgeon is going to be wearing something on their head not too dissimilar. 

Just like VR headsets, the hardware from DAQRI and Microsoft is not completely identical, this means experimenting and testing out each headset’s capabilities according to the Head of R&D at Touch Surgery, Danail Stoyanov: 

“We’d love to have a unified experience but currently the hardware platforms are quite different and that is not something that is straightforward to do, it requires quite a bit of work…they all have little nuances, little differences… so there are technical challenges and it takes time to transition between the platforms.” 

CEO Nehme also had a lot to say about both of the headsets the company is working on, and also realises that Touch Sugery is still working with first generation products, and that means there are improvements to be made on the ends of both DAQRI and Microsoft: 

“This [Hololens] is heavy right, it’s got a computer built in right, but it’s wireless right, but try wearing this for an hour, so there are a few design things that need to be adjusted. This [DAQRI] is a bit lighter, but it’s, you know, attached to something right. I don’t have a preference yet, we’re going to be testing these with lots of different surgeons…and we’ll have to see which works better.” 

So far Touch Surgery only has a couple of simulations working in AR, however, having made the application for mobile already, it means that content should find its way over to headsets quicker than making everything from scratch according to Stoyanov: 

“The app doesn’t help but having the content helps, the content is something that takes a very long time and a lot of experts and that’s something that, you know, we’re fortunate enough to have been building for a long time. So the app doesn’t help, but the 3D models, the work flows, the procedural kind of simulation aspect, that all basically helps.” 

Other companies in the surgery field such as Cambridge Medical Robotics (CMR) have kept a keen eye on what Touch Surgery is doing and given the company praise for its work in giving a surgeon a place to train without having to use a real life patient such as the Technology Director of CMR, Luke Hares:

“In some systems and in some parts of the world, even in large chunks of America for example, you end up with a lot of surgeons who, by necessity, do a comparatively small number of a wide range of procedures…a year. So having the tools that enable him to remind him what the steps are and to help him stay current on that, makes a tremendous difference.” 

However, Hares believes that when looking to implement a technology like AR into the operating room, the uses of the headset need to be clearly defined in order to eliminate the risk of any kind of social pushback:

“In terms of AR, I think a lot of it will depend on how it’s used, I mean if you just looked at the social pushback that Google Glass got for example, do I want to have my console with a GP and the guy is wearing Google Glass? I’m not sure I do. If, on the other hand, I’m laying on a bed, probably unconscious or shortly to be unconscious, am I concerned when I’m already surrounded by all kinds of medical paraphernalia, am I concerned when a guy is wearing a headset? I don’t think I personally am.” 

Patient reactions are not something that Touch Surgery CEO Nehme is worried about, instead he believes mass adoption will come when people see the quality of the software: 

“I’m on your side, I think AR’s cool, we both hypothesise there’s a great potential in healthcare and the question is: ‘how do you go out and prove it?’ Because at the end of the day, dealing with peoples lives, you don’t want to put something that, you know, I don’t want to ever be involved in anything that impairs or risks anyone’s lives right. So the validation and the evidence is going to be what gets us out there.” 

Augmented reality is still the younger brother to virtual reality, but AR is in the hands of talented developers like Touch Surgery that are showing the real life potential for this kind of technology and making the mass market much more aware of its existence. Sure, it’s in the early stages, there are problems that need to be overcome and technical aspects that need to be improved, but all the basic groundwork is there and it looks solid. Touch Surgery are one of many companies that are working with hardware of this nature, meaning it really won’t be long before you go in for surgery and see professionals wearing futuristic headsets.