Image Source: Microsoft
Immersive reality experiences such as augmented reality, mixed reality, and virtual reality continue to gain inroads as practical tools for performing complex tasks such as learning how to assemble a machine engine or designing an automobile. Ford, for example, is using Microsoft’s HoloLens immersive reality headset to assist in the design of cars. At the same time, immersive reality is a nascent technology, and we’re making significant strides learning from both successes and challenges. Both the breakthroughs and challenges are useful for advancing what is state of the art.
For example, a research team at the University of Pisa recently conducted a study that demonstrated people using HoloLens mixed reality headsets performed tasks with less accuracy than when they were not wearing headsets. Subjects were asked to complete a “connect-the-dots” test wearing a headset and not wearing it. Without their headsets, subjects showed more accurate results. The study concluded “that HoloLens should not be used for high-precision manual tasks.”
The study also said, “Although there is increasing interest in using commercial optical see-through head-mounted displays [for] manual tasks that require accurate alignment of VR data to the actual target - such as surgical tasks - attention must be paid to the current limitations of available technology.”
In addition, according to IEEE Spectrum:
As well, the team plans to further study issues related to AR-assisted surgery by developing a new type of hybrid AR system that can switch between OST and video-see through (VST)-AR. With this latter approach, a real view of the world is acquired by external cameras on a head-mounted device and presented to the user after being merged with the virtual content. By comparing the two AR systems, the researchers hope to isolate and better understand the visual limitations that occur across the two platforms, with the ultimate goal of improving the performance of AR systems during precision tasks such as performing surgery.
So, what does this mean? We, as practitioners and users, need to be (keenly) aware of:
- The technological limitations that exist especially when associated with certain use cases such as surgical assistance.
- The impact those limitations can have on the task being performed.
I do, however, have great confidence in immersive reality hardware and software industry engineering talent knowing that there is now a solidified, specific use case (surgery) to advance the technology. As users continue to find additional use cases, some will push the boundaries of the technology and the industry, but the teams of people working on and with the technology will find ways to adapt and accommodate those use cases - in time. We’ve seen this phenomenon throughout history with technology from the automobile (this car doesn’t work well in mud, but we need an automobile that does, so let’s innovate and engineer a car that does) to the mobile phone (I can’t always talk; is there a way for me to just text my message?).
I consider this kind of study to be useful and welcome information. But the study needs to be thought about in terms of what it actually means for the industry and the use cases. As I previously blogged, mixed reality is a form of immersive reality, where digital content interacts with physical/real world objects in real-time and involves extensive interaction with the user. Mixed reality is a far more immersive form of augmented reality involving more sophisticated spatial computing, but it also has limitations. Clearly, augmented and mixed reality have the potential to make our lives easier, more entertaining, more lovable, and more engaging as evidenced by its uptake among innovative businesses. There will come a time when the connectivity, technology, graphics, and intelligence (AI) will work together and reach the inflection point where society will collectively ask, “How did we live without this before!?”
I’m always writing about what I’m seeing in immersive reality from different vantage points. So feel free to start a conversation with me if you have any thoughts or ideas. Let me leave you with this - when designing for immersive reality, start with a design sprint to align a team and create a prototype (teaser). But more to come on that in a future post (if you’re particularly intrigued, reach out to me and we can chat further)!
