Augmented reality is usually seen as a successor technology to virtual reality, a different application with a different (and harder) set of problems to solve which will come into its own in a few years.
VR is here today, the problems are being rapidly knocked down, billions of dollars are being invested and there is a huge user base just waiting for the hardware to cross that availability threshold and Bam! the whole thing is going to explode.
That’s the buzz, anyway but there are a number of measured arguments being made which cast serious doubt on the viability of VR as a mass consumer technology for today, and there are good reasons to expect that the problems with VR can only be solved by merging it with AR.
Most of the arguments predicting limited success for the current crop of VR implementations are non-technical, which makes sense now that many of the technical barriers which doomed previous generations of VR are being rapidly demolished.
We have now had plenty of opportunity to try out a whole range of headsets and it is clear that the classic problems: field of view, lens correction, responsiveness, are either solved or well on the way.
Issues like user isolation, the difficulty of mapping established interactive paradigms onto VR and the problem of providing an equivalent visual experience have not gone away, or even really been addressed in a meaningful way and are the focus of these arguments.
And then there’s the headset.
Those last two issues: equivalent visual experience to the best of what is already available and the need for a headset, were the twin nemeses of 3D TV. VR boosters bristle at the mention of that epic failure, arguing that these are two different things but that is not the point. People are the same, which is what matters, and visual quality is important to them so we do need to pay attention to the lessons learned from that failure.
3D TV has the problem that, because of the need to deliver a different image to each eye, the visual quality is degraded by comparison with a standard video. This in itself has been enough to dissuade many viewers but gamers are more resilient in that regard than general consumers so why is it still a problem for VR? The answer lies in the extreme nature of the requirements and the gap between where graphics hardware is today and where it needs to be to deliver a true, no compromise experience.
The driving constraint of VR is latency, what is referred to as motion–to-photon delay, so that in addition to delivering a left/right pair, the system must also meet a very tight deadline for getting the pixels on the screen. From sensing a motion of the head, modern systems aim to get the resulting image on screen within a frame time and they aim to do it consistently, without the variations which are acceptable in a standard gaming system.
Standard game systems observe the twitch limit, the tenth of a second or so which represents the limit of human reaction time. This means that the various stages of the game can be pipelined: each stage must stay within the target frame rate but it’s OK to have the final displayed image several frames behind, as long as the overall delay stays within the twitch limit.
VR systems don’t have that luxury. Everything, including responding to the sensor plus animating the scene and then rendering it must happen an order of magnitude faster, with a corresponding reduction in the sophistication of the models and quality of the rendering. This is not a trivial point: it means that GPU performance must increase by 10 before the visual quality of today’s PC is available on a VR system and according to one study the last order-of-magnitude increase took eight years to happen. There’s a reason the best looking VR games are mostly static, and this is it.
VR has to offer a compelling reason, even to gamers, if they are to adopt it. Immersion is not enough: game play and realism are at least as important. If these are substandard then you are just immersed in a crappy game. Fortunately, VR has much more to offer than visual immersion so we are seeing games like “The Climb” from Crytek which tries to leverage that but in fact has been more useful in exposing more of the shortcomings of current equipment. Which brings us back to the headset.
If polarized glasses were an issue for 3D TV, the isolation and discomfort of a headset is the same issue in spades for VR.
I believe the hardware manufacturers when they say that the enormous sums being spent on design of lighter and more ergonomic headsets will result in devices no more inconvenient than a pair of sunglasses. But that only gets us to the same status as 3D glasses, which is still a problem, and it overlooks the real issue which is isolation. Isolation from the real environment and isolation from real people.
Once in a VR world, the user is blind to the real world, and now that we are starting to see apps and demos which encourage moving around, the disadvantages of that are starting to become obvious.
A world which gives no indication of its extent is not a comfortable place to be. Add a second VR user to the same space and the problem gets even worse; as long as the headsets are unaware of each other’s positions, there is the potential for collisions. This is a very poor user experience, no matter how compelling the immersion feels otherwise.
Without a solution to these problems, VR remains a solo activity conducted while seated or constrained to very limited movement (Google’s Tilt Brush app works nicely within those constraints but is hardly the basis for mass adoption of the technology). As long as that is the case, its consumer appeal will never cross over from enthusiast to mainstream.
Now, I am not going to argue that VR will fail completely but I am convinced that mass deployment is much further away than a lot of the hype would suggest. This is because the required technologies are coming from AR and are in a much earlier phase of development, which puts back the adoption curve by several years.
To be fair to the VR hardware companies, they are aware of the need for positioning and they have come out with systems like HTC’s Lighthouse to address it. The problem is that these are partial solutions and positioning is not the only problem.
Vision based systems can provide robust solutions to a lot of the problems outlined above, and we are starting to see VR headsets which include forward facing cameras (as well as internal, eye tracking cameras – an approach to solving the GPU performance problem).
With a camera the headset can autonomously map its environment, including stationary and moving objects such as other users, and can also eliminate the need for handheld controllers via gesture control.
More importantly, vision can convincingly locate the user within the virtual world in a way that is not possible today. The spooky disembodied hands in “The Climb” can be correctly connected to the user’s body, giving a feeling of immersion that goes well beyond the visual.
The point at which that becomes possible is the point a compelling user experience has been created. What makes it possible is vision; until now seen as the domain of augmented reality.
The downsides of including vision are that it adds expense, for the sensors plus a new vision-specific processing unit, as well as delay. Google’s Tango project plus recent events in the auto industry have shown us how much more there is to do on the software side but my opinion is that the benefits outweigh the costs, especially since I am convinced that current generation ‘blind’ headsets will soon be regarded as incomplete products.
So my conclusion is that we are not there yet for VR. It will take another hardware cycle to get the equipment right, and the succesful products will leverage a whole new set of skills coming from a different set of companies.