Actual Reality Beckons at Conference on Future of Virtual Reality

Article Content

A multi-panel VR display draws a crowd.

About a week before researchers gathered at the University of California, San Diego for a conference on the “Future of Virtual Reality,” actual reality was unfolding, albeit tragically, in the Syrian City of Palmyra. The ancient and historically significant Temple of Bel had been demolished by members of the ISIS military group. No one would ever set foot in the Temple of Bel again.

Or would they?

With the right images and photogrammetry techniques, the Temple could one day be recreated as it looked before it was destroyed – not in marble or stone, but in virtual reality. It’s an idea that’s been explored for nearly a decade at the UC San Diego Qualcomm Institute, one of the world’s foremost centers for digital archaeology, a long-time investor in virtual reality research and the meeting ground for the two-day conference.

Jürgen Schulze

With world heritage sites increasingly threatened by global conflict and climate change, so-called “rescue archaeology” – the creation of 3D virtual models of important sites – is a growing field. An example is Project Mosul, created by UC San Diego alumnus and QI affiliate Matthew Vincent and one of the many virtual reality (VR) and augmented reality (AR) projects discussed at the Future of VR, which was organized by QI virtual reality research scientist Jürgen Schulze and funded through a grant from the California Institute for Telecommunications and Information Technology’s Strategic Research Opportunities program (QI is the UC San Diego division of Calit2). Project Mosul uses digital photographs crowd-sourced from people around the world – including U.S. soldiers – to create 3D virtual models of cultural heritage sites destroyed in Mosul, Iraq. When the photos are “stitched” together into a stereoscopic model and then viewed with 3D glasses, the effect is photo-realistic and so convincing, some say, “it’s like being there.”

The applications for such technologies extend well beyond archaeology and into gaming, education, healthcare, training scenarios and even tourism – a scenario that provoked something of a philosophical discussion amongst the conference attendees. (“Can you really say you’ve visited the Temple of Bel if you’ve never actually set foot in the sands of Syria?”)

Among those attendees were faculty members from UC San Diego, including QI Director Ramesh Rao and affiliates Falko Kuester, Thomas Levy, Thomas DeFanti, Sheldon Brown and Peter Otto (Indiana University Prof. Margaret Dolinsky and Virtual Reality Producer Patty Rangel also spoke at the event). Also attending were a number of representatives from Qualcomm, Sixense, SimTable, OSSIC, Aero Glass and LSTC – industries that are all banking on the success of virtual reality in the coming years.

And it’s certainly not a guaranteed success. A week prior to the conference, The Economist published its own take on what will become of the field and its associated technologies. The title of the article? “Grand Illusions.” It’s a double entendre that alludes to both the widespread skepticism of virtual reality’s staying power (particularly given its reliance on cumbersome headsets) and a simultaneous reverence for the technology’s mind-bending capacity to alter the world as we know it.

“Virtual reality is still far from the verisimilitude imagined by novelists and screenwriters,” stated the article. “It is hard to tell which (if any) of the forthcoming headsets will take off. But it is also hard to believe that nothing of interest will be done with it. This time, for virtual reality, actual reality beckons.”

This sentiment was echoed by Schulze in his remarks at the UC San Diego conference, which featured sessions on how VR is already being used in industry and academia, as well as recent advances in panoramic 3D video and advanced spatial audio that will enable future applications.

A conference attendee tests a virtual reality headset at the “Future of Virtual Reality” conference.

“What we’re hoping,” said Schulze, “is that this time virtual reality will stick and stay … and we will see this field flourish and take tech to a new level.

“It’s possible,” he added, “and it’s up to us to make it happen.”

Predictions about the future value of VR and AR markets are all over the map. According to conference speaker Stuart Volkow of Venture DNA, “One market research firm says the industry will be worth $150 billion per year by 2020 and another says it will be worth $2.2 million. There’s a little bit of a delta there, and no one really knows what’s going to happen.”

Still, said Volkow, “You can count on the fact that there is an industry happening. There are lots of tadpoles around the ocean.”

Volkow noted that among the world’s nearly 7 billion people, half now have Internet access, and just before the conference Facebook announced that one in seven people worldwide had used Facebook in a single day. More than 500,000 cardboard virtual reality kits have been sold by Google and there are currently 250 VR apps in the Google Play store (most are free). And with Facebook announcing last year that it was investing heavily in Oculus Virtual Reality – the creators of the Oculus Rift device – “the addressable market,” added Volkow, “is huge.”

What’s more, AR and VR won’t just be about gaming, which is often cited as a prime application of virtual and augmented reality. Volkow noted that market predictions see the population of hardcore gamers leveling off, although the population of “casual” gamers (“those who play Candy Crush”) might continue to grow.

Emerging educational and industrial markets for virtual reality are where the real growth is projected, and all indications are that instead of a market dominated by a homogenous form of virtual reality transmitted via a single device, we’ll use multiple devices – including our mobile phones – to view immersive augmented, virtual and mixed reality environments. Volkow also mentioned emerging applications in journalism, real estate, travel, trainings and simulation, noting that a company called Augumenta has developed an augmented reality repair manual for the oil and gas industry.

“One problem, though, is that there are a lot of conflicting standards for these devices, which is confusing for consumers, difficult for developers and difficult for the general public to figure out how to use them and which one to buy,” he said. “Is 360-degree video VR? Your VR and my VR might be different things, and it depends on whether or not you’re orthodox or conservative about it. If we include that the market is much bigger. It still remains to be seen which one becomes standardized. That is the friction – that’s going to take a while to settle out.”

In the meantime, the wide variety of demos and presentations at the conference suggest there is no shortage of applications being developed for both AR and VR technologies. “Virtual archaeology,” said UC San Diego Anthropology Professor and conference speaker Thomas E. Levy, “is becoming a scientific discipline in its own right,” with its own international conferences, published books and research community. Levy reminisced about his days working at excavation sites in the Middle East in the ‘90s, when “the most high-tech thing we had was a four-color Bic pen.” Seeking a way to bring back as much data as possible, Levy and his colleagues began to “pre-adapt ourselves to this world of virtual reality” by employing robotic total stations, laser scanners, drones and 3D visualization systems, as well as a number of data capture and curation tools, many of which were developed at the Qualcomm Institute.

Amir Rubin

Virtual reality also has the capacity to “democratize healthcare on a global scale … and make people’s lives better all over the world,” said keynote speaker Amir Rubin, co-founder and CEO of Sixense, which creates control platforms for digital media, including virtual reality. Rubin said he “caught the VR bug” 20 years ago after recovering from injuries he sustained in the Lebanese war and encountering simulation environments via the U.S. Navy.

“When I put that 40-pound headgear on me, I felt that if I could have done my rehab using these kinds of tools, it would have been more effective,” he said, noting the effectiveness of VR environments provided by games like Limbs Alive, which aids in stroke recovery. “It was a life-changing experience to think I could have had the treatment personalized to me.”

Rubin noted that Sixense is working to overcome some of the challenges inherent to making VR more mainstream, in particular the cumbersome cables that are part and parcel of many head-mounted VR displays, such as Oculus Rift. Sixense’s STEM System™, for example, is a wireless, motion-tracking platform for video games and VR that enables players to interact naturally and intuitively with games by tracking full position and orientation at all times, whether at the desktop or throughout the entire living room.

Rubin added that the sense of “presence” enabled by wireless VR is important for gamification, but he also echoed Volkow’s prediction that gaming will be the least important segment of VR.

“Training scenarios are another application,” he noted. “With welding, for example, it’s very expensive from a liability insurance perspective to do traditional training. A real welding gun is dangerous, as you can imagine,” but a virtual environment provides a safe and cost-effective means of training welders and other professionals who require similar instruction.

Other researchers at UC San Diego and beyond are working to make those simulation environments more realistic from a sensory perspective – an effort rife with obstacles and challenges. Serafin Diaz of Qualcomm Computer Vision R&D, spoke of the company’s “obsession” with VR and AR tracking systems, which relay the orientation of the user’s point of view. Said Diaz, “In AR, if you fail in tracking, the magic goes away; in VR you can fail a little bit but later on you get a headache.”

What’s more, explained Arnaud Paris of VideoStitch, “If you really want a feeling of immersion, you can’t just put someone in a flat 360 environment.” His company’s premiere product attempts to solve the problem caused by what he says are “VR and 3D fighting each other. If you focus on 3D your VR will suffer from terrible stitches.”

And then there’s the problem of budgetary restrictions. As QI Programmer Analyst Andrew Allen of UC San Diego remarked, if virtual reality is ever to be as realistic as actual reality, audio will play a big role. Researchers in QI’s Audio Spatialization Laboratory, including lab director Peter Otto, are also working to create software for diverse applications in multi-channel and spatial audio (audio that corresponds with the surrounding environment), including binaural and multi-channel sound design environments and a variety of spatial audio imaging packages – some of which were displayed at the conference.

But VR production studios, lamented Allen, typically limit their audio expenditures to around one percent of the total budget. He recalled one studio’s computer memory footprint for audio was a mere five megabytes.

“There’s an implicit audio/visual correlation in VR,” Allen said during his talk. “Sound has to interact with the geometry of the space. It will reverberate more when a voice comes from inside a cathedral, for example. But as far as consideration of acoustics, it’s very coarse right now. Typically, if the user is in a large room, we switch on the large reverb; in a small room, we switch on the small reverb.”

David Brin

Overcoming virtual reality’s reputation for being coarse and cumbersome is perhaps the greatest hurdle for researchers working to refine the science. But for every remark at the Future of Virtual Reality conference about progress to be made or challenges to overcome, there was another vote of confidence for the field, including this pointed reminder from keynote speaker and renowned science fiction author David Brin: “Always there are optimists who think (a technology) is going to change everything for the better; always there are grouches who say the result is going to be catastrophic. The grouches are usually right – in the short-term.”

Brin noted that virtual and augmented reality have the potential to transform a dispersed, multinational society into one collective “village of nine billion people,” where it’s possible to instantly download someone’s “reputation profile” as you pass them on the street and get to know people on the intimate level humans once did when they lived in small villages.

“With virtual reality you have these augmentations of vision, memory, attention and reach. The village is returning, and once again you have grouches saying it’s going to cause a lot of problems and transcendentalists saying it’s going to make us god-like.

“Most ancient villages were under systems of feudalism and were automatically repressive – by police, leaders and local neighborhood gossip – this is one path for (this new village),” Brin continued. “Another is a village where people are tolerant of eccentricities. You will choose which one. This kind of connectivity affects the power of citizenship. It is resilience that comes in when anticipation fails.”