Student interacts with HTC Vive virtual reality system in the newly-opened VR Lab. Photos by Erik Jepsen/UC San Diego Publications

Giving Students a Place to Prep for Tomorrow’s Virtual (Reality) Economy

Computer science department opens high-tech teaching lab for undergraduates

The laboratory looks like a cross between a classroom and a tech pavilion at the annual Consumer Electronics Show in Las Vegas. There are virtual-reality headsets everywhere, and large flat screen 3D displays. College students work at computers, while teammates wearing goggles look from side to side, occasionally ducking or recoiling, as they react and engage with the virtual environments visible in their head-mounted displays.

Welcome to the Virtual Reality Lab—the first of its kind at an American university.

UC San Diego officially opened its VR Lab in early May with an open house. Located in the Computer Science and Engineering (CSE) department, the lab was created specifically to serve undergraduate students who take courses in software programming for virtual and augmented reality.

“Other universities may have labs that serve faculty and graduate students doing research in the field of virtual reality, and they may have a few VR systems for research use only,” said CSE Chair Dean Tullsen. “But to our knowledge, the VR Lab is the first designed primarily for undergrads learning how to create VR content.”

According to computer science junior Connor Smith, president of the student-run Virtual Reality Club at UC San Diego, the very existence of the VR Lab has galvanized interest in virtual reality across the campus. “We now have the biggest VR classroom in the country,” he said. “It’s really cool to say our school has this, and it has multiplied the VR community on campus.” Indeed, in the past six months membership in the VR Club shot up from fewer than 30 to more than 300 members.

The lab was two years in the making. It is equipped with 25 standalone systems combining an Oculus Rift head-mounted display, two controllers for navigating inside VR environments, a computer workstation equipped with a high-end graphics card, various peripherals and a custom-made breakout box for plugging in and unplugging cables. “This way students don’t have to get on their hands and knees under the table every time they have to plug in the Oculus Rift,” noted the VR Club’s Smith.

With 10 more Oculus Rifts with controllers about to be added, the VR Lab now easily accommodates classes of up to 70 students. “There will be a full set of equipment for each team of two students, as long as teams don’t show up at the same time,” said CSE adjunct professor Jurgen Schulze cautiously. “When that happens, two teams may have to alternate using a single PC, or bring a laptop computer, and we just have to hope they won’t conflict too much as they develop their respective projects.”

Schulze—a research scientist and VR expert in the Qualcomm Institute—co-led the project with Music professor Shahrokh Yadegari, who also directs the Sonic Arts Research and Development group in the Qualcomm Institute (QI). .

Another sense: in addition to sight and sound, VR can mimic the sense of touch. Student tests a novel haptic glove using soft robotics concepts now in development at UC San Diego.

In 2015, Schulze applied for a grant from the campus-wide Integrated Digital Infrastructure (IDI) program, which earmarked funds for technologies to support the university’s teaching mission. IDI provided $20,000 to purchase computers and other hardware, including 360-degree cameras, Leap Motion hand-tracking devices, and an HTC Vive VR system. Information Technology Services (ITS), UC San Diego’s central IT organization, provided 26 graphics PCs, and Oculus gave the lab 140 Rift goggles.

The opening of the lab coincides with the launch of new and updated courses in the CSE department that integrate VR into the computer-science curriculum. This spring, Schulze is teaching 70 students about Virtual Reality Technology (CSE 190), with an emphasis on immersive VR systems, including those based on head-mounted displays. Students are required to submit three separate VR projects during the quarter.

In the winter quarter, Schulze used a not-yet-fully-outfitted VR Lab to teach 3D User Interaction (CSE 165), requiring students to develop interaction techniques for the Oculus Rift and other VR devices. The course previously focused on user interaction design, with less emphasis on 3D.

Two students who took Schulze’s CSE 165 course in the winter quarter are graduating with computer-science degrees this summer. Both demonstrated VR apps at the open house, and both say they hope to find jobs that will allow them to use their new-found VR skills. “I never did VR before taking CSE 165 last quarter,” said Sun, a fifth-year computer science major. “The transition was not terribly difficult, especially with a great teacher like Professor Schulze. Now, I have my VR app to show potential employers when I apply for a job.”

For his part, computer-science senior Ty Stahnke opted to convert a 2D rhythm game he was working on into a 3D environment, which he demonstrated at the open house. “A lot of what we learned in that course was how to make appropriate 3D interfaces for the user so the interaction is more intuitive and easier to play,” said Stahnke. “I definitely want to work in VR but don’t know if I want to go into entertainment, or medical, or another area. I guess I’ll apply to a lot of different VR-related jobs.”

“It was an awesome course and it was essentially new, because previously it didn’t have the VR aspect,” added Stahnke. “We got a ton of hands-on experience, but at the same time, we learned a lot about the concepts and what goes on behind the scenes when you develop VR content.”

Student in CSE 190 this quarter wears Oculus Rift VR headset to interact with environmental app that lets viewer train to reduce carbon dioxide in the atmosphere.

This fall, CSE assistant research professor Nadir Weibel will teach Ubiquitous Computing (CSE 118), and he plans to use the VR Lab to give students the ability to understand the limitations and opportunities of VR and augmented reality, focusing on a ‘mixed reality’ surgery app he and his students are developing. At the open house, Weibel and Gasques-Rodriguez demonstrated an early prototype of the app on a Microsoft HoloLens system. It uses ‘mixed reality’ – i.e., real medical scans of the patient draped over a virtual computer model of the same patient in 3D – giving the surgeon with goggles a kind of “X-ray vision”.

“The field of view is still too narrow for what’s needed in the operating room, but we think that problem can be solved,” said Weibel in demonstrating the system. “The real question is whether mixed reality can be helpful to surgeons, for example when doing a biopsy. The jury is out, but we are working very hard to get the technology to the point where it will make an appreciable difference in surgical outcomes.”

CSE is also making the VR Lab available for undergraduate courses in other departments. In the spring quarter, Visual Arts professor Benjamin Bratton is teaching a course on Design Research Methods (VIS 135), including data visualization. Bratton is now scheduling time for students in the VR Lab to introduce them to VR applications and modeling tools.

Last quarter, Music professor Shahrokh Yadegari—whose team developed the surround-sound audio library for the lab—taught Music Technology (MUS 176), focusing on spatial audio and VR applications. Students were given access to the lab to learn how to combine audio and video for the total VR experience.

At the open house, Yadegari’s graduate student Grady Kestler demonstrated an interactive environment put together by combining visual effects with spatialized sound effects. “We fully implemented the library in an application programming interface (API), and Professor Yadegari asked me to port the implementation to an audio server,” he explained. “We are still catching all the bugs, but by the beginning of next year, we’ll have it installed on all of the computers in the VR Lab. Students will be able to run the API and anyone creating VR stuff will be able to use it.”

CSE professor Rajesh Gupta approved the plan to build the VR Lab in 2015 while he was department chair. “I wanted the space to be used for undergraduates and for teaching,” recalled Gupta. “When it became clear that CSE could create an entirely new facility with support from other partners on campus, we knew that the VR Lab would be a good fit for CSE’s future.”

Each two-person team in the virtual reality courses in CSE share a graphics PC, an Oculus Rift head-mounted display, a controller and other VR equipment.

Looking to the future, Schulze is revamping his Computer Graphics course (CSE 167) to include a hands-on component applying graphics to 3D virtual environments. He is also adapting his VR curriculum to learners around the world – through the first-ever (recently announced) series of online courses on VR App Development over the edX learning platform (jointly with CSE professor Ravi Ramamoorthy, director of the Center for Visual Computing). The first course launches May 15.

Schulze also expects faculty in other departments will want to integrate the VR Lab into what they teach. For example, Visual Arts professor Ricardo Dominguez will use the lab next year for his Virtual Environments (ICAM 120) course. In the past, the course focused largely on concepts, but Dominguez now sees a way to give his students first-hand experience with creating content for VR and augmented reality to reinforce the underlying theory.

In another sign that virtual reality is taking hold at UC San Diego, one of the hottest recent tech startups created by university alumni is a VR company that has hired 80 percent of its software development staff from among CSE graduates. Nanome, Inc., is winning awards and prizes for bringing VR tools to education in math, chemistry and other fields under its CEO, UC San Diego nanoengineering alumnus Steven McCloskey (B.S. ’15). The VR Lab and new VR courses are expected to funnel many more students to follow in the footsteps of Kai Wang. After working for nine months as an intern with Nanome, Wang (B.S. ’17) graduated in the winter quarter. “I worked with Jurgen Schulze on a VR project before graduating,” noted Wang, who demonstrated Nanome VR applications at the May 2 open house. “Before I graduated, I met Steve McCloskey, who was looking for interns to work on the VR software application, and I joined the company full-time after graduation.”

The VR Club’s president, Connor Smith, believes that the university can train many more students to work in this field. “The sky’s the limit,” he enthused. “I’m still not decided on a particular job after graduation, but I’ve even thought about doing a startup in VR, particularly using it for educational programming where it can make a big difference.”

Meanwhile, UC San Diego may not have the only undergraduate VR teaching lab for long. According to Schulze, UC Berkeley is now developing a similar classroom, and as more and more jobs open up in the emerging VR industry, it’s likely that other universities will follow suit.