A $1.5 million grant from the National Science Foundation will provide additional impetus to a University of Rochester initiative applying augmented and virtual reality in health, education, product design, remote communication, entertainment, and other fields.
The grant will enable 62 doctoral students to be trained in the skills needed to advance AR/VR technologies and will also help them gain an appreciation for the broader cultural and societal implications of the technologies, says Mujdat Cetin, the principal investigator behind the grant. Other Rochester faculty supporting this initiative are Jannick Rolland, the Brian J. Thompson Professor of Optical Engineering; Michele Rucci, professor of brain and cognitive sciences; and Zhen Bai, assistant professor of computer science.
The trainees—chosen from multiple departments, including electrical and computer engineering, optics, biomedical engineering, brain and cognitive sciences, computer science, and neuroscience—will also participate in industrial internships at companies such as Facebook, Microsoft, Nvidia, and Vuzix, which have provided letters of support for the program.
The companies “will be eager to consider the trainees for permanent research positions after graduation,” says Cetin, the interim director of the Goergen Institute for Data Science and a professor of electrical and computer engineering.
As many as 300 other STEM graduate students at the University will participate in aspects of the training and professional development.
“I’m really excited,” Cetin says. “This project builds very well on the AR/VR initiative the University already has in place.”
Typically, PhD students researching AR/VR at Rochester and other universities “are educated only in the teaching practices of their own department or their faculty advisor,” Cetin says.
“But they never get perspectives on other aspects of AR/VR.”
For example, virtual reality systems are being used in physical therapy and neurological rehabilitation, but a computer science student trained in human-computer interfaces may not be able to make meaningful contributions to these systems due to insufficient exposure to the biomedical engineering and neuroscience aspects of the problem.
“This grant will provide that [exposure],” Cetin says. “It will be unique.”
At the core of the program are three new courses.
- A required course providing “a broad introduction to AR/VR systems.” It will “build a common base of understanding and knowledge for all trainees in the program as well as provide a foundation on which they can build their research,” according to the grant application. To ensure the course is useful to diverse students with different backgrounds, it will consist of: lectures providing an overview and introduction to all aspects of AR/VR; and an individualized, guided self-study component for each student that provides more intensive training on aspects of AR/VR in which the student is less knowledgeable.
- A course offering nine modules—three per semester—that “span all layers” of the AR/VR challenge, examining AR/VR in the context of optics, silicon (next-generation computing), visual perception, computer audition, studies of the human brain, deep learning and visual recognition, brain-computer interfaces, 3D interfaces, and collaborative education and professional training.
- A practicum course trainees can take after the modular course, working in small teams composed of students from multiple departments to develop an end product that addresses a research question or application.
In addition to industry internships, trainees will have opportunities to participate in immersive professional development encounters with leaders from industry and academia, participate in an annual program showcase and student-run conference, and take on supervisory roles in undergraduate capstone projects.
The program is designed so that it can be completed by trainees without a significant increase in the amount of time it takes to complete their degrees.
The University recently launched a multi-faceted initiative to support AR/VR research and to establish AR/VR research facilities across the campus. This includes the Center for Augmented and Virtual Reality, a flagship program of the Del Monte Institute for Neuroscience that brings together 55 faculty members from across University. The University has launched a pilot award program to support exploratory projects related to the science, technology, and applications of AR/VR. The University has also started establishing facilities and mechanisms to enable AR/VR research, including a hands-on AR/VR creation and exploration space called Studio X planned for the Carlson Science and Engineering Library and an AR/VR equipment loan process through the Digital Media Laboratory.
“All of this has created a lively research environment on AR/VR throughout the campus. What is currently missing in this initiative, and what is needed to help realize the transformative potential of AR/VR technologies is a structured, multi-disciplinary PhD training program with AR/VR as its central theme,” Cetin says.
The grant is one of 17 awarded nationwide to institutions in the latest round of funding from the National Science Foundation (NSF) Research Traineeship (NRT) program. The goal of the program is to develop and implement graduate education traineeship models in science, technology, engineering and mathematics (STEM) fields. These projects will immerse students in interdisciplinary research and deliver training in career-aligned skillsets, that will enable the next generation of scientific leaders to tackle complex, societal problems.
