Skip directly to content

Minimize RSR Award Detail

Research Spending & Results

Award Detail

Doing Business As Name:University of Illinois at Urbana-Champaign
  • Sarita V Adve
  • (217) 333-8461
Award Date:08/02/2021
Estimated Total Award Amount: $ 1,000,000
Funds Obligated to Date: $ 1,000,000
  • FY 2021=$1,000,000
Start Date:10/01/2021
End Date:09/30/2024
Transaction Type:Grant
Awarding Agency Code:4900
Funding Agency Code:4900
CFDA Number:47.070
Primary Program Source:040100 NSF RESEARCH & RELATED ACTIVIT
Award Title or Description:CCRI: New: An Open End-to-End Extended Reality System Infrastructure: Enabling Domain-Specific Edge Systems Research
Federal Award ID Number:2120464
DUNS ID:041544081
Parent DUNS ID:041544081
Program:CCRI-CISE Cmnty Rsrch Infrstrc
Program Officer:
  • Yuanyuan Yang
  • (703) 292-8067

Awardee Location

Street:1901 South First Street
Awardee Cong. District:13

Primary Place of Performance

Organization Name:Board of Trustees of the University of Illinois
Street:506 S. Wright Street
Cong. District:13

Abstract at Time of Award

Extended reality (XR), which encompasses virtual, augmented, and mixed reality (AR, VR, MR) and also referred to as immersive computing, is expected to pervade most human endeavors --- it will affect the way we teach, conduct science, practice medicine, entertain ourselves, train professionals, interact socially, and more. Many have said it will be the next interface for most of computing. While current XR systems exist today, they are far from providing a tetherless experience approaching perceptual abilities of humans. There is a gap of several orders of magnitude between what is needed and achievable in performance, power, and usability, requiring deep innovations from system researchers. At the same time, with the end of Dennard scaling and Moore's law, application-driven specialization or domain-specific computing has emerged as a key architectural technique to meet the requirements of emerging applications, Computer architects have responded with an explosion of research on highly efficient accelerators, targeting machine learning and other domains. To truly achieve the promise of efficient domain-specific computing in general and for the XR domain in particular, however, requires systems researchers to broaden their portfolio beyond specialization for individual accelerators. Instead, researchers must develop the science for specializing for a domain-specific system which may consist of multiple sub-domains requiring multiple parallel heterogeneous accelerators that interact with each other to collectively meet end-user demands. A key obstacle to domain-specific systems research for XR is that (until our work) there have been no open source benchmarks or testbeds covering the entire XR workflow to drive such research. This project develops an open source end-to-end infrastructure for XR devices. It builds on an initial research prototype, ILLIXR (Illinois Extended Reality Testbed). The system is being designed to contain state-of-the-art components for a complete XR workflow, an extensible runtime that orchestrates the scheduling of these components, and extensive telemetry support to measure performance, power, and end-to-end quality of experience metrics. The system is extensible and supports a variety of operating systems (e.g., Linux, Android) and heterogeneous platforms (e.g., NVIDIA Jetson, Qualcomm Snapdragon, etc.), sensors (e.g., cameras, IMUs, etc.), and various XR applications. It enables new research opportunities in all parts of the computing stack to tackle end-to-end XR system innovations that were previously not possible. Systems researchers benefit from using the infrastructure to drive new research in post-Moore domain-specific systems, in the areas of computer architecture, programming languages, compilers, runtime systems, and security and privacy. The end-to-end infrastructure drives new techniques in co-designed systems that are optimized for end-to-end user experiences. For applications, XR encompasses multiple sub-domains such as computer vision, robotics, graphics, signal processing, and machine learning. Algorithms researchers in these areas can prototype and test new algorithms that are optimized for end-to-end system efficiencies without worrying about implementing the rest of the stack, and XR researchers in particular will be able to design systems optimized for the end-to-end user experience. This work addresses two of the most important problems in computing -- dealing with the end of Moore's law and designing systems that achieve the potential of immersive computing. Both have the potential for tremendous impact on society at large. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

For specific questions or comments about this information including the NSF Project Outcomes Report, contact us.