Trainers, Niantic and Qualcomm are working on a new virtual and augmented reality platform, the “AR glasses,” which is designed for AR and VR hardware.
Niantic announced their collab with Qualcomm on a reference design solution that will mix hardware, software and cloud services for mobile phones. Their plan is to find ways to make the first affordable AR smart glasses (see the image below) and create next-generation AR experiences through the Niantic Real World Platform and build new ways to map, share and understand reality.
The Niantic Real World Platform has been proven out by hundreds of millions of users, through consumer experiences such as Ingress, Pokemon GO and Harry Potter: Wizards Unite.
Here is an overview of the world’s first 5G XR platform AR glasses, thanks to Qualcomm:
“The Qualcomm® Snapdragon™ XR2 Platform is the world’s first 5G-supported extended reality (XR) platform. It unites Qualcomm Technologies, Inc.’s 5G and AI innovations with our industry leading XR technology to usher in a new era of mobile computing. This premium-quality platform unveils custom features and boasts multiple firsts that can be scaled across augmented reality (AR), virtual reality (VR) and mixed reality (MR).
The Snapdragon XR2 Platform is packed with significant performance improvements compared to our current widely adopted premium-tier XR platform[1] to deliver 2x the CPU and GPU performance, 4x more video bandwidth, 6x higher resolution and 11x AI improvement. The Snapdragon XR2 Platform is the world’s first XR platform to support seven (7) concurrent cameras and a dedicated computer vision processor. Additionally, this is the first XR platform to enable low latency camera pass-through to unlock true MR, which allow users to see, interact and create a hybrid of the virtual and real world while wearing a VR device. To meet the demands of truly immersive XR, the platform has customized visuals, interactivity and audio technologies, all of which utilize the foundational technology of AI and option of 5G connectivity.
Visuals
In order to experience a heightened sense of realism in XR, it is imperative to close the visual gap between the real and virtual world. This requires cutting-edge display and graphics. The Snapdragon XR2 Platform takes a leap in core GPU processing by supporting 1.5x the pixel rate and 3x the texel rate for efficient, high graphics rendering1. XR specific features like foveated rendering with eye tracking and enhanced variable rate shading for smoother refresh rates help render heavy workloads, while maintaining low power consumption. The Snapdragon XR2 display panel offers up to 3K by 3K resolution per eye at 90 frames per second (fps) and is the first XR platform to support 8K 360o videos at 60 fps for photorealistic visuals in streaming and local playback. Custom silicon was developed for unique AR displays to help reduce overall system latencies to maintain immersive AR experiences.
Interactivity
With XR, for the first time a user can be virtually teleported to a new environment. To do this accurately and efficiently, the Snapdragon XR2 introduces support for seven (7) concurrent cameras and a custom computer vision processor. Multiple concurrent cameras enable real-time and highly accurate tracking of the head, lips and eyes together with 26-point skeletal hand tracking. Computer vision provides highly efficient scene understanding and 3D reconstruction. Together, these features allow users to be transported to new environments where they can intuitively interact within a digital world.
Audio
n an augmented or virtual world, sound is the key to every immersive encounter. The Snapdragon XR2 Platform offers next-level audio layers in a rich, 3D spatial sound while providing crystal-clear voice interactions to deepen immersion. The platform has a custom built always-on, low power Qualcomm® Hexagon™ DSP to support hardware accelerated features like voice activation and context detection to help keep users’ head in the digital world, while having an ear in the real world.
AI & 5G
Two technologies shaping the future of communications and computing are AI and 5G. Qualcomm Technologies is leading the way in driving these technologies across mobile computing, including XR. Multiple Snapdragon XR2 features have been improved using AI, including visuals, interactivity and audio in order to immerse users in a smarter reality. The Snapdragon XR2 Platform is the world’s first XR platform to support 5G connectivity, which has the potential to unleash a tidal wave of innovative XR experiences that require low latency and ultra-fast data speeds. For example, 5G enables truly boundless XR using split processing between a device and edge cloud to present photorealistic and high-quality experiences without cords or being confined to any room.
The Snapdragon XR2 Platform delivers innovation that the new digital era requires. Multiple OEMs are committed to commercializing devices with the Snapdragon XR2 Platform and other customers are in various stages of prototyping and evaluation. Today marks a significant milestone for the entire XR industry and we are excited to continue our efforts to transform how the world connects and communicates using XR.”
Niantic didn’t share any details but they confirmed they are working together with Qualcomm Technologies “to define a true end-to-end architecture – inclusive of hardware, software and cloud technologies.”
