foveated rendering
{{short description|Rendering technique in which image quality is reduced in the peripheral vision}}
Foveated rendering is a rendering technique which uses an eye tracker integrated with a virtual reality headset to reduce the rendering workload by greatly reducing the image quality in the peripheral vision (outside of the zone gazed by the fovea).
A less sophisticated variant called fixed foveated rendering doesn't utilise eye tracking and instead assumes a fixed focal point.
History
Research into foveated rendering dates back at least to 1991.{{cite journal | doi=10.1007/s41095-022-0306-4 | title=Foveated rendering: A state-of-the-art survey | year=2023 | last1=Wang | first1=Lili | last2=Shi | first2=Xuehuai | last3=Liu | first3=Yi | journal=Computational Visual Media | volume=9 | issue=2 | pages=195–228 | s2cid=253523241 | arxiv=2211.07969 }}
At Tech Crunch Disrupt SF 2014, Fove unveiled a headset featuring foveated rendering. This was followed by a successful kickstarter in May 2015.
At CES 2016, SensoMotoric Instruments (SMI) demoed a new 250 Hz eye tracking system and a working foveated rendering solution. It resulted from a partnership with camera sensor manufacturer Omnivision who provided the camera hardware for the new system.{{Cite web|last=Mason|first=Will|date=2016-01-15|title=SMI's 250Hz Eye Tracking and Foveated Rendering Are For Real, and the Cost May Surprise You|url=https://uploadvr.com/smi-hands-on-250hz-eye-tracking/|access-date=2020-06-18|website=UploadVR|language=en-US}}
In July 2016, Nvidia demonstrated during SIGGRAPH a new method of foveated rendering claimed to be invisible to users.
In February 2017, Qualcomm announced their Snapdragon 835 Virtual Reality Development Kit (VRDK) which includes foveated rendering support called Adreno Foveation.{{Cite web|date=2017-02-23|title=Qualcomm Introduces Snapdragon 835 Virtual Reality Development Kit|url=https://www.qualcomm.com/news/releases/2017/02/23/qualcomm-introduces-snapdragon-835-virtual-reality-development-kit|access-date=2021-06-01|website=Qualcomm|language=en}}{{Cite web|title=Foveation in Game Engines|url=https://vr.tobii.com/sdk/learn/foveation/rendering/in-game-engines/#qualcomm-adreno-foveation|access-date=2021-06-01|website=vr.tobii.com}}
During CES 2019 on January 7 HTC announced an upcoming virtual reality headset called Vive Pro Eye featuring eye-tracking and support for foveated rendering.
In December 2019, Facebook's Oculus Quest SDK gave developers access to dynamic fixed foveated rendering, allowing the variation in level of detail to be changed on the fly via an API.{{Cite web|url=https://venturebeat.com/2019/12/22/oculus-quest-gets-dynamic-fixed-foveated-rendering/|title=Oculus Quest gets dynamic fixed foveated rendering|date=2019-12-22|website=VentureBeat|language=en-US|access-date=2020-01-21}}
On January 4, 2022, Sony announced that their follow-up to PlayStation VR will include eye tracking and foveated rendering.{{Cite web|url=https://blog.playstation.com/2022/01/04/playstation-vr2-and-playstation-vr2-sense-controller-the-next-generation-of-vr-gaming-on-ps5/|title = PlayStation VR2 and PlayStation VR2 Sense controller: The next generation of VR gaming on PS5|date = 4 January 2022}}
On June 5, 2023, Apple announced that the Apple Vision Pro extended reality headset includes dynamic foveated rendering.{{Cite web|url = https://developer.apple.com/visionos/|title = Discover visionOS.|date = 7 June 2023}}
Use
According to chief scientist Michael Abrash at Oculus, utilising foveated rendering in conjunction with sparse rendering and deep learning image reconstruction has the potential to require an order of magnitude fewer pixels to be rendered in comparison to a full image. Later, these results have been demonstrated and published.
Eye-tracked foveated rendering was demonstrated in products such as the Meta Quest Pro (released in 2022) and the Sony PlayStation VR2 (released in 2023) headsets.
See also
References
{{reflist|refs=
{{cite web | url=http://www.digitaltrends.com/virtual-reality/nvidia-research-foveated-rendering-vr-smi/ | title=Nvidia plans to prove that new method improves image quality in virtual reality | date=2016-07-22 | publisher=Digital Trends | last=Parrish|first=Kevin | access-date=2017-02-02}}
{{cite web | url=http://sensics.com/understanding-foveated-rendering/ | title=Understanding Foveated Rendering | date=2016-04-11 | publisher=Sensics | access-date=2017-02-04 | archive-date=2017-12-01 | archive-url=https://web.archive.org/web/20171201031817/http://sensics.com/understanding-foveated-rendering/ | url-status=dead }}
{{cite web | url=http://uploadvr.com/smi-hands-on-250hz-eye-tracking/ | title= SMI's 250Hz Eye Tracking and Foveated Rendering Are For Real, and the Cost May Surprise You | date=2016-01-15 | publisher=UploadVR | last=Mason|first=Will | access-date=2017-02-02}}
{{cite web | url=https://blogs.nvidia.com/blog/2016/07/21/rendering-foveated-vr/ | title= NVIDIA Partners with SMI on Innovative Rendering Technique That Improves VR | date=2016-01-21 | publisher=Nvidia | access-date=2017-02-02}}
{{Cite web|last=Kaplanyan|first=Anton|title=DeepFovea: AR/VR rendering, inspired by human vision|website=YouTube|date=2020-05-15|url=https://www.youtube.com/watch?v=o7OpS7pZ5ok&t=1h30m38s|access-date=2020-05-15}}
{{Cite web|url=https://www.uploadvr.com/psvr-2-eye-tracking-foveated-rendering-gdc/|title=PSVR 2 Foveated Rendering Provides 3.6x Faster Performance - Unity | date=2022-04-27 | publisher=UploadVR | access-date=2022-04-27}}
}}
External links
- [https://www.microsoft.com/en-us/research/publication/foveated-3d-graphics/ Microsoft white paper on foveated rendering]
{{Mixed reality}}