Augmented reality#Contact lenses
{{short description|View of the real world with computer-generated supplementary features}}
{{Use dmy dates|date=October 2019}}
File:Suteki Hololens applikation .jpg
File:Navit Reality View next to reality.jpg
Augmented reality (AR), also known as mixed reality (MR), is a technology that overlays real-time 3D-rendered computer graphics onto a portion of the real world through a display, such as a handheld device or head-mounted display. This experience is seamlessly interwoven with the physical world such that it is perceived as an immersive aspect of the real environment. In this way, augmented reality alters one's ongoing perception of a real-world environment, compared to virtual reality, which aims to completely replace the user's real-world environment with a simulated one.Steuer,{{Cite web |url=https://filtermaker.fr/en/augmented-reality/ |title=Defining virtual reality: Dimensions Determining Telepresence |access-date=27 November 2018 |archive-url=https://web.archive.org/web/20220717120913/https://filtermaker.fr/en/augmented-reality/ |archive-date=17 July 2022 |url-status=dead |df=dmy-all }}, Department of Communication, Stanford University. 15 October 1993.[http://archive.ncsa.illinois.edu/Cyberia/VETopLevels/VR.Overview.html Introducing Virtual Environments] {{Webarchive|url=https://web.archive.org/web/20160421000159/http://archive.ncsa.illinois.edu/Cyberia/VETopLevels/VR.Overview.html |date=21 April 2016 }} National Center for Supercomputing Applications, University of Illinois. Augmented reality is typically visual, but can span multiple sensory modalities, including auditory, haptic, and somatosensory.{{cite journal | last1=Cipresso | first1=Pietro | last2=Giglioli | first2=Irene Alice Chicchi | last3=Raya | first3=iz | last4=Riva | first4=Giuseppe | title=The Past, Present, and Future of Virtual and Augmented Reality Research: A Network and Cluster Analysis of the Literature | journal=Frontiers in Psychology | volume=9 | date=2011-12-07 | pmid=30459681 | doi=10.3389/fpsyg.2018.02086 | page=2086| pmc=6232426 | doi-access=free }}
The primary value of augmented reality is the manner in which components of a digital world blend into a person's perception of the real world, through the integration of immersive sensations, which are perceived as real in the user's environment. The earliest functional AR systems that provided immersive mixed reality experiences for users were invented in the early 1990s, starting with the Virtual Fixtures system developed at the U.S. Air Force's Armstrong Laboratory in 1992.{{cite book |doi=10.1109/VRAIS.1993.380795 |chapter=Virtual fixtures: Perceptual tools for telerobotic manipulation |title=Proceedings of IEEE virtual reality Annual International Symposium |pages=76–82 |year=1993 |last1=Rosenberg |first1=L.B. |s2cid=9856738 |isbn=0-7803-1363-1 }}{{Cite news|url=http://www.popularmechanics.com/technology/a22384/hololens-ar-breakthrough-awards/|title=I Saw the Future Through Microsoft's Hololens|last=Dupzyk|first=Kevin|work=Popular Mechanics|date = 6 September 2016}} Commercial augmented reality experiences were first introduced in entertainment and gaming businesses.{{Citation|last=|first=|title=Augmented Reality: Reflections at Thirty Years|url=https://link.springer.com/10.1007/978-3-030-89906-6_1|work=Proceedings of the Future Technologies Conference (FTC) 2021, Volume 1|series=Lecture Notes in Networks and Systems|year=2022|volume=358|pages=1–11|editor-last=Arai|editor-first=Kohei|place=Cham|publisher=Springer International Publishing|language=en|doi=10.1007/978-3-030-89906-6_1|isbn=978-3-030-89905-9|s2cid=239881216|access-date=}} Subsequently, augmented reality applications have spanned industries such as education, communications, medicine, and entertainment.
Augmented reality can be used to enhance natural environments or situations and offers perceptually enriched experiences. With the help of advanced AR technologies (e.g. adding computer vision, incorporating AR cameras into smartphone applications, and object recognition) the information about the surrounding real world of the user becomes interactive and digitally manipulated.{{Cite journal |url=https://doi.org/10.1007/s11831-022-09831-7/ |title=Augmented Reality: A Comprehensive Review|last1=Dargan|first1=Shaveta|last2=Bansal|first2=Shally|last3=Mittal|first3=Ajay|last4=Kumar|first4=Krishan|date=2023 | journal=Archives of Computational Methods in Engineering |volume=30 |issue=2 |pages=1057–1080 |doi=10.1007/s11831-022-09831-7 |access-date=27 February 2024}} Information about the environment and its objects is overlaid on the real world. This information can be virtual or real, e.g. seeing other real sensed or measured information such as electromagnetic radio waves overlaid in exact alignment with where they actually are in space.{{Cite web|url=http://wearcam.org/PhenomenalAugmentedReality.pdf|title=Phenomenal Augmented Reality, IEEE Consumer Electronics, Volume 4, No. 4, October 2015, cover+pp92-97}}Time-frequency perspectives, with applications, in Advances in Machine Vision, Strategies and Applications, World Scientific Series in Computer Science: Volume 32, C Archibald and Emil Petriu, Cover + pp 99–128, 1992.{{Cite book|last1=Mann|first1=Steve|last2=Feiner|first2=Steve|last3=Harner|first3=Soren|last4=Ali|first4=Mir Adnan|last5=Janzen|first5=Ryan|last6=Hansen|first6=Jayse|last7=Baldassi|first7=Stefano|s2cid=12247969|date=15 January 2015|publisher=ACM|pages=497–500|doi=10.1145/2677199.2683590|isbn=9781450333054|chapter=Wearable Computing, 3D Aug* Reality, Photographic/Videographic Gesture Sensing, and Veillance|title=Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction - TEI '14}} Augmented reality also has a lot of potential in the gathering and sharing of tacit knowledge. Immersive perceptual information is sometimes combined with supplemental information like scores over a live video feed of a sporting event. This combines the benefits of both augmented reality technology and heads up display technology (HUD).
Augmented reality frameworks include ARKit and ARCore. Commercial augmented reality headsets include the Magic Leap 1 and HoloLens. A number of companies have promoted the concept of smartglasses that have augmented reality capability.
Augmented reality can be defined as a system that incorporates three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects.{{cite journal |last1=Wu |first1=Hsin-Kai |last2=Lee |first2=Silvia Wen-Yu |last3=Chang |first3=Hsin-Yi |last4=Liang |first4=Jyh-Chong |title=Current status, opportunities and challenges of augmented reality in education... |journal=Computers & Education |date=March 2013 |volume=62 |pages=41–49 |doi=10.1016/j.compedu.2012.10.024 |s2cid=15218665 }} The overlaid sensory information can be constructive (i.e. additive to the natural environment), or destructive (i.e. masking of the natural environment).{{cite web |last1=Rosenberg |first1=Louis B. |title=The Use of Virtual Fixtures as Perceptual Overlays to Enhance Operator Performance in Remote Environments. |work=DTIC |date=1992 |url=https://apps.dtic.mil/docs/citations/ADA292450 |archive-url=https://web.archive.org/web/20190710211431/https://apps.dtic.mil/docs/citations/ADA292450 |url-status=live |archive-date=10 July 2019 }} As such, it is one of the key technologies in the reality-virtuality continuum.{{Cite journal |last1=Milgram |first1=Paul |last2=Takemura |first2=Haruo |last3=Utsumi |first3=Akira |last4=Kishino |first4=Fumio |date=1995-12-21 |title=Augmented reality: a class of displays on the reality-virtuality continuum |url=https://www.spiedigitallibrary.org/conference-proceedings-of-spie/2351/0000/Augmented-reality--a-class-of-displays-on-the-reality/10.1117/12.197321.full |journal=Telemanipulator and Telepresence Technologies |publisher=SPIE |volume=2351 |pages=282–292 |doi=10.1117/12.197321|bibcode=1995SPIE.2351..282M }} Augmented reality is largely synonymous with mixed reality. There is also overlap in terminology with extended reality and computer-mediated reality.
{{toclimit|3}}
Comparison with virtual reality
Augmented reality refers to experiences that are artificial and that add to the already existing reality.{{Cite magazine |url=https://www.wired.com/2009/08/augmented-reality/ |title=If You're Not Seeing Data, You're Not Seeing |last=Chen |first=Brian |date=25 August 2009 |magazine=Wired |access-date=18 June 2019}}{{Cite web |url=http://www.augmentedrealityon.com/ |title=Augmented Reality (AR) |website=augmentedrealityon.com |archive-url=https://web.archive.org/web/20120405071414/http://www.augmentedrealityon.com/ |archive-date=5 April 2012 |url-status=dead |access-date=18 June 2019}}{{cite journal |last=Azuma |first=Ronald |author-link=Ronald Azuma |date=August 1997 |title=A Survey of Augmented Reality |url=http://www.cs.unc.edu/~azuma/ARpresence.pdf |access-date=2 June 2021 |journal=Presence: Teleoperators and Virtual Environments |publisher=MIT Press |volume=6 |issue=4 |pages=355–385 |doi=10.1162/pres.1997.6.4.355|s2cid=469744 }}
In virtual reality (VR), the users' perception is completely computer-generated, whereas with augmented reality (AR), it is partially generated and partially from the real world.{{Cite journal|last1=Carmigniani|first1=Julie|last2=Furht|first2=Borko|last3=Anisetti|first3=Marco|last4=Ceravolo|first4=Paolo|last5=Damiani|first5=Ernesto|last6=Ivkovic|first6=Misa|s2cid=4325516|date=1 January 2011|title=Augmented reality technologies, systems and applications|journal=Multimedia Tools and Applications|language=en|volume=51|issue=1|pages=341–377|doi=10.1007/s11042-010-0660-6|issn=1573-7721}}{{Cite book|title=Virtual, Augmented Reality and Serious Games for Healthcare 1|last1=Ma|first1=Minhua|last2=C. Jain|first2=Lakhmi|last3=Anderson|first3=Paul|publisher=Springer Publishing|year=2014|isbn=978-3-642-54816-1|pages=120}} For example, in architecture, VR can be used to create a walk-through simulation of the inside of a new building; and AR can be used to show a building's structures and systems super-imposed on a real-life view. Another example is through the use of utility applications. Some AR applications, such as Augment, enable users to apply digital objects into real environments, allowing businesses to use augmented reality devices as a way to preview their products in the real world.{{Cite web|url=https://www.pcmag.com/news/augment-is-bringing-the-ar-revolution-to-business|title=Augment Is Bringing the AR Revolution to Business|last1=Marvin|first1=Rob|date=16 August 2016|website=PC Mag|language=en|access-date=2021-02-23}} Similarly, it can also be used to demo what products may look like in an environment for customers, as demonstrated by companies such as Mountain Equipment Co-op or Lowe's who use augmented reality to allow customers to preview what their products might look like at home.{{Cite web|url=https://archpaper.com/2019/08/retail-is-getting-reimagined-with-augmented-reality/|title=Retail is getting reimagined with augmented reality|last=Stamp|first=Jimmy|date=30 August 2019|website=The Architect's Newspaper|url-status=live|archive-url=https://web.archive.org/web/20191115233539/https://archpaper.com/2019/08/retail-is-getting-reimagined-with-augmented-reality/|archive-date=15 November 2019}}
Augmented reality (AR) differs from virtual reality (VR) in the sense that in AR, the surrounding environment is 'real' and AR is just adding virtual objects to the real environment. On the other hand, in VR, the surrounding environment is completely virtual and computer generated. A demonstration of how AR layers objects onto the real world can be seen with augmented reality games. WallaMe is an augmented reality game application that allows users to hide messages in real environments, utilizing geolocation technology in order to enable users to hide messages wherever they may wish in the world.{{Cite web|url=https://www.techradar.com/news/the-future-is-virtual-why-ar-and-vr-will-live-in-the-cloud|title=The future is virtual - why AR and VR will live in the cloud|last=Mahmood 2019-04-12T11:30:27Z|first=Ajmal|website=TechRadar|date=12 April 2019|language=en|access-date=2019-12-12}}
History
File:Virtual-Fixtures-USAF-AR.jpgs – first AR system, U.S. Air Force, Wright-Patterson Air Force Base (1992)]]
- 1901: Author L. Frank Baum, in his science-fiction novel The Master Key, first mentions the idea of an electronic display/spectacles that overlays data onto real life (in this case 'people'). It is named a 'character marker'.Johnson, Joel. [https://web.archive.org/web/20130522153011/http://moteandbeam.net/the-master-key-l-frank-baum-envisions-ar-glasses-in-1901 "The Master Key": L. Frank Baum envisions augmented reality glasses in 1901] Mote & Beam 10 September 2012.
- Heads-up displays (HUDs), a precursor technology to augmented reality, were first developed for pilots in the 1950s, projecting simple flight data into their line of sight, thereby enabling them to keep their "heads up" and not look down at the instruments. It is a transparent display.
- 1968: Ivan Sutherland creates the first head-mounted display that has graphics rendered by a computer.{{cite book |doi=10.1145/1476589.1476686 |chapter=A head-mounted three dimensional display |title=Proceedings of the December 9-11, 1968, fall joint computer conference, part I on - AFIPS '68 (Fall, part I) |pages=757 |year=1968 |last1=Sutherland |first1=Ivan E. |s2cid=4561103 }}
- 1975: Myron Krueger creates Videoplace to allow users to interact with virtual objects.
- 1980: The research by Gavan Lintern of the University of Illinois is the first published work to show the value of a heads up display for teaching real-world flight skills.
- 1980: Steve Mann creates the first wearable computer, a computer vision system with text and graphical overlays on a photographically mediated scene.{{cite news|last=Mann |first=Steve |url=https://techland.time.com/2012/11/02/eye-am-a-camera-surveillance-and-sousveillance-in-the-glassage/ |title=Eye Am a Camera: Surveillance and Sousveillance in the Glassage |publisher=Time |date=2 November 2012 |access-date=14 October 2013}}
- 1986: Within IBM, Ron Feigenblatt describes the most widely experienced form of AR today (viz. "magic window," e.g. smartphone-based Pokémon Go), use of a small, "smart" flat panel display positioned and oriented by hand.{{cite web|url=https://priorart.ip.com/IPCOM/000040923 |title=Absolute Display Window Mouse/Mice |access-date=19 October 2020 |url-status=live |archive-url=https://web.archive.org/web/20191106031325/https://priorart.ip.com/IPCOM/000040923 |archive-date=6 November 2019 |df=dmy }} (context & abstract only) IBM Technical Disclosure Bulletin 1 March 1987
{{cite web|url=https://priorart.ip.com/IPCOM/000040923 |title=Absolute Display Window Mouse/Mice |access-date=19 October 2020 |url-status=live |archive-url=https://web.archive.org/web/20201019143932/https://priorart.ip.com/first-page/IPCOM000040923D |archive-date=19 October 2020 |df=dmy }} (image of anonymous printed article) IBM Technical Disclosure Bulletin 1 March 1987
- 1987: Douglas George and Robert Morris create a working prototype of an astronomical telescope-based "heads-up display" system (a precursor concept to augmented reality) which superimposed in the telescope eyepiece, over the actual sky images, multi-intensity star, and celestial body images, and other relevant information.{{cite journal |title=A computer-driven astronomical telescope guidance and control system with superimposed star field and celestial coordinate graphics display |journal=Journal of the Royal Astronomical Society of Canada |volume=83 |pages=32 |bibcode=1989JRASC..83...32G |last1=George |first1=Douglas B. |last2=Morris |first2=L. Robert |year=1989 }}
- 1990: The term augmented reality is attributed to Thomas P. Caudell, a former Boeing researcher.{{cite journal |last1=Lee |first1=Kangdon |s2cid=40826055 |title=Augmented Reality in Education and Training |journal=TechTrends |date=7 February 2012 |volume=56 |issue=2 |pages=13–21 |doi=10.1007/s11528-012-0559-3 }}
- 1992: Louis Rosenberg developed one of the first functioning AR systems, called Virtual Fixtures, at the United States Air Force Research Laboratory—Armstrong, that demonstrated benefit to human perception.Louis B. Rosenberg. "The Use of Virtual Fixtures As Perceptual Overlays to Enhance Operator Performance in Remote Environments." Technical Report AL-TR-0089, USAF Armstrong Laboratory (AFRL), Wright-Patterson AFB OH, 1992.
- 1992: Steven Feiner, Blair MacIntyre and Doree Seligmann present an early paper on an AR system prototype, KARMA, at the Graphics Interface conference.
- 1993: Mike Abernathy, et al., report the first use of augmented reality in identifying space debris using Rockwell WorldView by overlaying satellite geographic trajectories on live telescope video.
- 1993: A widely cited version of the paper above is published in Communications of the ACM – Special issue on computer augmented environments, edited by Pierre Wellner, Wendy Mackay, and Rich Gold.{{cite journal |last1=Wellner |first1=Pierre |last2=Mackay |first2=Wendy |last3=Gold |first3=Rich |s2cid=21169183 |title=Back to the real world |journal=Communications of the ACM |date=1 July 1993 |volume=36 |issue=7 |pages=24–27 |doi=10.1145/159544.159555 |doi-access=free }}
- 1993: Loral WDL, with sponsorship from STRICOM, performed the first demonstration combining live AR-equipped vehicles and manned simulators. Unpublished paper, J. Barrilleaux, "Experiences and Observations in Applying Augmented Reality to Live Training", 1999.Barrilleaux, Jon. Experiences and Observations in Applying Augmented Reality to Live Training.
- 1995: S. Ravela et al. at University of Massachusetts introduce a vision-based system using monocular cameras to track objects (engine blocks) across views for augmented reality.{{Cite journal|url=https://scholarworks.umass.edu/entities/publication/84c55891-d457-47f6-878b-abe58212ab57|title=Tracking Object Motion Across Aspect Changes for Augmented Reality|first=S.|last=Ravela|date=16 March 1996|via=scholarworks.umass.edu}}{{Cite book|chapter-url=https://ieeexplore.ieee.org/document/525793|chapter=Adaptive tracking and model registration across distinct aspects|first1=S.|last1=Ravela|first2=B.|last2=Draper|first3=J.|last3=Lim|first4=R.|last4=Weiss|title=Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots |date=16 August 1995|volume=1|pages=174–180 vol.1|via=IEEE Xplore|doi=10.1109/IROS.1995.525793|isbn=0-8186-7108-4 |url=https://scholarworks.umass.edu/cs_faculty_pubs/219 }}
- 1996: General Electric develops system for projecting information from 3D CAD models onto real-world instances of those models.{{Cite web|title=US Patent for Projection of images of computer models in three dimensional space Patent (Patent # 5,687,305 issued November 11, 1997) - Justia Patents Search|url=https://patents.justia.com/patent/5687305|access-date=2021-10-17|website=patents.justia.com}}
- 1998: Spatial augmented reality introduced at University of North Carolina at Chapel Hill by Ramesh Raskar, Greg Welch, Henry Fuchs.
- 1999: Frank Delgado, Mike Abernathy et al. report successful flight test of LandForm software video map overlay from a helicopter at Army Yuma Proving Ground overlaying video with runways, taxiways, roads and road names.
- 1999: The US Naval Research Laboratory engages on a decade-long research program called the Battlefield Augmented Reality System (BARS) to prototype some of the early wearable systems for dismounted soldier operating in urban environment for situation awareness and training.{{Cite web|url=https://www.nrl.navy.mil/itd/imda/research/5581/augmented-reality/|title=Information Technology|website=www.nrl.navy.mil}}
- 1999: NASA X-38 flown using LandForm software video map overlays at Dryden Flight Research Center.AviationNow.com Staff, "X-38 Test Features Use of Hybrid Synthetic Vision" AviationNow.com, 11 December 2001
- 2000: Rockwell International Science Center demonstrates tetherless wearable augmented reality systems receiving analog video and 3D audio over radio-frequency wireless channels. The systems incorporate outdoor navigation capabilities, with digital horizon silhouettes from a terrain database overlain in real time on the live outdoor scene, allowing visualization of terrain made invisible by clouds and fog.{{cite book |doi=10.1109/ISAR.2000.880918 |chapter=A wearable augmented reality testbed for navigation and control, built solely with commercial-off-the-shelf (COTS) hardware |title=Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000) |pages=12–19 |year=2000 |last1=Behringer |first1=R. |last2=Tam |first2=C. |last3=McGee |first3=J. |last4=Sundareswaran |first4=S. |last5=Vassiliou |first5=M. |s2cid=18892611 |isbn=0-7695-0846-4 }}{{cite book |doi=10.1109/ISWC.2000.888495 |chapter=Two wearable testbeds for augmented reality: ItWARNS and WIMMIS |title=Digest of Papers. Fourth International Symposium on Wearable Computers |pages=189–190 |year=2000 |last1=Behringer |first1=R. |last2=Tam |first2=C. |last3=McGee |first3=J. |last4=Sundareswaran |first4=S. |last5=Vassiliou |first5=M. |s2cid=13459308 |isbn=0-7695-0795-6 }}
- 2004: An outdoor helmet-mounted AR system was demonstrated by Trimble Navigation and the Human Interface Technology Laboratory (HIT lab).
- 2006: Outland Research develops AR media player that overlays virtual content onto a users view of the real world synchronously with playing music, thereby providing an immersive AR entertainment experience.{{Cite patent|country=|number=7732694|title=United States Patent: 7732694 - Portable music player with synchronized transmissive visual overlays|status=|pubdate=9 Aug 2006|gdate=8 June 2010|invent1=|inventor1-first=|url=http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=/netahtml/PTO/search-adv.htm&r=1&f=G&l=50&d=PALL&S1=07732694&OS=PN/07732694&RS=PN/07732694}}{{Cite web|last=Slawski|first=Bill|date=2011-09-04|title=Google Picks Up Hardware and Media Patents from Outland Research|url=https://www.seobythesea.com/2011/09/google-picks-up-hardware-and-media-patents-from-outland-research/|website=SEO by the Sea ⚓|language=en-US}}
- 2008: Wikitude AR Travel Guide launches on 20 Oct 2008 with the G1 Android phone.[https://www.youtube.com/watch?v=8EA8xlicmT8 Wikitude AR Travel Guide]. YouTube.com. Retrieved 9 June 2012.
- 2009: ARToolkit was ported to Adobe Flash (FLARToolkit) by Saqoosha, bringing augmented reality to the web browser.Cameron, Chris. [http://www.readwriteweb.com/archives/flash-based_ar_gets_high-quality_markerless_upgrade.php Flash-based AR Gets High-Quality Markerless Upgrade], ReadWriteWeb 9 July 2010.
- 2012: Launch of Lyteshot, an interactive AR gaming platform that utilizes smart glasses for game data
- 2013: Niantic releases "Ingress", an augmented reality mobile game for iOS and Android operating systems (and a predecessor of Pokémon Go).
- 2015: Microsoft announced the HoloLens augmented reality headset, which uses various sensors and a processing unit to display virtual imagery over the real world.Microsoft Channel, YouTube [https://www.youtube.com/watch?v=aThCr0PsyuA], 23 January 2015.
- 2016: Niantic released Pokémon Go for iOS and Android in July 2016. The game quickly became one of the most popular smartphone applications and in turn spikes the popularity of augmented reality games.{{cite news|last1=Bond|first1=Sarah|title=After the Success of Pokémon Go, How Will Augmented Reality Impact Archaeological Sites?|url=https://www.forbes.com/sites/drsarahbond/2016/07/17/after-the-success-of-pokemon-go-how-will-augmented-reality-impact-archaeological-sites/|access-date=17 July 2016|date=17 July 2016}}
- 2018: Magic Leap launched the Magic Leap One augmented reality headset.{{cite web | last=Haselton | first=Todd | title=After almost a decade and billions in outside investment, Magic Leap's first product is finally on sale for $2,295. Here's what it's like. | website=CNBC | date=2018-08-08 | url=https://www.cnbc.com/2018/08/08/magic-leap-one-creators-edition-first-look.html | access-date=2024-06-02}} Leap Motion announced the Project North Star augmented reality headset, and later released it under an open source license.{{cite web |title=Leap Motion's 'Project North Star' could help make cheap AR headsets a reality |website=Mashable |date=9 April 2018 |url=https://mashable.com/article/leap-motion-project-north-star-ar-headset |access-date=26 March 2024}}{{cite web |title=Leap Motion designed a $100 augmented reality headset with super-powerful hand tracking |url=https://www.theverge.com/2018/4/9/17208192/leap-motion-project-north-star-augmented-reality-headset-open-source-concept |website=The Verge |date=9 April 2018 |access-date=26 March 2024}}{{cite web |title=Project North Star is Now Open Source |url=https://blog.leapmotion.com/north-star-open-source/ |website=Leap Motion |date=6 June 2018 |access-date=26 March 2024}}{{cite web |title=Leap Motion Open-sources Project North Star, An AR Headset Prototype With Impressive Specs |url=https://www.roadtovr.com/leap-motion-reveals-project-north-star-an-open-source-wide-fov-ar-headset-dev-kit/ |website=Road to VR |date=6 June 2018 |access-date=26 March 2024}}
- 2019: Microsoft announced HoloLens 2 with significant improvements in terms of field of view and ergonomics.Official Blog, Microsoft [https://blogs.microsoft.com/blog/2019/02/24/microsoft-at-mwc-barcelona-introducing-microsoft-hololens-2/], 24 February 2019.
- 2022: Magic Leap launched the Magic Leap 2 headset.{{cite web |title=Magic Leap 2 is the best AR headset yet, but will an enterprise focus save the company? |url=https://www.engadget.com/magic-leap-2-ar-headset-tech-dive-143046676.html |website=Engadget |date=11 November 2022 |access-date=26 March 2024}}
- 2024: Meta Platforms revealed the Orion AR glasses prototype.{{Cite web |last=Vanian |first=Jonathan |date=2024-09-27 |title=Hands-on with Meta's Orion AR glasses prototype and the possible future of computing |url=https://www.cnbc.com/2024/09/27/hands-on-with-metas-orion-augmented-reality-smart-glasses-prototype.html |access-date=2024-09-28 |website=CNBC |language=en}}
Displays
File:MicrosoftHoloLensBloomGesture.JPG
Augmented reality requires a head-mounted display or a handheld device, which includes a processor, display, sensors, and one or more input devices. Modern mobile computing devices like smartphones and tablet computers contain these elements, which often include a camera and microelectromechanical systems (MEMS) sensors such as an accelerometer, GPS, and solid state compass.{{Cite web |url=http://www.technologyreview.com/news/428654/augmented-reality-is-finally-getting-real/ |title=Augmented Reality Is Finally Getting Real |last=Metz |first=Rachael |date=2 August 2012 |website=technologyreview.com |access-date=18 June 2019}}{{cite journal|title=Benchmarking Built-In Tracking Systems for Indoor AR Applications on Popular Mobile Devices|journal= Sensors|date=2022|doi= 10.3390/s22145382|doi-access= free|last1= Marino|first1= Emanuele|last2= Bruno|first2= Fabio|last3= Barbieri|first3= Loris|last4= Lagudi|first4= Antonio|volume= 22|issue= 14|page= 5382|pmid= 35891058|pmc= 9320911|bibcode= 2022Senso..22.5382M}}
Various technologies can be used to display augmented reality, including optical see-through head mounted displays, monitors, and handheld devices. Two of the display technologies used in augmented reality are diffractive waveguides and reflective waveguides.
=Head-mounted displays=
A head-mounted display (HMD) is a display device worn on the forehead, such as a harness or helmet-mounted. HMDs place images of virtual objects over the user's field of view. Augmented reality HMDs are either optical see-through or video passthrough.{{cite web | last=Boland | first=Mike | title=Passthrough vs. Seethrough AR: The Answer is 'Both' | website=AR Insider | date=2024-11-07 | url=https://arinsider.co/2024/11/07/passthrough-vs-seethrough-ar-the-answer-is-both/ | access-date=2025-04-21}}{{cite conference | last=Rolland | first=Jannick P. | last2=Holloway | first2=Richard L. | last3=Fuchs | first3=Henry | title=Comparison of optical and video see-through, head-mounted displays | date=1995-12-21 | doi=10.1117/12.197322 | url=https://www.spiedigitallibrary.org/redirect/proceedings/proceeding?articleid=981546 | access-date=2025-04-21 | page=293–307}} Modern HMDs often employ sensors for six degrees of freedom monitoring that allow the system to align virtual information to the physical world and adjust accordingly with the user's head movements.{{Cite web |url=http://www.eweek.com/c/a/Security/Fleet-Week-Office-of-Naval-Research-Technology/4/ |title=Fleet Week: Office of Naval Research Technology |date=28 May 2012 |website=eweek.com |access-date=18 June 2019}}Rolland, Jannick; Baillott, Yohan; Goon, Alexei.[https://web.archive.org/web/20200227120212/https://pdfs.semanticscholar.org/ce53/48128f94f3383bdc4eb15fb4eaf3721d521f.pdf A Survey of Tracking Technology for Virtual Environments], Center for Research and Education in Optics and Lasers, University of Central Florida.{{Cite web |url=http://campar.in.tum.de/twiki/pub/Chair/TeachingSs07ArProseminar/1_Display-Systems_Klepper_Report.pdf |title=Augmented Reality - Display Systems |last=Klepper |first=Sebastian |website=campar.in.tum.de |archive-url=https://web.archive.org/web/20130128175343/http://campar.in.tum.de/twiki/pub/Chair/TeachingSs07ArProseminar/1_Display-Systems_Klepper_Report.pdf |archive-date=28 January 2013 |url-status=dead |access-date=18 June 2019}} When using AR technology, the HMDs only require relatively small displays. In this situation, liquid crystal on silicon (LCOS) and micro-OLED (organic light-emitting diodes) are commonly used.{{Cite journal |last=Komura |first=Shinichi |date=2024-07-19 |title=Optics of AR/VR using liquid crystals |journal=Molecular Crystals and Liquid Crystals |volume=768 |issue=17 |language=en |pages=1014–1039 |doi=10.1080/15421406.2024.2379694 |issn=1542-1406|doi-access=free |bibcode=2024MCLC..768.1014K }} HMDs can provide VR users with mobile and collaborative experiences.{{cite journal |last1=Rolland |first1=Jannick P. |last2=Biocca |first2=Frank |last3=Hamza-Lup |first3=Felix |last4=Ha |first4=Yanggang |last5=Martins |first5=Ricardo |title=Development of Head-Mounted Projection Displays for Distributed, Collaborative, Augmented Reality Applications |journal=Presence: Teleoperators and Virtual Environments |date=October 2005 |volume=14 |issue=5 |pages=528–549 |doi=10.1162/105474605774918741 |s2cid=5328957 |url=https://stars.library.ucf.edu/facultybib2000/5607 |arxiv=1902.07769 }} Specific providers, such as uSens and Gestigon, include gesture controls for full virtual immersion.{{cite web|title=Gestigon Gesture Tracking – TechCrunch Disrupt|url=https://techcrunch.com/video/gestigon-gesture-tracking/517762030/|website=TechCrunch|access-date=11 October 2016}}{{cite web|last1=Matney|first1=Lucas|title=uSens shows off new tracking sensors that aim to deliver richer experiences for mobile VR|url=https://techcrunch.com/2016/08/29/usens-unveils-vr-sensor-modules-with-hand-tracking-and-mobile-positional-tracking-tech-baked-in/|website=TechCrunch|date=29 August 2016 |access-date=29 August 2016}}
AR headsets typically have a field of view of about 30 to 50 degrees per eye.{{cite web | last=Crisostomo | first=Christian | title=Introduction To AR Headsets Technology: The Field Of View | website=ARPost | date=2018-09-27 | url=https://arpost.co/2018/09/27/introduction-to-ar-headsets-technology-the-field-of-view/ | access-date=2025-04-21}}
Vuzix is a company that has produced a number of head-worn optical see through displays marketed for augmented reality.{{cite web |title=Images Of The Vuzix STAR 1200 Augmented Reality Glasses |url=https://techcrunch.com/2011/06/04/images-of-the-vuzix-star-1200-augmented-reality-glasses/ |website=TechCrunch |date=5 June 2011 |access-date=26 March 2024}}{{cite web |title=Vuzix Blade AR glasses are the next-gen Google Glass we've all been waiting for |date=9 January 2018 |url=https://www.theverge.com/2018/1/9/16869174/vuzix-blade-ar-glasses-augmented-reality-amazon-alexa-ai-ces-2018 |access-date=26 March 2024}}{{cite web |title=Hands On: Vuzix's No-Nonsense AR Smart Glasses |date=28 February 2017 |url=https://www.pcmag.com/news/hands-on-vuzixs-no-nonsense-ar-smart-glasses |access-date=26 March 2024}}
Near-eye augmented reality devices can be used as portable head-up displays as they can show data, information, and images while the user views the real world. This is basically what a head-up display does; however, practically speaking, augmented reality is expected to include registration and tracking between the superimposed perceptions, sensations, information, data, and images and some portion of the real world.{{Cite web|title = What is Augmented Reality (AR): Augmented Reality Defined, iPhone Augmented Reality Apps and Games and More|url = http://www.digitaltrends.com/features/what-is-augmented-reality-iphone-apps-games-flash-yelp-android-ar-software-and-more/|website = Digital Trends|access-date = 8 October 2015|date = 3 November 2009}}
=AR glasses=
{{Update section|date=August 2024}}
AR displays can be rendered on devices resembling eyeglasses. Versions include eyewear that employs cameras to intercept the real world view and re-display its augmented view through the eyepiecesGrifatini, Kristina. [http://www.technologyreview.com/news/421606/augmented-reality-goggles/ Augmented Reality Goggles], Technology Review 10 November 2010. and devices in which the AR imagery is projected through or reflected off the surfaces of the eyewear lens pieces.Arthur, Charles. [https://www.theguardian.com/technology/2012/sep/10/augmented-reality-glasses-google-project UK company's 'augmented reality' glasses could be better than Google's], The Guardian, 10 September 2012.Gannes, Liz. {{cite web |url=http://allthingsd.com/20120404/google-unveils-project-glass-wearable-augmented-reality-glasses/ |title=Google Unveils Project Glass: Wearable Augmented-Reality Glasses |work=allthingsd.com |access-date=4 April 2012}}, All Things D.Benedetti, Winda. [https://web.archive.org/web/20120823000655/https://www.nbcnews.com/technology/ingame/xbox-leak-reveals-kinect-2-augmented-reality-glasses-833583 Xbox leak reveals Kinect 2, augmented reality glasses] NBC News. Retrieved 23 August 2012.
The EyeTap (also known as Generation-2 Glass[https://web.archive.org/web/20131004212812/http://wearcam.org/glass.pdf "GlassEyes": The Theory of EyeTap Digital Eye Glass, supplemental material for IEEE Technology and Society, Volume Vol. 31, Number 3, 2012, pp. 10–14].) captures rays of light that would otherwise pass through the center of the lens of the wearer's eye, and substitutes synthetic computer-controlled light for each ray of real light.
An example of an AR glasses product is the Snap Spectacles.{{cite web | last=Stein | first=Scott | title=I Tried Snap's New Standalone AR Glasses, Which Do Mixed Reality, AI and Work Outdoors | website=CNET | date=2024-09-17 | url=https://www.cnet.com/tech/computing/i-tried-snaps-new-standalone-ar-glasses-which-do-mixed-reality-ai-and-work-outdoors/ | access-date=2025-04-21}}
=Handheld=
File:Wikitude.jpg and a solid state compass]]
A handheld display employs a small display that fits in a user's hand. All handheld AR solutions to date opt for video passthrough. Initially handheld AR employed fiducial markers,[http://researchguides.dartmouth.edu/content.php?pid=227212&sid=1891183 Marker vs Markerless AR] {{webarchive|url=https://web.archive.org/web/20130128175349/http://researchguides.dartmouth.edu/content.php?pid=227212&sid=1891183 |date=28 January 2013 }}, Dartmouth College Library. and later GPS units and MEMS sensors such as digital compasses and six degrees of freedom accelerometer–gyroscope. Today simultaneous localization and mapping (SLAM) markerless trackers such as PTAM (parallel tracking and mapping) are starting to come into use. Handheld display AR promises to be the first commercial success for AR technologies. The two main advantages of handheld AR are the portable nature of handheld devices and the ubiquitous nature of camera phones. The disadvantages are the physical constraints of the user having to hold the handheld device out in front of them at all times, as well as the distorting effect of classically wide-angled mobile phone cameras when compared to the real world as viewed through the eye.{{cite web |last=Feiner |first=Steve |title=Augmented reality: a long way off? |url=http://www.pocket-lint.com/news/38869/augmented-reality-interview-steve-feiner |work=AR Week |publisher=Pocket-lint |access-date=3 March 2011|date=3 March 2011}}
=Contact lenses=
Contact lenses that display AR imaging are in development. These bionic contact lenses might contain the elements for display embedded into the lens including integrated circuitry, LEDs and an antenna for wireless communication.
The first contact lens display was patented in 1999 by Steve Mann and was intended to work in combination with AR spectacles, but the project was abandoned,{{Cite web|title=Full Page Reload|url=https://spectrum.ieee.org/profile-innovega|website=IEEE Spectrum: Technology, Engineering, and Science News|date=10 April 2013|language=en|access-date=2020-05-06}}{{Cite web|url=https://patents.google.com/patent/CA2280022/en|title=Contact lens for the display of information such as text, graphics, or pictures}} then 11 years later in 2010–2011.Greenemeier, Larry. [http://blogs.scientificamerican.com/observations/2011/11/23/computerized-contact-lenses-could-enable-in-eye-augmented-reality/ Computerized Contact Lenses Could Enable In-Eye Augmented Reality]. Scientific American, 23 November 2011.Yoneda, Yuka. [http://inhabitat.com/solar-powered-augmented-contact-lenses-cover-your-eye-with-100s-of-leds/ Solar Powered Augmented Contact Lenses Cover Your Eye with 100s of LEDs]. inhabitat, 17 March 2010.{{cite web |last=Rosen |first=Kenneth |title=Contact Lenses Can Display Your Text Messages |url=http://mashable.com/2012/12/08/contact-lenses-text-messages/|work=Mashable.com |date=8 December 2012 |access-date=13 December 2012}}{{cite news|last=O'Neil |first=Lauren |title=LCD contact lenses could display text messages in your eye |url=http://www.cbc.ca/news/yourcommunity/2012/12/lcd-contact-lenses-could-display-text-messages-in-your-eye.html |publisher=CBC News |access-date=12 December 2012 |url-status=dead |archive-url=https://web.archive.org/web/20121211075000/http://www.cbc.ca/news/yourcommunity/2012/12/lcd-contact-lenses-could-display-text-messages-in-your-eye.html |archive-date=11 December 2012 }} Another version of contact lenses, in development for the U.S. military, is designed to function with AR spectacles, allowing soldiers to focus on close-to-the-eye AR images on the spectacles and distant real world objects at the same time.Anthony, Sebastian. [http://www.extremetech.com/computing/126043-us-military-developing-multi-focus-augmented-reality-contact-lenses US military developing multi-focus augmented reality contact lenses]. ExtremeTech, 13 April 2012.Bernstein, Joseph. [http://www.popsci.com/diy/article/2012-05/2012-invention-awards-augmented-reality-contact-lenses 2012 Invention Awards: Augmented-Reality Contact Lenses] Popular Science, 5 June 2012.
At CES 2013, a company called Innovega also unveiled similar contact lenses that required being combined with AR glasses to work.{{Cite web|title=Innovega combines glasses and contact lenses for an unusual take on augmented reality|url=https://www.theverge.com/2013/1/10/3863550/innovega-augmented-reality-glasses-contacts-hands-on|last=Robertson|first=Adi|date=2013-01-10|website=The Verge|language=en|access-date=2020-05-06}}
The first publicly unveiled working prototype of an AR contact lens not requiring the use of glasses in conjunction was developed by Mojo Vision and announced and shown off at CES 2020.{{Cite web|title=Full Page Reload|url=https://spectrum.ieee.org/ar-in-a-contact-lens-its-the-real-deal|website=IEEE Spectrum: Technology, Engineering, and Science News|date=16 January 2020|language=en|access-date=2020-05-06}}{{Cite web|title=Mojo Vision's AR contact lenses are very cool, but many questions remain|url=https://techcrunch.com/2020/01/16/mojo-visions-ar-contact-lenses-are-very-cool-but-many-questions-remain/|website=TechCrunch|date=16 January 2020 |language=en-US|access-date=2020-05-06}}{{Cite web|title=Mojo Vision is developing AR contact lenses|url=https://techcrunch.com/video/mojo-vision-is-developing-ar-contact-lenses/|website=TechCrunch|date=16 January 2020 |language=en-US|access-date=2020-05-06}}
=Virtual retinal display=
A virtual retinal display (VRD) is a type of theoretical display device that was in development at the University of Washington's Human Interface Technology Laboratory under Dr. Thomas A. Furness III.{{Cite journal|last1=Viirre|first1=E.|last2=Pryor|first2=H.|last3=Nagata|first3=S.|last4=Furness|first4=T. A.|date=1998|title=The virtual retinal display: a new technology for virtual reality and augmented vision in medicine|journal=Studies in Health Technology and Informatics|volume=50|issue=Medicine Meets virtual reality|pages=252–257|issn=0926-9630|pmid=10180549|doi=10.3233/978-1-60750-894-6-252}} With this technology, a display is scanned directly onto the retina of a viewer's eye. This results in bright images with high resolution and high contrast. The viewer sees what appears to be a conventional display floating in space.Tidwell, Michael; Johnson, Richard S.; Melville, David; Furness, Thomas A.[http://www.hitl.washington.edu/publications/p-95-1/ The Virtual Retinal Display – A Retinal Scanning Imaging System] {{webarchive|url=https://web.archive.org/web/20101213134809/http://www.hitl.washington.edu/publications/p-95-1/ |date=13 December 2010 }}, Human Interface Technology Laboratory, University of Washington.
Several of tests were done to analyze the safety of the VRD. In one test, patients with partial loss of vision—having either macular degeneration (a disease that degenerates the retina) or keratoconus—were selected to view images using the technology. In the macular degeneration group, five out of eight subjects preferred the [https://immersivestudio.co.uk/services/virtual-reality-services/ VRD] images to the cathode-ray tube (CRT) or paper images and thought they were better and brighter and were able to see equal or better resolution levels. The Keratoconus patients could all resolve smaller lines in several line tests using the VRD as opposed to their own correction. They also found the VRD images to be easier to view and sharper. As a result of these several tests, virtual retinal display is considered safe technology.
Virtual retinal display creates images that can be seen in ambient daylight and ambient room light. The VRD is considered a preferred candidate to use in a surgical display due to its combination of high resolution and high contrast and brightness. Additional tests show high potential for VRD to be used as a display technology for patients that have low vision.
=Projection mapping=
Projection mapping augments real-world objects and scenes without the use of special displays such as monitors, head-mounted displays or hand-held devices. Projection mapping makes use of digital projectors to display graphical information onto physical objects. The key difference in projection mapping is that the display is separated from the users of the system. Since the displays are not associated with each user, projection mapping scales naturally up to groups of users, allowing for collocated collaboration between users.
Examples include shader lamps, mobile projectors, virtual tables, and smart projectors. Shader lamps mimic and augment reality by projecting imagery onto neutral objects. This provides the opportunity to enhance the object's appearance with materials of a simple unit—a projector, camera, and sensor.
Other applications include table and wall projections. Virtual showcases, which employ beam splitter mirrors together with multiple graphics displays, provide an interactive means of simultaneously engaging with the virtual and the real.
A projection mapping system can display on any number of surfaces in an indoor setting at once. Projection mapping supports both a graphical visualization and passive haptic sensation for the end users. Users are able to touch physical objects in a process that provides passive haptic sensation.Ramesh Raskar, Greg Welch, Henry Fuchs [https://web.archive.org/web/19981205111134/http://www.cs.unc.edu/~raskar/Office/ Spatially Augmented Reality], First International Workshop on Augmented Reality, Sept 1998.Knight, Will. [https://www.newscientist.com/article/dn7695 Augmented reality brings maps to life] 19 July 2005.Sung, Dan. [http://www.pocket-lint.com/news/38802/augmented-reality-maintenance-and-repair Augmented reality in action – maintenance and repair]. Pocket-lint, 1 March 2011.
3D tracking
{{Main|Positional tracking}}
3D tracking is an integral part of augmented reality, as it allows a headset and controllers to be tracked in the user's environment. Tracking is often camera-based, which uses cameras on the device.
Mobile augmented-reality systems use one or more of the following motion tracking technologies: digital cameras and/or other optical sensors, accelerometers, GPS, gyroscopes, solid state compasses, radio-frequency identification (RFID). These technologies offer varying levels of accuracy and precision. These technologies are implemented in the ARKit API by Apple and ARCore API by Google to allow tracking for their respective mobile device platforms.
CMOS camera sensors are widely used for camera-based tracking in AR technology.{{cite book |last1=Schmalstieg |first1=Dieter |last2=Hollerer |first2=Tobias |title=Augmented Reality: Principles and Practice |date=2016 |publisher=Addison-Wesley Professional |isbn=978-0-13-315320-0 |pages=209–10 |url=https://books.google.com/books?id=qPU2DAAAQBAJ&pg=PT209}}
=Camera-based tracking=
File:comparison_of_augmented_reality_fiducial_markers.svg
Augmented reality systems must realistically integrate virtual imagery into the real world. The software must derive real world coordinates, independent of camera, and camera images. That process is called image registration, and uses different methods of computer vision, mostly related to video tracking.Maida, James; Bowen, Charles; Montpool, Andrew; Pace, John. [http://research.jsc.nasa.gov/PDF/SLiSci-14.pdf Dynamic registration correction in augmented-reality systems] {{webarchive|url=https://web.archive.org/web/20130518032710/http://research.jsc.nasa.gov/PDF/SLiSci-14.pdf |date=18 May 2013 }}, Space Life Sciences, NASA. Many computer vision methods of augmented reality are inherited from visual odometry.
Usually those methods consist of two parts. The first stage is to detect interest points, fiducial markers or optical flow in the camera images. This step can use feature detection methods like corner detection, blob detection, edge detection or thresholding, and other image processing methods.State, Andrei; Hirota, Gentaro; Chen, David T; Garrett, William; Livingston, Mark. [http://www.cs.princeton.edu/courses/archive/fall01/cs597d/papers/state96.pdf Superior Augmented Reality Registration by Integrating Landmark Tracking and Magnetic Tracking], Department of Computer Science, University of North Carolina at Chapel Hill.Bajura, Michael; Neumann, Ulrich. [http://graphics.usc.edu/cgit/publications/papers/DynamicRegistrationVRAIS95.pdf Dynamic Registration Correction in Augmented-Reality Systems] [https://web.archive.org/web/20120713224616/https://graphics.usc.edu/cgit/publications/papers/DynamicRegistrationVRAIS95.pdf Archived] 13 July 2012, University of North Carolina, University of Southern California. The second stage restores a real world coordinate system from the data obtained in the first stage.
Some methods assume objects with known geometry (or fiducial markers) are present in the scene. In some of those cases the scene 3D structure should be calculated beforehand. If part of the scene is unknown, simultaneous localization and mapping (SLAM) can map relative positions. If no information about scene geometry is available, structure from motion methods like bundle adjustment are used. Mathematical methods used in the second stage include: projective (epipolar) geometry, kalman and particle filters, nonlinear optimization, robust statistics.{{citation needed|date=February 2017}}
There are two methods of camera-based tracking: marker-based tracking and markerless tracking.{{Cite journal |url=https://codegres.com/augmented-reality/ |title=What is Augmented Reality |last=Hegde |first=Naveen |date=19 March 2023 | journal=Codegres |access-date=19 March 2023}} Marker-based tracking uses fiducial markers, whereas markerless tracking stores a representation of the real world using visual-inertial odometry (VIO) or simultaneous localization and mapping (SLAM). A piece of paper with some distinct geometries can be used for marker-based tracking. The camera recognizes the geometries by identifying specific points in the drawing. Markerless tracking, also called instant tracking, does not use markers. It uses sensors in mobile devices to accurately detect the real-world environment, such as the locations of walls and points of intersection.{{Cite news|url=https://www.marxentlabs.com/what-is-markerless-augmented-reality-dead-reckoning/|title=Markerless Augmented Reality is here.|date=9 May 2014|work=Marxent {{!}} Top Augmented Reality Apps Developer|access-date=23 January 2018|language=en-US}}
Input devices
Techniques include gesture recognition systems that interpret a user's body movements by visual detection or from sensors embedded in a peripheral device such as a wand, stylus, pointer, glove or other body wear.Marshall, Gary.[http://www.techradar.com/news/computing/beyond-the-mouse-how-input-is-evolving-626794?artc_pg=1 Beyond the mouse: how input is evolving, Touch, voice and gesture recognition and augmented reality]TechRadar.computing\PC Plus 23 August 2009.Simonite, Tom. [http://www.technologyreview.com/news/425431/augmented-reality-meets-gesture-recognition/ Augmented Reality Meets Gesture Recognition], Technology Review, 15 September 2011.Chaves, Thiago; Figueiredo, Lucas; Da Gama, Alana; de Araujo, Christiano; Teichrieb, Veronica. [http://dl.acm.org/citation.cfm?id=2377147 Human Body Motion and Gestures Recognition Based on Checkpoints]. SVR '12 Proceedings of the 2012 14th Symposium on Virtual and Augmented Reality pp. 271–278.Barrie, Peter; Komninos, Andreas; Mandrychenko, Oleksii.[http://www.buccleuchpark.net/MUCOM/publi/acmMobility09.pdf A Pervasive Gesture-Driven Augmented Reality Prototype using Wireless Sensor Body Area Networks]. Products which are trying to serve as a controller of AR headsets include Wave by Seebright Inc. and Nimble by Intugine Technologies.
Processing
Computers are responsible for graphics and processing 3D tracking data for augmented reality. For camera-based 3D tracking methods, a computer analyzes the sensed visual and other data to synthesize and position virtual objects. With the improvement of technology and computers, augmented reality is going to lead to a drastic change on ones perspective of the real world.{{Cite web|url=https://computer.howstuffworks.com/augmented-reality.htm|title=How Augmented Reality Works|last=Bosnor|first=Kevin|website=howstuffworks|date=19 February 2001}}
Computers are improving at a very fast rate, leading to new ways to improve other technology. Computers are the core of augmented reality.{{Cite web|date=6 April 1999|first1=Jeffrey |last1=Meisner |first2=Walter P. |last2=Donnelly |first3=Richard |last3=Roosen |title=Augmented reality technology|url=https://patents.google.com/patent/US6625299B1/en}} The computer receives data from the sensors which determine the relative position of an objects' surface. This translates to an input to the computer which then outputs to the users by adding something that would otherwise not be there. The computer comprises memory and a processor.{{Cite book|title=A Survey of Augmented Reality Technologies, Applications and Limitations|last=Krevelen, Poelman|first=D.W.F, Ronald|publisher=International Journal of virtual reality|year=2010|pages=3, 6}} The computer takes the scanned environment then generates images or a video and puts it on the receiver for the observer to see. The fixed marks on an object's surface are stored in the memory of a computer. The computer also withdraws from its memory to present images realistically to the onlooker.
Software
Augmented Reality Markup Language (ARML) is a data standard developed within the Open Geospatial Consortium (OGC),{{cite web | title = ARML 2.0 SWG | work = Open Geospatial Consortium website | publisher = Open Geospatial Consortium | url = http://www.opengeospatial.org/projects/groups/arml2.0swg | access-date = 12 November 2013 | archive-date = 12 November 2013 | archive-url = https://web.archive.org/web/20131112013312/http://www.opengeospatial.org/projects/groups/arml2.0swg | url-status = dead }} which consists of Extensible Markup Language (XML) grammar to describe the location and appearance of virtual objects in the scene, as well as ECMAScript bindings to allow dynamic access to properties of virtual objects.
{{anchor|Spark_AR}}
To enable rapid development of augmented reality applications, software development applications have emerged, including Lens Studio from Snapchat and Spark AR from Facebook. Augmented reality Software Development Kits (SDKs) have been launched by Apple and Google: Apple's ARKit and Google's ARCore.{{cite web|url=http://augmentedrealitynews.org/ar-sdk/top-5-augmented-reality-sdks/|title=Top 5 AR SDKs|publisher=Augmented Reality News|url-status=dead|archive-url=https://web.archive.org/web/20131213111219/http://augmentedrealitynews.org/ar-sdk/top-5-augmented-reality-sdks/|archive-date=13 December 2013|access-date=15 November 2013}}{{cite web|url = http://augmentedworldexpo.com/news/tutorial-top-10-mobile-augmented-reality-sdks-for-developers//|title = Top 10 AR SDKs|publisher = Augmented World Expo|access-date = 15 November 2013|archive-url = https://web.archive.org/web/20131123011106/http://augmentedworldexpo.com/news/tutorial-top-10-mobile-augmented-reality-sdks-for-developers/|archive-date = 23 November 2013|url-status = dead|df = dmy-all}}
According to a 2017 Time article, in about 15 to 20 years it is predicted that augmented reality and virtual reality are going to become the primary method for computer interactions.{{Cite magazine|url=https://time.com/4654944/this-technology-could-replace-the-keyboard-and-mouse/|title=This Technology Could Replace the Keyboard and Mouse|last=Bajarin|first=Tim|magazine=Time|date=31 January 2017 |access-date=19 June 2019}}
=Rendering=
Software that renders onto displays can create a realistic view by using occlusion, which hides parts of virtual objects behind parts of the real world. Having accurate occlusion creates a much more realistic view of virtual objects integrated into the real world.{{cite web | last=Mathew | first=Neil | title=Why is Occlusion in Augmented Reality So Hard? | website=HackerNoon | date=2018-01-28 | url=https://hackernoon.com/why-is-occlusion-in-augmented-reality-so-hard-7bc8041607f9 | access-date=2025-04-21}}
=Design=
AR systems rely heavily on the immersion of the user. The following lists some considerations for designing augmented reality applications:
==Environmental/context design==
Context Design focuses on the end-user's physical surrounding, spatial space, and accessibility that may play a role when using the AR system. Designers should be aware of the possible physical scenarios the end-user may be in such as:
- Public, in which the users use their whole body to interact with the software
- Personal, in which the user uses a smartphone in a public space
- Intimate, in which the user is sitting with a desktop and is not really moving
- Private, in which the user has on a wearable.{{Cite web|url=https://uxdesign.cc/the-principles-of-good-user-experience-design-for-augmented-reality-d8e22777aabd|title="The Principles of Good UX for Augmented Reality – UX Collective." UX Collective|last=Wilson|first=Tyler|date=30 January 2018|access-date=19 June 2019}}
By evaluating each physical scenario, potential safety hazards can be avoided and changes can be made to greater improve the end-user's immersion. UX designers will have to define user journeys for the relevant physical scenarios and define how the interface reacts to each.
Another aspect of context design involves the design of the system's functionality and its ability to accommodate user preferences.{{Cite web|url=https://blog.google/products/google-vr/best-practices-mobile-ar-design/|title=Best Practices for Mobile AR Design- Google|date=13 December 2017|website=blog.google}}{{Cite web|url=http://www.eislab.fim.uni-passau.de/files/publications/2014/TR2014-HCIwithAR_1.pdf|title=Human Computer Interaction with Augmented Reality|website=eislab.fim.uni-passau.de|archive-url=https://web.archive.org/web/20180525000513/http://www.eislab.fim.uni-passau.de/files/publications/2014/TR2014-HCIwithAR_1.pdf|archive-date=25 May 2018|url-status=dead|df=dmy-all}} While accessibility tools are common in basic application design, some consideration should be made when designing time-limited prompts (to prevent unintentional operations), audio cues and overall engagement time. In some situations, the application's functionality may hinder the user's ability. For example, applications that is used for driving should reduce the amount of user interaction and use audio cues instead.
==Interaction design==
Interaction design in augmented reality technology centers on the user's engagement with the end product to improve the overall user experience and enjoyment. The purpose of interaction design is to avoid alienating or confusing the user by organizing the information presented. Since user interaction relies on the user's input, designers must make system controls easier to understand and accessible. A common technique to improve usability for augmented reality applications is by discovering the frequently accessed areas in the device's touch display and design the application to match those areas of control.{{Cite web|url=https://theblog.adobe.com/basic-patterns-of-mobile-navigation/|title=Basic Patterns of Mobile Navigation|date=9 May 2017|website=theblog.adobe.com|access-date=12 April 2018|archive-date=13 April 2018|archive-url=https://web.archive.org/web/20180413044751/https://theblog.adobe.com/basic-patterns-of-mobile-navigation/|url-status=dead}} It is also important to structure the user journey maps and the flow of information presented which reduce the system's overall cognitive load and greatly improves the learning curve of the application.{{Cite web|url=https://www.thinkwithgoogle.com/marketing-resources/experience-design/principles-of-mobile-app-design-engage-users-and-drive-conversions/|title=Principles of Mobile App Design: Engage Users and Drive Conversions|website=thinkwithgoogle.com|archive-url=https://web.archive.org/web/20180413185621/https://www.thinkwithgoogle.com/marketing-resources/experience-design/principles-of-mobile-app-design-engage-users-and-drive-conversions/|archive-date=13 April 2018|url-status=dead}}
In interaction design, it is important for developers to utilize augmented reality technology that complement the system's function or purpose.{{Cite web|url=https://www.uxmatters.com/mt/archives/2009/08/inside-out-interaction-design-for-augmented-reality.php|title=Inside Out: Interaction Design for Augmented Reality-UXmatters|website=uxmatters.com}} For instance, the utilization of exciting AR filters and the design of the unique sharing platform in Snapchat enables users to augment their in-app social interactions. In other applications that require users to understand the focus and intent, designers can employ a reticle or raycast from the device.
==Visual design==
In some augmented reality applications that use a 2D device as an interactive surface, the 2D control environment does not translate well in 3D space, which can make users hesitant to explore their surroundings. To solve this issue, designers should apply visual cues to assist and encourage users to explore their surroundings.
It is important to note the two main objects in AR when developing VR applications: 3D volumetric objects that are manipulated and realistically interact with light and shadow; and animated media imagery such as images and videos which are mostly traditional 2D media rendered in a new context for augmented reality. When virtual objects are projected onto a real environment, it is challenging for augmented reality application designers to ensure a perfectly seamless integration relative to the real-world environment, especially with 2D objects. As such, designers can add weight to objects, use depths maps, and choose different material properties that highlight the object's presence in the real world. Another visual design that can be applied is using different lighting techniques or casting shadows to improve overall depth judgment. For instance, a common lighting technique is simply placing a light source overhead at the 12 o’clock position, to create shadows on virtual objects.
Uses
Augmented reality has been explored for many uses, including gaming, medicine, and entertainment. It has also been explored for education and business.{{Cite journal|last1=Moro|first1=Christian|last2=Štromberga|first2=Zane|last3=Raikos|first3=Athanasios|last4=Stirling|first4=Allan|date=2017|title=The effectiveness of virtual and augmented reality in health sciences and medical anatomy|url=https://pubmed.ncbi.nlm.nih.gov/28419750|journal=Anatomical Sciences Education|volume=10|issue=6|pages=549–559|doi=10.1002/ase.1696|issn=1935-9780|pmid=28419750|s2cid=25961448}} Some of the earliest cited examples include augmented reality used to support surgery by providing virtual overlays to guide medical practitioners, to AR content for astronomy and welding.{{Cite news|url=https://www.slashgear.com/dont-be-blind-on-wearable-cameras-insists-ar-genius-20239514/|title=Don't be blind on wearable cameras insists AR genius|date=20 July 2012|work=SlashGear|access-date=21 October 2018|language=en-US}} Example application areas described below include archaeology, architecture, commerce and education.
=Education and training=
In educational settings, text, graphics, video, and audio may be superimposed into a student's real-time environment.[https://web.archive.org/web/20111024105916/http://www.prweb.com/releases/2011/10/prweb8899908.htm Groundbreaking Augmented Reality-Based Reading Curriculum Launches], PRweb, 23 October 2011.Stewart-Smith, Hanna. [https://www.zdnet.com/article/education-with-augmented-reality-ar-textbooks-released-in-japan-video/ Education with Augmented Reality: AR textbooks released in Japan], ZDnet, 4 April 2012.[http://smarterlearning.wordpress.com/2011/11/10/augmented-reality-in-education/ Augmented reality in education] smarter learning. The 2015 Virtual, Augmented and Mixed Reality: 7th International Conference mentioned Google Glass as an example of augmented reality that can replace the physical classroom.{{Cite book|url=https://books.google.com/books?id=O7g0CgAAQBAJ&q=virternity|title=Virtual, Augmented and Mixed Reality: 7th International Conference, VAMR 2015, Held as Part of HCI International 2015, Los Angeles, CA, USA, 2–7 August 2015, Proceedings|last1=Shumaker|first1=Randall|last2=Lackey|first2=Stephanie|date=20 July 2015|publisher=Springer|isbn=9783319210674|language=en}} Content may be accessed by scanning or viewing an image with a mobile device or by using markerless AR techniques.{{Cite journal|last1=Moro|first1=Christian|last2=Birt|first2=James|last3=Stromberga|first3=Zane|last4=Phelps|first4=Charlotte|last5=Clark|first5=Justin|last6=Glasziou|first6=Paul|last7=Scott|first7=Anna Mae|date=2021|title=Virtual and Augmented Reality Enhancements to Medical and Science Student Physiology and Anatomy Test Performance: A Systematic Review and Meta-Analysis|url=https://onlinelibrary.wiley.com/doi/10.1002/ase.2049|journal=Anatomical Sciences Education|language=en|volume=14|issue=3|pages=368–376|doi=10.1002/ase.2049|pmid=33378557|s2cid=229929326|issn=1935-9772}}{{Cite web | url=https://www.edsurge.com/news/2015-11-02-how-to-transform-your-classroom-with-augmented-reality | title=How to Transform Your Classroom with Augmented Reality - EdSurge News| date=2 November 2015}}{{Cite web|url=https://medium.com/ancient-eu/why-we-need-more-tech-in-history-education-805fa10a7251|title=Why We Need More Tech in History Education|last=Crabben|first=Jan van der|date=16 October 2018|website=ancient.eu|access-date=2018-10-23|archive-date=23 October 2018|archive-url=https://web.archive.org/web/20181023195947/https://medium.com/ancient-eu/why-we-need-more-tech-in-history-education-805fa10a7251|url-status=dead}}
In 2017, a local government in Australia, used augmented and mixed reality technology to transform urban planning and community engagement. By integrating immersive 3D visualizations with real-time data, allowing stakeholders and citizens to interact with proposed developments in a more intuitive and dynamic way.{{cite journal |last1=An|first1=A |title=Adopting metaverse-related mixed reality technologies to tackle urban development challenges: An empirical study of an Australian municipal government |journal=IET Smart Cities |date=14 February 2023|volume=5 |issue=1 |pages=64–72|doi=10.1049/smc2.12051 |doi-access=free}}
In higher education, Construct3D, a Studierstube system, allows students to learn mechanical engineering concepts, math or geometry.{{Cite web |url=http://acdc.sav.us.es/pixelbit/images/stories/p41/15.pdf |title=Augmented reality, an evolution of the application of mobile devices |access-date=19 June 2014 |archive-url=https://web.archive.org/web/20150417053823/http://acdc.sav.us.es/pixelbit/images/stories/p41/15.pdf |archive-date=17 April 2015 |url-status=dead |df=dmy-all }} Chemistry AR apps allow students to visualize and interact with the spatial structure of a molecule using a marker object held in the hand.Maier, Patrick; Tönnis, Marcus; Klinker, Gudron. [http://ar.in.tum.de/pub/maierp2009ijas/maierp2009ijas.pdf Augmented Reality for teaching spatial relations] {{Webarchive|url=https://web.archive.org/web/20130128175343/http://ar.in.tum.de/pub/maierp2009ijas/maierp2009ijas.pdf |date=28 January 2013 }}, Conference of the International Journal of Arts & Sciences (Toronto 2009). Others have used HP Reveal, a free app, to create AR notecards for studying organic chemistry mechanisms or to create virtual demonstrations of how to use laboratory instrumentation.{{cite journal |last1=Plunkett |first1=Kyle N. |title=A Simple and Practical Method for Incorporating Augmented Reality into the Classroom and Laboratory |journal=Journal of Chemical Education |date=12 November 2019 |volume=96 |issue=11 |pages=2628–2631 |doi=10.1021/acs.jchemed.9b00607 |bibcode=2019JChEd..96.2628P |doi-access=free}}
Anatomy students can visualize different systems of the human body in three dimensions.{{cite web|url=https://www.vuforia.com/case-studies/anatomy-4d |title=Anatomy 4D |work=Qualcomm |access-date=2 July 2015 |url-status=dead |archive-url=https://web.archive.org/web/20160311085744/http://vuforia.com/case-studies/anatomy-4d |archive-date=11 March 2016 |df=dmy }} Using AR as a tool to learn anatomical structures has been shown to increase the learner knowledge and provide intrinsic benefits, such as increased engagement and learner immersion.{{Cite journal|last1=Moro|first1=Christian|last2=Štromberga|first2=Zane|last3=Raikos|first3=Athanasios|last4=Stirling|first4=Allan|date=November 2017|title=The effectiveness of virtual and augmented reality in health sciences and medical anatomy: VR and AR in Health Sciences and Medical Anatomy|journal=Anatomical Sciences Education|language=en|volume=10|issue=6|pages=549–559|doi=10.1002/ase.1696|pmid=28419750|s2cid=25961448|url=https://research.bond.edu.au/en/publications/d761ced8-4406-4a5e-ae3f-01862a09a36e}}{{Cite journal|last1=Birt|first1=James|last2=Stromberga|first2=Zane|last3=Cowling|first3=Michael|last4=Moro|first4=Christian|date=2018-01-31|title=Mobile Mixed Reality for Experiential Learning and Simulation in Medical and Health Sciences Education|journal=Information|language=en|volume=9|issue=2|pages=31|doi=10.3390/info9020031|issn=2078-2489|doi-access=free}}
AR has been used to develop different safety training applications for several types of disasters, such as, earthquakes and building fire, and health and safety tasks.{{Cite journal |last1=Catal |first1=Cagatay |last2=Akbulut |first2=Akhan |last3=Tunali |first3=Berkay |last4=Ulug |first4=Erol |last5=Ozturk |first5=Eren |date=2020-09-01 |title=Evaluation of augmented reality technology for the design of an evacuation training game |journal=Virtual Reality |language=en |volume=24 |issue=3 |pages=359–368 |doi=10.1007/s10055-019-00410-z |issn=1434-9957|doi-access=free }}{{Cite journal |last1=Gong |first1=Peizhen |last2=Lu |first2=Ying |last3=Lovreglio |first3=Ruggiero |last4=Lv |first4=Xiaofeng |last5=Chi |first5=Zexun |date=2024-10-01 |title=Applications and effectiveness of augmented reality in safety training: A systematic literature review and meta-analysis |url=https://www.sciencedirect.com/science/article/pii/S0925753524002145 |journal=Safety Science |volume=178 |pages=106624 |doi=10.1016/j.ssci.2024.106624 |issn=0925-7535}}{{Cite journal |last1=Paes |first1=Daniel |last2=Feng |first2=Zhenan |last3=King |first3=Maddy |last4=Khorrami Shad |first4=Hesam |last5=Sasikumar |first5=Prasanth |last6=Pujoni |first6=Diego |last7=Lovreglio |first7=Ruggiero |date=June 2024 |title=Optical see-through augmented reality fire safety training for building occupants |journal=Automation in Construction |language=en |volume=162 |pages=105371 |doi=10.1016/j.autcon.2024.105371|doi-access=free }} Further, several AR solutions have been proposed and tested to navigate building evacuees towards safe places in both large scale and small scale disasters.{{Cite journal |last1=Lovreglio |first1=Ruggiero |last2=Kinateder |first2=Max |date=October 2020 |title=Augmented reality for pedestrian evacuation research: Promises and limitations |url=https://linkinghub.elsevier.com/retrieve/pii/S0925753520301478 |journal=Safety Science |language=en |volume=128 |pages=104750 |doi=10.1016/j.ssci.2020.104750}}{{Cite book |last1=Mantoro |first1=Teddy |last2=Alamsyah |first2=Zaenal |last3=Ayu |first3=Media Anugerah |chapter=Pathfinding for Disaster Emergency Route Using Sparse A* and Dijkstra Algorithm with Augmented Reality |date=October 2021 |title=2021 IEEE 7th International Conference on Computing, Engineering and Design (ICCED) |chapter-url=https://ieeexplore.ieee.org/document/9664869/;jsessionid=ji9AewRr7XUqMhvK4eTjYawVSsfm_uYd8B6qi3p56mlvzZQMEkTV!1091101768 |pages=1–6 |doi=10.1109/ICCED53389.2021.9664869|isbn=978-1-6654-3996-1 }} AR applications can have several overlapping with many other digital technologies, such as BIM, internet of things and artificial intelligence, to generate smarter safety training and navigation solutions.{{Citation |last1=Lovreglio |first1=R. |title=Digital Technologies for Fire Evacuations |date=2024 |work=Intelligent Building Fire Safety and Smart Firefighting |pages=439–454 |editor-last=Huang |editor-first=Xinyan |url=https://link.springer.com/10.1007/978-3-031-48161-1_18 |access-date=2024-03-15 |place=Cham |publisher=Springer Nature Switzerland |language=en |doi=10.1007/978-3-031-48161-1_18 |isbn=978-3-031-48160-4 |last2=Paes |first2=D. |last3=Feng |first3=Z. |last4=Zhao |first4=X. |editor2-last=Tam |editor2-first=Wai Cheong}}
=Remote collaboration=
Primary school children learn easily from interactive experiences. As an example, astronomical constellations and the movements of objects in the Solar System were oriented in 3D and overlaid in the direction the device was held, and expanded with supplemental video information. Paper-based science book illustrations could seem to come alive as video without requiring the child to navigate to web-based materials.
In 2013, a project was launched on Kickstarter to teach about electronics with an educational toy that allowed children to scan their circuit with an iPad and see the electric current flowing around.{{Cite web|url=https://circuits.lightup.io/|title=LightUp - An award-winning toy that teaches kids about circuits and coding|website=LightUp|language=en-US|access-date=29 August 2018|archive-url=https://web.archive.org/web/20180829110100/https://circuits.lightup.io/|archive-date=29 August 2018|url-status=dead}} While some educational apps were available for AR by 2016, it was not broadly used. Apps that leverage augmented reality to aid learning included SkyView for studying astronomy,{{Cite web|title = Terminal Eleven: SkyView – Explore the Universe|url = http://www.terminaleleven.com/skyview/iphone/|website = www.terminaleleven.com|access-date = 15 February 2016}} AR Circuits for building simple electric circuits,{{Cite web|title = AR Circuits – Augmented Reality Electronics Kit|url = http://arcircuits.com|website = arcircuits.com|access-date = 15 February 2016}} and SketchAR for drawing.{{Cite web|url=http://sketchar.tech|title=SketchAR - start drawing easily using augmented reality|website=sketchar.tech|access-date=20 May 2018}}
AR would also be a way for parents and teachers to achieve their goals for modern education, which might include providing more individualized and flexible learning, making closer connections between what is taught at school and the real world, and helping students to become more engaged in their own learning.
=Visual art=
AR applied in the visual arts allows objects or places to trigger artistic multidimensional experiences and interpretations of reality.
The Australian new media artist Jeffrey Shaw pioneered Augmented Reality in three artworks: Viewpoint in 1975, Virtual Sculptures in 1987 and The Golden Calf in 1993.{{Cite book |last=Duguet |first=Anne-Marie |title=Jeffrey Shaw, Future Cinema. The Cinematic Imaginary after Film |publisher=ZKM Karlsruhe and MIT Press, Cambridge, Massachusetts |year=2003 |isbn=9780262692861 |pages=376–381}}{{Cite book |last1=Duguet |first1=Anne-Marie |title=Jeffrey Shaw: A User's Manual. From Expanded Cinema to Virtual Reality |last2=Klotz |first2=Heinrich |last3=Weibel |first3=Peter |publisher=ZKM Cantz |year=1997 |isbn= |pages=9–20}} He continues to explore new permutations of AR in numerous recent works.
Manifest.AR was an international artists' collective founded in 2010 that specialized in augmented reality (AR) art and interventions. The collective typically created site-specific AR installations that could be viewed through mobile devices using custom-developed applications. Their work often placed virtual artworks in spaces without institutional permission. The collective gained prominence in 2010 when they staged an unauthorized virtual exhibition at the Museum of Modern Art (MoMA) in New York City, overlaying their digital artworks throughout the museum's spaces using AR technology. The collective's unauthorized AR intervention at MoMA involved placing virtual artworks throughout the museum's spaces, viewable through mobile devices. The group published their "AR Art Manifesto" in 2011, which outlined their artistic philosophy and approach to augmented reality as a medium. The manifesto emphasized the democratic potential of AR technology and its ability to challenge traditional institutional control over public space and art display.Freeman, John Craig. "ManifestAR: An Augmented Reality Manifesto." Leonardo Electronic Almanac, Vol. 19, No. 1, 2013. Manifest.AR has been influential in: Pioneering artistic applications of AR technology; Developing new forms of institutional critique; Expanding concepts of public art and digital space; and Influencing subsequent generations of new media artists. Their work has been documented and discussed in various publications about digital art and new media, and has influenced contemporary discussions about virtual and augmented reality in artistic practice.Paul, Christiane. "Digital Art" (Third edition). Thames & Hudson, 2015.
Augmented reality can aid in the progression of visual art in museums by allowing museum visitors to view artwork in galleries in a multidimensional way through their phone screens.{{Cite journal|last1=tom Dieck|first1=M. Claudia|last2=Jung|first2=Timothy|last3=Han|first3=Dai-In|date=July 2016|title=Mapping requirements for the wearable smart glasses augmented reality museum application|url=https://www.emerald.com/insight/content/doi/10.1108/JHTT-09-2015-0036/full/html|journal=Journal of Hospitality and Tourism Technology|language=en|volume=7|issue=3|pages=230–253|doi=10.1108/JHTT-09-2015-0036|issn=1757-9880}} The Museum of Modern Art in New York has created an exhibit in their art museum showcasing AR features that viewers can see using an app on their smartphone.{{Cite book|url=https://books.google.com/books?id=OyGiW2OYI8AC&q=augmented+reality:+an+emerging+technologies+guide+to+AR&pg=PR1|title=Augmented Reality: An Emerging Technologies Guide to AR|last1=Kipper|first1=Greg|last2=Rampolla|first2=Joseph|date=31 December 2012|publisher=Elsevier|isbn=9781597497343|language=en}} The museum has developed their personal app, called MoMAR Gallery, that museum guests can download and use in the augmented reality specialized gallery in order to view the museum's paintings in a different way.{{Cite magazine|url=https://www.wired.com/story/augmented-reality-art-museums/|title=Augmented Reality Is Transforming Museums|magazine=WIRED|access-date=30 September 2018 |language=en-US}} This allows individuals to see hidden aspects and information about the paintings, and to be able to have an interactive technological experience with artwork as well.
AR technology aided the development of eye tracking technology to translate a disabled person's eye movements into drawings on a screen.{{cite magazine |title=The 50 Best Inventions of 2010 - EyeWriter|url=http://www.time.com/time/specials/packages/article/0,28804,2029497_2030618_2029822,00.html |magazine=Time |access-date=26 March 2024|archive-url=https://web.archive.org/web/20101114075903/http://www.time.com/time/specials/packages/article/0,28804,2029497_2030618_2029822,00.html|archive-date=2010-11-14|date=11 November 2010|last=Webley|first=Kayla}}
A Danish artist, Olafur Eliasson, has placed objects like burning suns, extraterrestrial rocks, and rare animals, into the user's environment.{{Cite web|url=https://www.dezeen.com/2020/05/14/olafur-eliasson-augmented-reality-wunderkammer/|title=Olafur Eliasson creates augmented-reality cabinet of curiosities|date=14 May 2020|language=en-US|access-date=2020-05-17}} Martin & Muñoz started using Augmented Reality (AR) technology in 2020 to create and place virtual works, based on their snow globes, in their exhibitions and in user's environments. Their first AR work was presented at the Cervantes Institute in New York in early 2022.{{Cite web|url=https://www.spainculture.us/city/new-york/walter-martin-paloma-munoz-the-houses-are-blind-but-the-trees-can-see/|title=The Houses are Blind but the Trees Can See|date= March 2022|language=en-US|access-date=2023-02-07}}
{{Further|topic=the 2004 augmented reality outdoor art project|LifeClipper}}
= Industrial manufacturing =
AR is used to substitute paper manuals with digital instructions which are overlaid on the manufacturing operator's field of view, reducing mental effort required to operate.{{Cite journal|last1=Mourtzis|first1=Dimitris|last2=Zogopoulos|first2=Vasilios|last3=Xanthi|first3=Fotini|s2cid=189904235|date=2019-06-11|title=Augmented reality application to support the assembly of highly customized products and to adapt to production re-scheduling|journal=The International Journal of Advanced Manufacturing Technology|volume=105|issue=9|pages=3899–3910|language=en|doi=10.1007/s00170-019-03941-6|issn=0268-3768}} AR makes machine maintenance efficient because it gives operators direct access to a machine's maintenance history.{{Citation|last1=Boccaccio|first1=A.|title=Exploiting Augmented Reality to Display Technical Information on Industry 4.0 P&ID|date=2019|work=Advances on Mechanics, Design Engineering and Manufacturing II|pages=282–291|editor-last=Cavas-Martínez|editor-first=Francisco|publisher=Springer International Publishing|language=en|doi=10.1007/978-3-030-12346-8_28|isbn=978-3-030-12345-1|last2=Cascella|first2=G. L.|last3=Fiorentino|first3=M.|last4=Gattullo|first4=M.|last5=Manghisi|first5=V. M.|last6=Monno|first6=G.|last7=Uva|first7=A. E.|series=Lecture Notes in Mechanical Engineering |s2cid=150159603|editor2-last=Eynard|editor2-first=Benoit|editor3-last=Fernández Cañavate|editor3-first=Francisco J.|editor4-last=Fernández-Pacheco|editor4-first=Daniel G.}} Virtual manuals help manufacturers adapt to rapidly-changing product designs, as digital instructions are more easily edited and distributed compared to physical manuals.
Digital instructions increase operator safety by removing the need for operators to look at a screen or manual away from the working area, which can be hazardous. Instead, the instructions are overlaid on the working area.{{Cite journal|last1=Mourtzis|first1=Dimitris|last2=Zogopoulos|first2=Vasilios|last3=Katagis|first3=Ioannis|last4=Lagios|first4=Panagiotis|date=2018|title=Augmented Reality based Visualization of CAM Instructions towards Industry 4.0 paradigm: a CNC Bending Machine case study|journal=Procedia CIRP|language=en|volume=70|pages=368–373|doi=10.1016/j.procir.2018.02.045|doi-access=free}}{{cite journal|title=An Augmented Reality inspection tool to support workers in Industry 4.0 environments|journal=Computers in Industry|date=2021|doi=10.1016/j.compind.2021.103412 |url=https://doi.org/10.1016/j.compind.2021.103412 |last1=Marino |first1=Emanuele |last2=Barbieri |first2=Loris |last3=Colacino |first3=Biagio |last4=Fleri |first4=Anna Kum |last5=Bruno |first5=Fabio |volume=127 |s2cid=232272256 }} The use of AR can increase operators' feeling of safety when working near high-load industrial machinery by giving operators additional information on a machine's status and safety functions, as well as hazardous areas of the workspace.{{Cite journal|last1=Michalos|first1=George|last2=Kousi|first2=Niki|last3=Karagiannis|first3=Panagiotis|last4=Gkournelos|first4=Christos|last5=Dimoulas|first5=Konstantinos|last6=Koukas|first6=Spyridon|last7=Mparis|first7=Konstantinos|last8=Papavasileiou|first8=Apostolis|last9=Makris|first9=Sotiris|date=November 2018|title=Seamless human robot collaborative assembly – An automotive case study|journal=Mechatronics|volume=55|pages=194–211|doi=10.1016/j.mechatronics.2018.08.006|s2cid=115979090|issn=0957-4158}}
=Video games=
{{redirect-distinguish|Augmented reality game|alternate reality game}}
{{Redirect|AR games|the Nintendo 3DS game|AR Games{{!}}AR Games}}
{{See also|List of augmented reality video games}}
File:Desjardins AR Augmented Reality Game, March 2013.png]]
The gaming industry embraced AR technology. A number of games were developed for prepared indoor environments, such as AR air hockey, Titans of Space, collaborative combat against virtual enemies, and AR-enhanced pool table games.Hawkins, Mathew. [http://www.gamesetwatch.com/2011/10/augmented_reality_used_to_enhance_both_pool_and_air_hockey.php Augmented Reality Used To Enhance Both Pool And Air Hockey] Game Set Watch15 October 2011.[http://combathelo.blogspot.com/2012/07/one-week-only-augmented-reality-project.html One Week Only – Augmented Reality Project] {{webarchive |url=https://web.archive.org/web/20131106180740/http://combathelo.blogspot.com/2012/07/one-week-only-augmented-reality-project.html |date=6 November 2013 }} Combat-HELO Dev Blog 31 July 2012.{{Cite web |url=http://getandroidstuff.com/best-augmented-reality-apps-vr-games-android/ |title=Best VR, Augmented Reality apps & games on Android |access-date=14 February 2017 |archive-url=https://web.archive.org/web/20170215114103/http://getandroidstuff.com/best-augmented-reality-apps-vr-games-android/ |archive-date=15 February 2017 |url-status=dead }}
In 2010, Ogmento became the first AR gaming startup to receive venture capital funding. The company went on to produce early location-based AR games for titles like Paranormal Activity: Sanctuary, NBA: King of the Court, and Halo: King of the Hill. Ogmento's computer vision technology was eventually repackaged and sold to Apple, became a major contribution to ARKit.{{cite web | url=https://techcrunch.com/2010/05/26/ogmento-first-ar-gaming-startup-to-win-vc-funding/ | title=Ogmento First AR Gaming Startup to Win VC Funding | date=26 May 2010 }}
Augmented reality allows video game players to experience digital game play in a real-world environment. Niantic released the augmented reality mobile game Pokémon Go.{{cite web|last1=Swatman|first1=Rachel|title=Pokémon Go catches five new world records|url=http://www.guinnessworldrecords.com/news/2016/8/pokemon-go-catches-five-world-records-439327|publisher=Guinness World Records|access-date=28 August 2016|date=10 August 2016}} Disney has partnered with Lenovo to create the augmented reality game Star Wars: Jedi Challenges that works with a Lenovo Mirage AR headset, a tracking sensor and a Lightsaber controller, scheduled to launch in December 2017.{{Cite web | url=https://www.cnbc.com/2017/08/31/star-wars-jedi-challenges-augmented-reality-game-launches-with-lenovo-mirage-headset.html | title='Star Wars' augmented reality game that lets you be a Jedi launched| website=CNBC| date=31 August 2017}}
=Industrial design=
{{Main|Industrial augmented reality}}
AR allows industrial designers to experience a product's design and operation before completion. Volkswagen has used AR for comparing calculated and actual crash test imagery.{{cite book |doi=10.1109/ISMAR.2002.1115108 |chapter=Stereo augmentation of simulation results on a projection wall by combining two basic ARVIKA systems |title=Proceedings. International Symposium on Mixed and Augmented Reality |pages=271–322 |year=2002 |last1=Noelle |first1=S. |isbn=0-7695-1781-1 |citeseerx=10.1.1.121.1268 |s2cid=24876142 }} AR has been used to visualize and modify car body structure and engine layout. It has also been used to compare digital mock-ups with physical mock-ups to find discrepancies between them.{{cite web|last1=Verlinden |first1=Jouke |last2=Horvath |first2=Imre |title=Augmented Prototyping as Design Means in Industrial Design Engineering |publisher=Delft University of Technology |url=http://www.io.tudelft.nl/index.php?id=24954&L=1 |access-date=7 October 2012 |url-status=dead |archive-url=https://web.archive.org/web/20130616010611/http://www.io.tudelft.nl/index.php?id=24954&L=1 |archive-date=16 June 2013 |df=dmy }}{{cite news |last1=Pang |first1=Y. |last2=Nee |first2=Andrew Y. C. |last3=Youcef-Toumi |first3=Kamal |last4=Ong |first4=S. K. |last5=Yuan |first5=M. L. |title=Assembly Design and Evaluation in an Augmented Reality Environment |date=January 2005 |hdl=1721.1/7441 }}
=Architecture=
AR can aid in visualizing building projects. Computer-generated images of a structure can be superimposed onto a real-life local view of a property before the physical building is constructed there; this was demonstrated publicly by Trimble Navigation in 2004. AR can also be employed within an architect's workspace, rendering animated 3D visualizations of their 2D drawings. Architecture sight-seeing can be enhanced with AR applications, allowing users viewing a building's exterior to virtually see through its walls, viewing its interior objects and layout.Divecha, Devina.[http://www.designmena.com/inspiration/augmented-reality-ar-part-architecture-design Augmented Reality (AR) used in architecture and design] {{Webarchive|url=https://web.archive.org/web/20130214173708/http://www.designmena.com/inspiration/augmented-reality-ar-part-architecture-design |date=14 February 2013 }}. designMENA 8 September 2011.[http://www.news.uwa.edu.au/201203054410/events/architectural-dreams-agumented-reality Architectural dreams in augmented reality]. University News, University of Western Australia. 5 March 2012.[https://www.youtube.com/watch?v=jL3C-OVQKWU Outdoor AR]. TV One News, 8 March 2004.
With continual improvements to GPS accuracy, businesses are able to use augmented reality to visualize georeferenced models of construction sites, underground structures, cables and pipes using mobile devices.{{cite web|last=Churcher|first=Jason|title=Internal accuracy vs external accuracy|url=http://www.augview.net/blog/archive-7May2013.html|access-date=7 May 2013}} Augmented reality is applied to present new projects, to solve on-site construction challenges, and to enhance promotional materials.{{cite web|title=Augment for Architecture & Construction|url=http://www.augmentedev.com/augmented-reality-architecture/|access-date=12 October 2015|archive-url=https://web.archive.org/web/20151108054418/http://www.augmentedev.com/augmented-reality-architecture/|archive-date=8 November 2015|url-status=dead|df=dmy-all}} Examples include the Daqri Smart Helmet, an Android-powered hard hat used to create augmented reality for the industrial worker, including visual instructions, real-time alerts, and 3D mapping.
Following the Christchurch earthquake, the University of Canterbury released CityViewAR,{{Cite web|url=https://www.stuff.co.nz/technology/digital-living/6121248/App-gives-a-view-of-city-as-it-used-to-be|title=App gives a view of city as it used to be|website=Stuff|date=10 December 2011|language=en|access-date=20 May 2018}} which enabled city planners and engineers to visualize buildings that had been destroyed.{{cite book|last=Lee|first=Gun|s2cid=34199215|chapter=CityViewAR outdoor AR visualization|year=2012|publisher=ACM|isbn=978-1-4503-1474-9|chapter-url=http://dl.acm.org/citation.cfm?id=2379281|doi=10.1145/2379256.2379281|title=Proceedings of the 13th International Conference of the NZ Chapter of the ACM's Special Interest Group on Human-Computer Interaction - CHINZ '12|pages=97|hdl=10092/8693}} This not only provided planners with tools to reference the previous cityscape, but it also served as a reminder of the magnitude of the resulting devastation, as entire buildings had been demolished.
=Archaeology=
AR has been used to aid archaeological research. By augmenting archaeological features onto the modern landscape, AR allows archaeologists to formulate possible site configurations from extant structures.{{cite journal |title=Augmenting Phenomenology: Using Augmented Reality to Aid Archaeological Phenomenology in the Landscape |author=Stuart Eve |doi=10.1007/s10816-012-9142-7 | volume=19 |issue=4 |journal=Journal of Archaeological Method and Theory |pages=582–600|url=http://discovery.ucl.ac.uk/1352447/1/Eve_2012_Augmented_Phenomenology.pdf |year=2012 |s2cid=4988300 }} Computer generated models of ruins, buildings, landscapes or even ancient people have been recycled into early archaeological AR applications.{{cite book |url=http://portal.acm.org/citation.cfm?id=854948 |title=Archeoguide: System Architecture of a Mobile Outdoor Augmented Reality System |author1=Dähne, Patrick |author2=Karigiannis, John N. |access-date=6 January 2010|isbn=9780769517810 |year=2002 |publisher=IEEE Computer Society Press }}{{cite web |url=http://archpro.lbg.ac.at/press-release/school-gladiators-discovered-roman-carnuntum-austria |title=School of Gladiators discovered at Roman Carnuntum, Austria |author=LBI-ArchPro |access-date=29 December 2014|date=5 September 2011}}{{Cite journal|title = Mixing virtual and real scenes in the site of ancient Pompeii|journal = Computer Animation and Virtual Worlds|date = 1 February 2005|issn = 1546-427X|pages = 11–24|volume = 16|issue = 1|doi = 10.1002/cav.53|first1 = George|last1 = Papagiannakis|first2 = Sébastien|last2 = Schertenleib|first3 = Brian|last3 = O'Kennedy|first4 = Marlene|last4 = Arevalo-Poizat|first5 = Nadia|last5 = Magnenat-Thalmann|first6 = Andrew|last6 = Stoddart|first7 = Daniel|last7 = Thalmann|citeseerx = 10.1.1.64.8781|s2cid = 5341917}} For example, implementing a system like VITA (Visual Interaction Tool for Archaeology) will allow users to imagine and investigate instant excavation results without leaving their home. Each user can collaborate by mutually "navigating, searching, and viewing data". Hrvoje Benko, a researcher in the computer science department at Columbia University, points out that these particular systems and others like them can provide "3D panoramic images and 3D models of the site itself at different excavation stages" all the while organizing much of the data in a collaborative way that is easy to use. Collaborative AR systems supply multimodal interactions that combine the real world with virtual images of both environments.{{Cite book |doi = 10.1109/ISMAR.2004.23|chapter = Collaborative Mixed Reality Visualization of an Archaeological Excavation|title = Third IEEE and ACM International Symposium on Mixed and Augmented Reality|pages = 132–140|year = 2004|last1 = Benko|first1 = H.|last2 = Ishak|first2 = E.W.|last3 = Feiner|first3 = S.|s2cid = 10122485|isbn = 0-7695-2191-6}}
=Commerce=
{{main|Commercial augmented reality}}
AR is used to integrate print and video marketing. Printed marketing material can be designed with certain "trigger" images that, when scanned by an AR-enabled device using image recognition, activate a video version of the promotional material. A major difference between augmented reality and straightforward image recognition is that one can overlay multiple media at the same time in the view screen, such as social media share buttons, the in-page video even audio and 3D objects. Traditional print-only publications are using augmented reality to connect different types of media.Katts, Rima. [http://www.mobilemarketer.com/cms/news/software-technology/13810.html Elizabeth Arden brings new fragrance to life with augmented reality] Mobile Marketer, 19 September 2012.Meyer, David. [http://gigaom.com/europe/telefonica-bets-on-augmented-reality-with-aurasma-tie-in/ Telefónica bets on augmented reality with Aurasma tie-in] gigaom, 17 September 2012.Mardle, Pamela.[http://www.printweek.com/news/1153133/Video-becomes-reality-Stuprintcom/ Video becomes reality for Stuprint.com] {{webarchive |url=https://web.archive.org/web/20130312171811/http://www.printweek.com/news/1153133/Video-becomes-reality-Stuprintcom/ |date=12 March 2013 }}. PrintWeek, 3 October 2012.Giraldo, Karina.[http://www.solinix.co/blog/marketing-movil-su-importancia-para-las-marcas/ Why mobile marketing is important for brands?] {{webarchive|url=https://web.archive.org/web/20150402135323/http://solinix.co/blog/marketing-movil-su-importancia-para-las-marcas/ |date=2 April 2015 }}. SolinixAR, Enero 2015.{{cite news|title=Augmented reality could be advertising world's best bet|url=http://www.financialexpress.com/article/industry/companies/augmented-reality-could-be-advertising-worlds-best-bet/64855/|agency=The Financial Express|date=18 April 2015|url-status=dead|archive-url=https://web.archive.org/web/20150521061314/http://www.financialexpress.com/article/industry/companies/augmented-reality-could-be-advertising-worlds-best-bet/64855/|archive-date=21 May 2015|df=dmy-all}}
AR can enhance product previews such as allowing a customer to view what's inside a product's packaging without opening it.Humphries, Mathew.[http://www.geek.com/articles/gadgets/lego-demos-augmented-reality-boxes-with-gesture-recognition-20110919/] {{Webarchive|url=https://web.archive.org/web/20120626192637/http://www.geek.com/articles/gadgets/lego-demos-augmented-reality-boxes-with-gesture-recognition-20110919/|date=26 June 2012}}.Geek.com 19 September 2011. AR can also be used as an aid in selecting products from a catalog or through a kiosk. Scanned images of products can activate views of additional content such as customization options and additional images of the product in its use.Netburn, Deborah.[https://www.latimes.com/business/technology/la-ikeas-augmented-reality-app-20120723,0,1261315.story Ikea introduces augmented reality app for 2013 catalog] {{Webarchive|url=https://web.archive.org/web/20121202070158/http://www.latimes.com/business/technology/la-ikeas-augmented-reality-app-20120723,0,1261315.story |date=2 December 2012 }}. Los Angeles Times, 23 July 2012.
In 2018, Apple announced Universal Scene Description (USDZ) AR file support for iPhones and iPads with iOS 12. Apple has created an AR QuickLook Gallery that allows people to experience augmented reality through their own Apple device.{{cite web | url = https://www.computerworld.com/article/3307437/mobile-wireless/this-small-ios-12-feature-is-the-birth-of-a-whole-industry.html | title = This small iOS 12 feature is the birth of a whole industry | publisher = Jonny Evans | access-date = 19 September 2018| date = 19 September 2018 }}
In 2018, Shopify, the Canadian e-commerce company, announced AR Quick Look integration. Their merchants will be able to upload 3D models of their products and their users will be able to tap on the models inside the Safari browser on their iOS devices to view them in their real-world environments.{{cite web | url = https://techcrunch.com/2018/09/17/shopify-is-bringing-apples-latest-ar-tech-to-their-platform/ | title = Shopify is bringing Apple's latest AR tech to their platform | date = 17 September 2018 | publisher = Lucas Matney | access-date = 3 December 2018}}
In 2018, Twinkl released a free AR classroom application. Pupils can see how York looked over 1,900 years ago.{{cite journal | url = https://www.qaeducation.co.uk/article/ar-classroom-york | title = History re-made: New AR classroom application lets pupils see how York looked over 1,900 years ago | journal = QA Education| access-date = 4 September 2018| date = 4 September 2018 | last1 = Magazine | first1 = QA Education }} Twinkl launched the first ever multi-player AR game, Little Red{{cite journal | url = https://www.prolificnorth.co.uk/news/digital/2018/09/sheffields-twinkl-claims-ar-first-new-game| title = Sheffield's Twinkl claims AR first with new game | journal = Prolific North| access-date = 19 September 2018| date = 19 September 2018}} and has over 100 free AR educational models.{{cite journal | url = http://www.the-educator.org/technology-from-twinkl-brings-never-seen-before-objects-to-the-classroom/ | title = Technology from Twinkl brings never seen before objects to the classroom | journal = The Educator UK| access-date = 21 December 2018| date = 21 September 2018}}
Augmented reality is becoming more frequently used for online advertising. Retailers offer the ability to upload a picture on their website and "try on" various clothes which are overlaid on the picture. Even further, companies such as Bodymetrics install dressing booths in department stores that offer full-body scanning. These booths render a 3D model of the user, allowing the consumers to view different outfits on themselves without the need of physically changing clothes.Pavlik, John V., and Shawn McIntosh. "Augmented Reality." Converging Media: a New Introduction to Mass Communication, 5th ed., Oxford University Press, 2017, pp. 184–185. For example, JC Penney and Bloomingdale's use "virtual dressing rooms" that allow customers to see themselves in clothes without trying them on.{{cite journal |last1=Dacko |first1=Scott G. |title=Enabling smart retail settings via mobile augmented reality shopping apps |journal=Technological Forecasting and Social Change |date=November 2017 |volume=124 |pages=243–256 |doi=10.1016/j.techfore.2016.09.032 |url=http://wrap.warwick.ac.uk/81922/5/WRAP-enabling-smart-retail-Dacko-2017.pdf }} Another store that uses AR to market clothing to its customers is Neiman Marcus.{{Cite news|url=https://www.retaildive.com/news/how-neiman-marcus-is-turning-technology-innovation-into-a-core-value/436590/|title=How Neiman Marcus is turning technology innovation into a 'core value'|work=Retail Dive|access-date=23 September 2018|language=en-US}} Neiman Marcus offers consumers the ability to see their outfits in a 360-degree view with their "memory mirror". Makeup stores like L'Oreal, Sephora, Charlotte Tilbury, and Rimmel also have apps that utilize AR. These apps allow consumers to see how the makeup will look on them. According to Greg Jones, director of AR and VR at Google, augmented reality is going to "reconnect physical and digital retail".
AR technology is also used by furniture retailers such as IKEA, Houzz, and Wayfair.{{Cite news|url=https://www.forbes.com/sites/rachelarthur/2017/10/31/augmented-reality-is-set-to-transform-fashion-and-retail/#364c701b3151|title=Augmented Reality Is Set To Transform Fashion And Retail|last=Arthur|first=Rachel|work=Forbes|access-date=23 September 2018|language=en}} These retailers offer apps that allow consumers to view their products in their home prior to purchasing anything.{{cite web |url=https://archvisualizations.com/augmented-reality-apps-for-interior-visualization/ |title=Augmented Reality Apps for Interior Visualization |access-date=2024-04-09 |website=archvisualizations.com|date=30 January 2024 }}
In 2017, Ikea announced the Ikea Place app. It contains a catalogue of over 2,000 products—nearly the company's full collection of sofas, armchairs, coffee tables, and storage units which one can place anywhere in a room with their phone.{{cite magazine | url = https://www.wired.com/story/ikea-place-ar-kit-augmented-reality/ | title = IKEA's new app flaunts what you'll love most about AR| magazine = Wired | access-date = 20 September 2017| date = 20 September 2017| last1 = Pardes| first1 = Arielle}} The app made it possible to have 3D and true-to-scale models of furniture in the customer's living space. IKEA realized that their customers are not shopping in stores as often or making direct purchases anymore.{{Cite web|url=https://www.ikea.com/ms/en_CH/this-is-ikea/ikea-highlights/2017/ikea-place-app/index.html|title=IKEA Highlights 2017|access-date=8 October 2018|archive-date=8 October 2018|archive-url=https://web.archive.org/web/20181008214446/https://www.ikea.com/ms/en_CH/this-is-ikea/ikea-highlights/2017/ikea-place-app/index.html|url-status=dead}}{{Cite web|url=https://www.inter.ikea.com/en/performance|archiveurl=https://web.archive.org/web/20180626015939/https://highlights.ikea.com/2017/facts-and-figures/|url-status=dead|title=Performance|archivedate=26 June 2018|website=www.inter.ikea.com}} Shopify's acquisition of Primer, an AR app aims to push small and medium-sized sellers towards interactive AR shopping with easy to use AR integration and user experience for both merchants and consumers.{{Cite web|title=How Shopify is setting the future of AR shopping and what it means for sellers|date=29 June 2021 |url=https://www.suntecindia.com/blog/how-shopify-is-setting-the-future-of-ar-shopping-and-what-it-means-for-sellers/|access-date=2021-06-29|language=en-US}} AR helps the retail industry reduce operating costs. Merchants upload product information to the AR system, and consumers can use mobile terminals to search and generate 3D maps.{{Cite journal |last1=Indriani |first1=Masitoh |last2=Liah Basuki Anggraeni |date=2022-06-30 |title=What Augmented Reality Would Face Today? The Legal Challenges to the Protection of Intellectual Property in Virtual Space |journal=Media Iuris |volume=5 |issue=2 |pages=305–330 |doi=10.20473/mi.v5i2.29339 |s2cid=250464007 |issn=2621-5225|doi-access=free }}
= Literature =
The first description of AR as it is known today was in Virtual Light, the 1994 novel by William Gibson. In 2011, AR was blended with poetry by ni ka from Sekai Camera in Tokyo, Japan. The prose of these AR poems come from Paul Celan, Die Niemandsrose, expressing the aftermath of the 2011 Tōhoku earthquake and tsunami.{{Cite web|url=http://yaplog.jp/tipotipo/category_33/|title=AR詩 | にかにかブログ! (おぶんがく&包丁&ちぽちぽ革命)|website=にかにかブログ! (おぶんがく&包丁&ちぽちぽ革命)|language=ja-JP|access-date=20 May 2018}}
=Fitness=
AR hardware and software for use in fitness includes smart glasses made for biking and running, with performance analytics and map navigation projected onto the user's field of vision,{{Cite web|title=Augmented Reality (AR) vs. virtual reality (VR): What's the Difference?|url=https://www.pcmag.com/news/augmented-reality-ar-vs-virtual-reality-vr-whats-the-difference|access-date=2020-11-06|website=PCMAG|language=en}} and boxing, martial arts, and tennis, where users remain aware of their physical environment for safety.{{Cite web|author=Sandee LaMotte|title=The very real health dangers of virtual reality|url=https://www.cnn.com/2017/12/13/health/virtual-reality-vr-dangers-safety/index.html|access-date=2020-11-06|website=CNN|date=13 December 2017}} Fitness-related games and software include Pokémon Go and Jurassic World Alive.{{Cite web|last=Thier|first=Dave|title='Jurassic World Alive' Makes Two Big Improvements Over 'Pokémon GO'|url=https://www.forbes.com/sites/davidthier/2018/06/04/jurassic-world-alive-makes-two-big-improvements-over-pokemon-go/|access-date=2020-11-06|website=Forbes|language=en}}
=Emergency management/search and rescue=
Augmented reality systems are used in public safety situations, from super storms to suspects at large.
As early as 2009, two articles from Emergency Management discussed AR technology for emergency management. The first was "Augmented Reality—Emerging Technology for Emergency Management", by Gerald Baron."Augmented Reality—Emerging Technology for Emergency Management", Emergency Management 24 September 2009. According to Adam Crow,: "Technologies like augmented reality (ex: Google Glass) and the growing expectation of the public will continue to force professional emergency managers to radically shift when, where, and how technology is deployed before, during, and after disasters.""What Does the Future Hold for Emergency Management?", Emergency Management Magazine, 8 November 2013
Another early example was a search aircraft looking for a lost hiker in rugged mountain terrain. Augmented reality systems provided aerial camera operators with a geographic awareness of forest road names and locations blended with the camera video. The camera operator was better able to search for the hiker knowing the geographic context of the camera image. Once located, the operator could more efficiently direct rescuers to the hiker's location because the geographic position and reference landmarks were clearly labeled.{{cite thesis |type=Master's thesis |last1=Cooper |first1=Joseph |title=Supporting Flight Control for UAV-Assisted Wilderness Search and Rescue Through Human Centered Interface Design |publisher=Brigham Young University |date=15 November 2007 |url=https://scholarsarchive.byu.edu/etd/1217/ }}
=Social interaction=
AR can be used to facilitate social interaction. An augmented reality social network framework called Talk2Me enables people to disseminate information and view others' advertised information in an augmented reality way. The timely and dynamic information sharing and viewing functionalities of Talk2Me help initiate conversations and make friends for users with people in physical proximity.{{cite book |doi=10.1109/PERCOM.2018.8444578 |chapter=Talk2Me: A Framework for Device-to-Device Augmented Reality Social Network |title=2018 IEEE International Conference on Pervasive Computing and Communications (Per Com) |pages=1–10 |year=2018 |last1=Shu |first1=Jiayu |last2=Kosta |first2=Sokol |last3=Zheng |first3=Rui |last4=Hui |first4=Pan |s2cid=44017349 |isbn=978-1-5386-3224-6 }} However, use of an AR headset can inhibit the quality of an interaction between two people if one isn't wearing one if the headset becomes a distraction.{{cite web |title=Effects of Augmented Reality on Social Interactions |url=https://www.electronicsdiary.com/2019/05/effects-of-augmented-reality-on-social.html |website=Electronics Diary|date=27 May 2019 }}
Augmented reality also gives users the ability to practice different forms of social interactions with other people in a safe, risk-free environment. Hannes Kauffman, Associate Professor for virtual reality at TU Vienna, says: "In collaborative augmented reality multiple users may access a shared space populated by virtual objects, while remaining grounded in the real world. This technique is particularly powerful for educational purposes when users are collocated and can use natural means of communication (speech, gestures, etc.), but can also be mixed successfully with immersive VR or remote collaboration."{{quote without source|date=October 2019}} Hannes cites education as a potential use of this technology.
=Healthcare planning, practice and education=
One of the first applications of augmented reality was in healthcare, particularly to support the planning, practice, and training of surgical procedures. As far back as 1992, enhancing human performance during surgery was a formally stated objective when building the first augmented reality systems at U.S. Air Force laboratories. AR provides surgeons with patient monitoring data in the style of a fighter pilot's heads-up display, and allows patient imaging records, including functional videos, to be accessed and overlaid. Examples include a virtual X-ray view based on prior tomography or on real-time images from ultrasound and confocal microscopy probes,{{cite book |doi=10.1007/978-3-642-04268-3_60 |pmid=20426023 |chapter=Optical Biopsy Mapping for Minimally Invasive Cancer Screening |title=Medical Image Computing and Computer-Assisted Intervention – MICCAI 2009 |volume=5761 |issue=Pt 1 |pages=483–490 |series=Lecture Notes in Computer Science |year=2009 |last1=Mountney |first1=Peter |last2=Giannarou |first2=Stamatia |last3=Elson |first3=Daniel |last4=Yang |first4=Guang-Zhong |isbn=978-3-642-04267-6 }} visualizing the position of a tumor in the video of an endoscope,{{youTube|4emmCcBb4s|Scopis Augmented Reality: Path guidance to craniopharyngioma}} or radiation exposure risks from X-ray imaging devices.{{cite book |doi=10.1007/978-3-319-10404-1_52 |pmid=25333145 |chapter=3D Global Estimation and Augmented Reality Visualization of Intra-operative X-ray Dose |title=Medical Image Computing and Computer-Assisted Intervention – MICCAI 2014 |volume=8673 |issue=Pt 1 |pages=415–422 |series=Lecture Notes in Computer Science |year=2014 |last1=Loy Rodas |first1=Nicolas |last2=Padoy |first2=Nicolas |isbn=978-3-319-10403-4 |s2cid=819543 }}{{youTube|pINE2gaOVOY|3D Global Estimation and Augmented Reality Visualization of Intra-operative X-ray Dose}} AR can enhance viewing a fetus inside a mother's womb.{{cite web |url=http://www.cs.unc.edu/Research/us/ |title=UNC Ultrasound/Medical Augmented Reality Research |access-date=6 January 2010 |archive-url=https://web.archive.org/web/20100212231230/http://www.cs.unc.edu/Research/us/ |archive-date=12 February 2010 |url-status=live}} Siemens, Karl Storz and IRCAD have developed a system for laparoscopic liver surgery that uses AR to view sub-surface tumors and vessels.{{cite book |doi=10.1007/978-3-319-10404-1_53 |pmid=25333146 |chapter=An Augmented Reality Framework for Soft Tissue Surgery |title=Medical Image Computing and Computer-Assisted Intervention – MICCAI 2014 |volume=8673 |issue=Pt 1 |pages=423–431 |series=Lecture Notes in Computer Science |year=2014 |last1=Mountney |first1=Peter |last2=Fallert |first2=Johannes |last3=Nicolau |first3=Stephane |last4=Soler |first4=Luc |last5=Mewes |first5=Philip W. |isbn=978-3-319-10403-4 }}
AR has been used for cockroach phobia treatment{{cite journal |last1=Botella |first1=Cristina |last2=Bretón-López |first2=Juani |last3=Quero |first3=Soledad |last4=Baños |first4=Rosa |last5=García-Palacios |first5=Azucena |title=Treating Cockroach Phobia With Augmented Reality |journal=Behavior Therapy |date=September 2010 |volume=41 |issue=3 |pages=401–413 |doi=10.1016/j.beth.2009.07.002 |pmid=20569788 |s2cid=29889630 }} and to reduce the fear of spiders.{{Cite journal|last1=Zimmer|first1=Anja|last2=Wang|first2=Nan|last3=Ibach|first3=Merle K.|last4=Fehlmann|first4=Bernhard|last5=Schicktanz|first5=Nathalie S.|last6=Bentz|first6=Dorothée|last7=Michael|first7=Tanja|last8=Papassotiropoulos|first8=Andreas|last9=de Quervain|first9=Dominique J. F.|date=2021-08-01|title=Effectiveness of a smartphone-based, augmented reality exposure app to reduce fear of spiders in real-life: A randomized controlled trial|journal=Journal of Anxiety Disorders|language=en|volume=82|pages=102442|doi=10.1016/j.janxdis.2021.102442|pmid=34246153|s2cid=235791626|issn=0887-6185|doi-access=free}} Patients wearing augmented reality glasses can be reminded to take medications.{{cite web | url = http://www.healthtechevent.com/technology/augmented-reality-revolutionizing-medicine-healthcare/ | title = Augmented Reality Revolutionizing Medicine | publisher = Health Tech Event | access-date = 9 October 2014 | date = 6 June 2014 | archive-date = 12 October 2014 | archive-url = https://web.archive.org/web/20141012184851/http://www.healthtechevent.com/technology/augmented-reality-revolutionizing-medicine-healthcare/ | url-status = dead }} Augmented reality can be very helpful in the medical field.{{Cite journal|last=Thomas|first=Daniel J.|date=December 2016|title=Augmented reality in surgery: The Computer-Aided Medicine revolution|journal=International Journal of Surgery |volume=36|issue=Pt A|pages=25|doi=10.1016/j.ijsu.2016.10.003|issn=1743-9159|pmid=27741424|doi-access=free}} It could be used to provide crucial information to a doctor or surgeon without having them take their eyes off the patient.
On 30 April 2015, Microsoft announced the Microsoft HoloLens, their first attempt at augmented reality. The HoloLens is capable of displaying images for image-guided surgery.{{Cite book|last1=Cui|first1=Nan|last2=Kharel|first2=Pradosh|last3=Gruev|first3=Viktor|s2cid=125528534|date=8 February 2017|title=Augmented reality with Microsoft HoloLens holograms for near-infrared fluorescence based image guided surgery|publisher=International Society for Optics and Photonics|volume=10049|pages=100490I|doi=10.1117/12.2251625|series=Molecular-Guided Surgery: Molecules, Devices, and Applications III|chapter=Augmented reality with Microsoft Holo Lens holograms for near-infrared fluorescence based image guided surgery|editor1-last=Pogue|editor1-first=Brian W|editor2-last=Gioux|editor2-first=Sylvain}} As augmented reality advances, it finds increasing applications in healthcare. Augmented reality and similar computer based-utilities are being used to train medical professionals.{{cite journal |last1=Moro |first1=C |last2=Birt |first2=J |last3=Stromberga |first3=Z |last4=Phelps |first4=C |last5=Clark |first5=J |last6=Glasziou |first6=P |last7=Scott |first7=AM |title=Virtual and Augmented Reality Enhancements to Medical and Science Student Physiology and Anatomy Test Performance: A Systematic Review and Meta-Analysis. |journal=Anatomical Sciences Education |date=May 2021 |volume=14 |issue=3 |pages=368–376 |doi=10.1002/ase.2049 |pmid=33378557|s2cid=229929326 |url=https://research.bond.edu.au/en/publications/63e5a776-f3fd-48f2-b0ba-f47ca4ca96e2 }}{{Cite journal|last1=Barsom|first1=E. Z.|last2=Graafland|first2=M.|last3=Schijven|first3=M. P.|date=1 October 2016|title=Systematic review on the effectiveness of augmented reality applications in medical training|journal=Surgical Endoscopy|language=en|volume=30|issue=10|pages=4174–4183|doi=10.1007/s00464-016-4800-6|pmid=26905573|issn=0930-2794|pmc=5009168}} In healthcare, AR can be used to provide guidance during diagnostic and therapeutic interventions e.g. during surgery. Magee et al.,{{Cite journal|last1=Magee|first1=D.|last2=Zhu|first2=Y.|last3=Ratnalingam|first3=R.|last4=Gardner|first4=P.|last5=Kessel|first5=D.|date=1 October 2007|title=An augmented reality simulator for ultrasound guided needle placement training|journal=Medical & Biological Engineering & Computing|language=en|volume=45|issue=10|pages=957–967|doi=10.1007/s11517-007-0231-9|pmid=17653784|s2cid=14943048|issn=1741-0444|url=http://eprints.whiterose.ac.uk/75786/8/Combine.pdf}} for instance, describe the use of augmented reality for medical training in simulating ultrasound-guided needle placement. Recently, augmented reality began seeing adoption in neurosurgery, a field that requires heavy amounts of imaging before procedures.{{Cite journal|last1=Tagaytayan|first1=Raniel|last2=Kelemen|first2=Arpad|last3=Sik-Lanyi|first3=Cecilia|title=Augmented reality in neurosurgery|journal=Archives of Medical Science |volume=14|issue=3|pages=572–578|doi=10.5114/aoms.2016.58690|issn=1734-1922|pmc=5949895|pmid=29765445|year=2018}}
=Spatial immersion and interaction=
Augmented reality applications, running on handheld devices utilized as virtual reality headsets, can also digitize human presence in space and provide a computer generated model of them, in a virtual space where they can interact and perform various actions. Such capabilities are demonstrated by Project Anywhere, developed by a postgraduate student at ETH Zurich, which was dubbed as an "out-of-body experience".{{cite news | url=https://www.theguardian.com/technology/2015/jan/07/project-anywhere-digital-route-to-an-out-of-body-experience | title=Project Anywhere: digital route to an out-of-body experience | newspaper=The Guardian | date=7 January 2015 | access-date=21 September 2016 | author=Davis, Nicola}}{{cite web | url=http://www.euronews.com/2015/02/25/project-anywhere-an-out-of-body-experience-of-a-new-kind | title=Project Anywhere: an out-of-body experience of a new kind | work=Euronews | date=25 February 2015 | access-date=21 September 2016}}[http://www.studioany.com/#!projectanywhere/c1g1s Project Anywhere] at studioany.com
=Flight training=
Building on decades of perceptual-motor research in experimental psychology, researchers at the Aviation Research Laboratory of the University of Illinois at Urbana–Champaign used augmented reality in the form of a flight path in the sky to teach flight students how to land an airplane using a flight simulator. An adaptive augmented schedule in which students were shown the augmentation only when they departed from the flight path proved to be a more effective training intervention than a constant schedule.{{cite journal |last1=Lintern |first1=Gavan |last2=Roscoe |first2=Stanley N. |last3=Sivier |first3=Jonathan E. |title=Display Principles, Control Dynamics, and Environmental Factors in Pilot Training and Transfer |journal=Human Factors |date=June 1990 |volume=32 |issue=3 |pages=299–317 |doi=10.1177/001872089003200304 |s2cid=110528421}} Flight students taught to land in the simulator with the adaptive augmentation learned to land a light aircraft more quickly than students with the same amount of landing training in the simulator but with constant augmentation or without any augmentation.{{cite journal|last1=Lintern|first1=Gavan|title=Transfer of landing skill after training with supplementary visual cues|journal=Human Factors|date=1980|volume=22|issue=1|pages=81–88|doi=10.1177/001872088002200109|pmid=7364448|s2cid=113087380}}
=Military=
An interesting early application of AR occurred when Rockwell International created video map overlays of satellite and orbital debris tracks to aid in space observations at Air Force Maui Optical System. In their 1993 paper "Debris Correlation Using the Rockwell WorldView System" the authors describe the use of map overlays applied to video from space surveillance telescopes. The map overlays indicated the trajectories of various objects in geographic coordinates. This allowed telescope operators to identify satellites, and also to identify and catalog potentially dangerous space debris.Abernathy, M., Houchard, J., Puccetti, M., and Lambert, J,"Debris Correlation Using the Rockwell WorldView System", Proceedings of 1993 Space Surveillance Workshop 30 March to 1 April 1993, pages 189–195
Starting in 2003 the US Army integrated the SmartCam3D augmented reality system into the Shadow Unmanned Aerial System to aid sensor operators using telescopic cameras to locate people or points of interest. The system combined fixed geographic information including street names, points of interest, airports, and railroads with live video from the camera system. The system offered a "picture in picture" mode that allows it to show a synthetic view of the area surrounding the camera's field of view. This helps solve a problem in which the field of view is so narrow that it excludes important context, as if "looking through a soda straw". The system displays real-time friend/foe/neutral location markers blended with live video, providing the operator with improved situational awareness.
Researchers at USAF Research Lab (Calhoun, Draper et al.) found an approximately two-fold increase in the speed at which UAV sensor operators found points of interest using this technology.Calhoun, G. L., Draper, M. H., Abernathy, M. F., Delgado, F., and Patzek, M. "Synthetic Vision System for Improving Unmanned Aerial Vehicle Operator Situation Awareness," 2005 Proceedings of SPIE Enhanced and Synthetic Vision, Vol. 5802, pp. 219–230. This ability to maintain geographic awareness quantitatively enhances mission efficiency. The system is in use on the US Army RQ-7 Shadow and the MQ-1C Gray Eagle Unmanned Aerial Systems.
In combat, AR can serve as a networked communication system that renders useful battlefield data onto a soldier's goggles in real time. From the soldier's viewpoint, people and various objects can be marked with special indicators to warn of potential dangers. Virtual maps and 360° view camera imaging can also be rendered to aid a soldier's navigation and battlefield perspective, and this can be transmitted to military leaders at a remote command center.Cameron, Chris. [http://www.readwriteweb.com/archives/military_grade_augmented_reality_could_redefine_modern_warfare.php Military-Grade Augmented Reality Could Redefine Modern Warfare] ReadWriteWeb 11 June 2010. The combination of 360° view cameras visualization and AR can be used on board combat vehicles and tanks as circular review system.
AR can be an effective tool for virtually mapping out the 3D topologies of munition storages in the terrain, with the choice of the munitions combination in stacks and distances between them with a visualization of risk areas.{{cite news |last1=Slyusar |first1=Vadym |title=Augmented reality in the interests of ESMRM and munitions safety |date=19 July 2019 }}{{unreliable source?|date=October 2019}} The scope of AR applications also includes visualization of data from embedded munitions monitoring sensors.
=Navigation=
{{See also|Automotive navigation system}}
The NASA X-38 was flown using a hybrid synthetic vision system that overlaid map data on video to provide enhanced navigation for the spacecraft during flight tests from 1998 to 2002. It used the LandForm software which was useful for times of limited visibility, including an instance when the video camera window frosted over leaving astronauts to rely on the map overlays.Delgado, F., Abernathy, M., White J., and Lowrey, B. [http://adsabs.harvard.edu/abs/1999SPIE.3691..149D Real-Time 3-D Flight Guidance with Terrain for the X-38], SPIE Enhanced and Synthetic Vision 1999, Orlando Florida, April 1999, Proceedings of the SPIE Vol. 3691, pages 149–156 The LandForm software was also test flown at the Army Yuma Proving Ground in 1999. In the photo at right one can see the map markers indicating runways, air traffic control tower, taxiways, and hangars overlaid on the video.Delgado, F., Altman, S., Abernathy, M., White, J. [http://adsabs.harvard.edu/abs/2000SPIE.4023...63D Virtual Cockpit Window for the X-38], SPIE Enhanced and Synthetic Vision 2000, Orlando Florida, Proceedings of the SPIE Vol. 4023, pages 63–70
AR can augment the effectiveness of navigation devices. Information can be displayed on an automobile's windshield indicating destination directions and meter, weather, terrain, road conditions and traffic information as well as alerts to potential hazards in their path.[https://techcrunch.com/2010/03/17/gms-enhanced-vision-system-brings-augmented-reality-to-vehicle-huds/ GM's Enhanced Vision System]. Techcrunch.com (17 March 2010). Retrieved 9 June 2012.Couts, Andrew. [http://www.digitaltrends.com/cars/new-augmented-reality-system-shows-3d-gps-navigation-through-your-windshield/ New augmented reality system shows 3D GPS navigation through your windshield] Digital Trends,27 October 2011.Griggs, Brandon. [http://www.cnn.com/2012/01/13/tech/innovation/ces-future-driving/index.html Augmented-reality' windshields and the future of driving] CNN Tech, 13 January 2012. Since 2012, a Swiss-based company WayRay has been developing holographic AR navigation systems that use holographic optical elements for projecting all route-related information including directions, important notifications, and points of interest right into the drivers' line of sight and far ahead of the vehicle.{{Cite news|url=https://techcrunch.com/2018/01/09/wayrays-ar-in-car-hud-convinced-me-huds-can-be-better/|title=WayRay's AR in-car HUD convinced me HUDs can be better|work=TechCrunch|access-date=3 October 2018|language=en-US}}{{Cite web|url=http://www.futurecar.com/1013/WayRay-Creates-Holographic-Navigation-Alibaba-Invests-$18-Million|title=WayRay Creates Holographic Navigation: Alibaba Invests $18 Million|last=Walz|first=Eric|date=22 May 2017|website=FutureCar|access-date=2018-10-17}} Aboard maritime vessels, AR can allow bridge watch-standers to continuously monitor important information such as a ship's heading and speed while moving throughout the bridge or performing other tasks.{{cite web |url=http://cimsec.org/bridgegoggles/ |title=CIMSEC: Google's AR Goggles|author=Cheney-Peters, Scott|date=12 April 2012 |access-date=20 April 2012}}
=Workplace=
In a research project, AR was used to facilitate collaboration among distributed team members via conferences with local and virtual participants. AR tasks included brainstorming and discussion meetings utilizing common visualization via touch screen tables, interactive digital whiteboards, shared design spaces and distributed control rooms.{{cite web |url=http://www.hog3d.net/ |title=Hand of God |author1=Stafford, Aaron |author2=Piekarski, Wayne |author3=Thomas, Bruce H. |access-date=18 December 2009 |archive-url=https://web.archive.org/web/20091207022651/http://www.hog3d.net/ |archive-date=7 December 2009 |url-status=dead |df=dmy-all }}{{cite journal |last1=Benford |first1=Steve |last2=Greenhalgh |first2=Chris |last3=Reynard |first3=Gail |last4=Brown |first4=Chris |last5=Koleva |first5=Boriana |s2cid=672378 |title=Understanding and constructing shared spaces with mixed-reality boundaries |journal=ACM Transactions on Computer-Human Interaction |date=1 September 1998 |volume=5 |issue=3 |pages=185–223 |doi=10.1145/292834.292836 }}[http://mi-lab.org/projects/office-of-tomorrow/ Office of Tomorrow] Media Interaction Lab.
In industrial environments, augmented reality is proving to have a substantial impact with use cases emerging across all aspect of the product lifecycle, starting from product design and new product introduction (NPI) to manufacturing to service and maintenance, to material handling and distribution. For example, labels were displayed on parts of a system to clarify operating instructions for a mechanic performing maintenance on a system.[https://web.archive.org/web/20110511082745/http://ngm.nationalgeographic.com/big-idea/14/augmented-reality-pg1 The big idea:Augmented Reality]. Ngm.nationalgeographic.com (15 May 2012). Retrieved 9 June 2012.{{cite web |url=http://graphics.cs.columbia.edu/projects/armar/ |title=Augmented Reality for Maintenance and Repair (ARMAR) |author1=Henderson, Steve |author2=Feiner, Steven |access-date=6 January 2010 |archive-date=6 March 2010 |archive-url=https://web.archive.org/web/20100306202422/http://graphics.cs.columbia.edu/projects/armar/ |url-status=dead }} Assembly lines benefited from the usage of AR. In addition to Boeing, BMW and Volkswagen were known for incorporating this technology into assembly lines for monitoring process improvements.Sandgren, Jeffrey. [http://brandtechnews.net/tag/augmented-reality/ The Augmented Eye of the Beholder] {{Webarchive|url=https://web.archive.org/web/20130621054848/http://brandtechnews.net/tag/augmented-reality/ |date=21 June 2013 }}, BrandTech News 8 January 2011.Cameron, Chris. [http://www.slideshare.net/readwriteweb/augmented-reality-for-marketers-and-developers-analysis-of-the-leaders-the-challenges-and-the-future Augmented Reality for Marketers and Developers], ReadWriteWeb.Dillow, Clay [http://www.popsci.com/scitech/article/2009-09/bmw-developing-augmented-reality-help-mechanics BMW Augmented Reality Glasses Help Average Joes Make Repairs], Popular Science September 2009. Big machines are difficult to maintain because of their multiple layers or structures. AR permits people to look through the machine as if with an x-ray, pointing them to the problem right away.King, Rachael. [https://web.archive.org/web/20120704074014/http://www.businessweek.com/stories/2009-11-03/augmented-reality-goes-mobilebusinessweek-business-news-stock-market-and-financial-advice Augmented Reality Goes Mobile], Bloomberg Business Week Technology 3 November 2009.
As AR technology has progressed, the impact of AR in enterprise has grown. In the Harvard Business Review, Magid Abraham and Marco Annunziata discussed how AR devices are now being used to "boost workers' productivity on an array of tasks the first time they're used, even without prior training".{{Cite journal|url=https://hbr.org/2017/03/augmented-reality-is-already-improving-worker-performance|title=Augmented Reality Is Already Improving Worker Performance|last1=Abraham|first1=Magid|last2=Annunziata|first2=Marco|date=13 March 2017|journal=Harvard Business Review|access-date=13 January 2019}} They contend that "these technologies increase productivity by making workers more skilled and efficient, and thus have the potential to yield both more economic growth and better jobs".
=Broadcast and live events=
Weather visualizations were the first application of augmented reality in television. It has now become common in weather casting to display full motion video of images captured in real-time from multiple cameras and other imaging devices. Coupled with 3D graphics symbols and mapped to a common virtual geospatial model, these animated visualizations constitute the first true application of AR to TV.
AR has become common in sports telecasting. Sports and entertainment venues are provided with see-through and overlay augmentation through tracked camera feeds for enhanced viewing by the audience. Examples include the yellow "first down" line seen in television broadcasts of American football games showing the line the offensive team must cross to receive a first down. AR is also used in association with football and other sporting events to show commercial advertisements overlaid onto the view of the playing area. Sections of rugby fields and cricket pitches also display sponsored images. Swimming telecasts often add a line across the lanes to indicate the position of the current record holder as a race proceeds to allow viewers to compare the current race to the best performance. Other examples include hockey puck tracking and annotations of racing car performanceArchived at [https://ghostarchive.org/varchive/youtube/20211211/1jQUkqqnZIc Ghostarchive]{{cbignore}} and the [https://web.archive.org/web/20210714184600/https://www.youtube.com/watch?v=1jQUkqqnZIc Wayback Machine]{{cbignore}}: {{Citation|title=Arti AR highlights at SRX -- the first sports augmented reality live from a moving car!| date=14 July 2021 |url=https://www.youtube.com/watch?v=1jQUkqqnZIc|language=en|access-date=2021-07-14}}{{cbignore}} and snooker ball trajectories.Azuma, Ronald; Balliot, Yohan; Behringer, Reinhold; Feiner, Steven; Julier, Simon; MacIntyre, Blair. [http://www.cc.gatech.edu/~blair/papers/ARsurveyCGA.pdf Recent Advances in Augmented Reality] Computers & Graphics, November 2001.Marlow, Chris. [http://www.dmwmedia.com/news/2012/04/27/hey-hockey-puck-nhl-preplay-adds-a-second-screen-experience-to-live-games Hey, hockey puck! NHL PrePlay adds a second-screen experience to live games], digitalmediawire 27 April 2012.
AR has been used to enhance concert and theater performances. For example, artists allow listeners to augment their listening experience by adding their performance to that of other bands/groups of users.{{cite book |doi=10.1109/ART.2002.1107010 |chapter=The Duran Duran project: The augmented reality toolkit in live performance |title=The First IEEE International Workshop Agumented Reality Toolkit |pages=2 |year=2002 |last1=Pair |first1=J. |last2=Wilson |first2=J. |last3=Chastine |first3=J. |last4=Gandy |first4=M. |s2cid=55820154 |isbn=0-7803-7680-3 }}Broughall, Nick. [http://www.gizmodo.com.au/2009/10/sydney-band-uses-augmented-reality-for-video-clip/ Sydney Band Uses Augmented Reality For Video Clip.] Gizmodo, 19 October 2009.Pendlebury, Ty. [http://www.cnet.com.au/augmented-reality-in-aussie-film-clip-339299097.htm Augmented reality in Aussie film clip]. c|net 19 October 2009.
=Tourism and sightseeing=
Travelers may use AR to access real-time informational displays regarding a location, its features, and comments or content provided by previous visitors. Advanced AR applications include simulations of historical events, places, and objects rendered into the landscape.Saenz, Aaron [http://singularityhub.com/2009/11/19/augmented-reality-does-time-travel-tourism/ Augmented Reality Does Time Travel Tourism] SingularityHUB 19 November 2009.Sung, Dan [http://www.pocket-lint.com/news/38806/augmented-reality-travel-tourism-apps Augmented reality in action – travel and tourism] Pocket-lint 2 March 2011.Dawson, Jim [http://www.livescience.com/5644-augmented-reality-reveals-history-tourists.html Augmented Reality Reveals History to Tourists] Life Science 16 August 2009.
AR applications linked to geographic locations present location information by audio, announcing features of interest at a particular site as they become visible to the user.{{Cite journal | doi=10.1111/j.1467-9671.2006.00244.x| title=Development of a Speech-Based Augmented Reality System to Support Exploration of Cityscape| journal=Transactions in GIS| volume=10| pages=63–86| year=2006| last1=Bartie| first1=Phil J.| last2=MacKaness| first2=William A.| issue=1| bibcode=2006TrGIS..10...63B| s2cid=13325561}}Benderson, Benjamin B. [http://www.cs.umd.edu/~bederson/papers/chi-95-aar/ Audio Augmented Reality: A Prototype Automated Tour Guide] {{webarchive |url=https://web.archive.org/web/20020701071038/http://www.cs.umd.edu/~bederson/papers/chi-95-aar/ |date=1 July 2002 }} Bell Communications Research, ACM Human Computer in Computing Systems Conference, pp. 210–211.Jain, Puneet and Manweiler, Justin and Roy Choudhury, Romit. [http://synrg.csl.illinois.edu/papers/overlay.pdf OverLay: Practical Mobile Augmented Reality]
ACM MobiSys, May 2015.
=Translation=
AR applications such as Word Lens can interpret the foreign text on signs and menus and, in a user's augmented view, re-display the text in the user's language. Spoken words of a foreign language can be translated and displayed in a user's view as printed subtitles.Tsotsis, Alexia. [https://techcrunch.com/2010/12/16/world-lens-translates-words-inside-of-images-yes-really Word Lens Translates Words Inside of Images. Yes Really.] TechCrunch (16 December 2010).N.B. [https://www.economist.com/blogs/gulliver/2010/12/instant_translation Word Lens: This changes everything] The Economist: Gulliver blog 18 December 2010.Borghino, Dario [http://www.gizmag.com/language-translating-glasses/23494/ Augmented reality glasses perform real-time language translation]. gizmag, 29 July 2012.
=Music=
It has been suggested that augmented reality may be used in new methods of music production, mixing, control and visualization.{{cite web|title=Music Production in the Era of Augmented Reality|url=https://medium.com/@Soundspringstudio/music-production-in-the-era-of-augmented-reality-2e79f4926275|website=Medium|access-date=5 January 2017|date=14 October 2016}}{{cite web|title=Augmented Reality music making with Oak on Kickstarter – gearnews.com|url=https://www.gearnews.com/augmented-reality-music-making-oak-kickstarter/|website=gearnews.com|access-date=5 January 2017|date=3 November 2016}}{{cite web|last1=Clouth|first1=Robert|title=Mobile Augmented Reality as a Control Mode for Real-time Music Systems|url=http://mtg.upf.edu/node/2846|access-date=5 January 2017|date=1 January 2013}}{{cite book |doi=10.1109/ICICS.2007.4449564 |chapter=A multimodal augmented reality DJ music system |title=2007 6th International Conference on Information, Communications & Signal Processing |pages=1–5 |year=2007 |last1=Farbiz |first1=Farzam |last2=Tang |first2=Ka Yin |last3=Wang |first3=Kejian |last4=Ahmad |first4=Waqas |last5=Manders |first5=Corey |last6=Jyh Herng |first6=Chong |last7=Kee Tan |first7=Yeow |s2cid=17807179 |isbn=978-1-4244-0982-2 }}
In a proof-of-concept project Ian Sterling, an interaction design student at California College of the Arts, and software engineer Swaroop Pal demonstrated a HoloLens app whose primary purpose is to provide a 3D spatial UI for cross-platform devices—the Android Music Player app and Arduino-controlled Fan and Light—and also allow interaction using gaze and gesture control.{{cite web|title=HoloLens concept lets you control your smart home via augmented reality|url=http://www.digitaltrends.com/cool-tech/hololens-hackathon-smart-home/|publisher=Digital Trends|access-date=5 January 2017|date=26 July 2016}}{{cite web|title=Hololens: Entwickler zeigt räumliches Interface für Elektrogeräte|url=https://mixed.de/hololens-entwickler-zeigt-raeumliches-interface-fuer-elektrogeraete/|publisher=MIXED|access-date=5 January 2017|language=de-DE|date=22 July 2016}}{{cite web|title=Control Your IoT Smart Devices Using Microsoft HoloLen (video) – Geeky Gadgets|url=http://www.geeky-gadgets.com/control-your-iot-smart-devices-using-microsoft-hololen-27-07-2016/|publisher=Geeky Gadgets|access-date=5 January 2017|date=27 July 2016}}{{cite web|title=Experimental app brings smart home controls into augmented reality with HoloLens|url=http://www.windowscentral.com/experimental-app-brings-smart-home-controls-augmented-reality-hololens|publisher=Windows Central|access-date=5 January 2017|date=22 July 2016}}
Apps
Snapchat users have access to augmented reality features. In September 2017, Snapchat announced a feature called "Sky Filters" that will be available on its app. This new feature makes use of augmented reality to alter the look of a picture taken of the sky, much like how users can apply the app's filters to other pictures. Users can choose from sky filters such as starry night, stormy clouds, beautiful sunsets, and rainbow.Miller, Chance. "Snapchat's Latest Augmented Reality Feature Lets You Paint the Sky with New Filters." 9to5Mac, 9to5Mac, 25 Sept. 2017, 9to5mac.com/2017/09/25/how-to-use-snapchat-sky-filters/.
Google launched an augmented reality feature for Google Maps on Pixel phones that identifies users' location and places signs and arrows on the device screen to show a user navigation directions.{{cite web | last=Bastone | first=Nick | title=We tried Google’s new augmented reality feature for Maps that’s currently available only on its Pixel smartphones, and we don’t know what we’d do without it. | website=Business Insider | date=2019-05-31 | url=https://www.businessinsider.com/we-tried-new-google-maps-augmented-reality-feature-2019-2 | access-date=2025-04-21}}
Concerns
=Reality modifications=
In a paper titled "Death by Pokémon GO", researchers at Purdue University's Krannert School of Management claim the game caused "a disproportionate increase in vehicular crashes and associated vehicular damage, personal injuries, and fatalities in the vicinity of locations, called PokéStops, where users can play the game while driving."{{cite news |last1=Faccio |first1=Mara |last2=McConnell |first2=John J. |title=Death by Pokémon GO |date=2017 |doi=10.2139/ssrn.3073723 |ssrn=3073723 }} Using data from one municipality, the paper extrapolates what that might mean nationwide and concluded "the increase in crashes attributable to the introduction of Pokémon GO is 145,632 with an associated increase in the number of injuries of 29,370 and an associated increase in the number of fatalities of 256 over the period of 6 July 2016, through 30 November 2016." The authors extrapolated the cost of those crashes and fatalities at between $2bn and $7.3 billion for the same period. Furthermore, more than one in three surveyed advanced Internet users would like to edit out disturbing elements around them, such as garbage or graffiti.Peddie, J., 2017, Agumented Reality, Springer{{page needed|date=October 2019}} They would like to even modify their surroundings by erasing street signs, billboard ads, and uninteresting shopping windows. Consumers want to use augmented reality glasses to change their surroundings into something that reflects their own personal opinions. Around two in five want to change the way their surroundings look and even how people appear to them. {{Citation needed|date=September 2020}}
=Privacy concerns=
Augmented reality devices that use cameras for 3D tracking or video passthrough depend on the ability of the device to record and analyze the environment in real time. Because of this, there are potential legal concerns over privacy.
In late 2024, Meta's collaboration with Ray-Ban on smart glasses faced heightened scrutiny due to significant privacy concerns. A notable incident involved two Harvard students who developed a program named I-XRAY, which utilized the glasses' camera in conjunction with facial recognition software to identify individuals in real-time.{{Cite web |last=Song |first=Victoria |date=2024-10-02 |title=College students used Meta's smart glasses to dox people in real time |url=https://www.theverge.com/2024/10/2/24260262/ray-ban-meta-smart-glasses-doxxing-privacy |access-date=2025-01-27 |website=The Verge |language=en}}
According to recent studies, users are especially concerned that augmented reality smart glasses might compromise the privacy of others, potentially causing peers to become uncomfortable or less open during interactions.{{Cite journal |last1=Rauschnabel |first1=Philipp A. |last2=He |first2=Jun |last3=Ro |first3=Young K. |date=2018-11-01 |title=Antecedents to the adoption of augmented reality smart glasses: A closer look at privacy risks |url=https://linkinghub.elsevier.com/retrieve/pii/S0148296318303849 |journal=Journal of Business Research |volume=92 |pages=374–384 |doi=10.1016/j.jbusres.2018.08.008 |issn=0148-2963}}
While the First Amendment to the United States Constitution allows for such recording in the name of public interest, the constant recording of an AR device makes it difficult to do so without also recording outside of the public domain. Legal complications would be found in areas where a right to a certain amount of privacy is expected or where copyrighted media are displayed.
In terms of individual privacy, there exists the ease of access to information that one should not readily possess about a given person. This is accomplished through facial recognition technology. Assuming that AR automatically passes information about persons that the user sees, there could be anything seen from social media, criminal record, and marital status.{{cite book |doi=10.1145/2638728.2641709 |chapter=Augmented reality |title=Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct Publication - UbiComp '14 Adjunct |pages=1283–1288 |year=2014 |last1=Roesner |first1=Franziska |last2=Kohno |first2=Tadayoshi |last3=Denning |first3=Tamara |last4=Calo |first4=Ryan |last5=Newell |first5=Bryce Clayton |s2cid=15190154 |isbn=978-1-4503-3047-3 }}
Notable researchers
- Ronald Azuma is a scientist and author of works on AR.
- Jeri Ellsworth headed a research effort for Valve on augmented reality (AR), later taking that research to her own start-up CastAR. The company, founded in 2013, eventually shuttered. Later, she created another start-up based on the same technology called Tilt Five; another AR start-up formed by her with the purpose of creating a device for digital board games.{{Cite news|url=https://www.nytimes.com/2019/10/24/technology/jeri-ellsworth-augmented-reality.html|title=Always Building, From the Garage to Her Company|last=Markoff|first=John|date=2019-10-24|work=The New York Times|access-date=2019-12-12|language=en-US|issn=0362-4331}}
- Steve Mann formulated an earlier concept of mediated reality in the 1970s and 1980s, using cameras, processors, and display systems to modify visual reality to help people see better (dynamic range management), building computerized welding helmets, as well as "augmediated reality" vision systems for use in everyday life. He is also an adviser to Meta.{{cite journal |last1=Mann |first1=S. |title=Wearable computing: a first step toward personal imaging |journal=Computer |date=1997 |volume=30 |issue=2 |pages=25–32 |doi=10.1109/2.566147 |s2cid=28001657 }}
- Dieter Schmalstieg and Daniel Wagner developed a marker tracking systems for mobile phones and PDAs in 2009.{{cite book |url=http://portal.acm.org/citation.cfm?id=946910 |title=First Steps Towards Handheld Augmented Reality |author=Wagner, Daniel |date=29 September 2009 |publisher=ACM |access-date=29 September 2009|isbn=9780769520346 }}
- Ivan Sutherland invented the first VR head-mounted display at Harvard University.
In media
The futuristic short film Sight{{Cite web |url=https://vimeo.com/46304267 |title=Sight |last=Robot Genius |website=vimeo.com |access-date=18 June 2019|date=24 July 2012 }} features contact lens-like augmented reality devices.{{cite magazine|last1=Kosner|first1=Anthony Wing|title=Sight: An 8-Minute Augmented Reality Journey That Makes Google Glass Look Tame|url=https://www.forbes.com/sites/anthonykosner/2012/07/29/sight-an-8-minute-augmented-reality-journey-that-makes-google-glass-look-tame/|magazine=Forbes|access-date=3 August 2015|date=29 July 2012}}{{cite web|last1=O'Dell|first1=J.|title=Beautiful short film shows a frightening future filled with Google Glass-like devices|url=https://venturebeat.com/2012/07/27/sight-systems/|access-date=3 August 2015|date=27 July 2012}}
See also
{{div col}}
- {{annotated link|ARTag}}
- {{annotated link|Augmented reality-based testing}}
- {{annotated link|WebAR}}
- {{annotated link|Automotive head-up display}}
- {{annotated link|Bionic contact lens}}
- {{annotated link|Computer-mediated reality}}
- {{annotated link|Cyborg}}
- {{annotated link|Holography}}
- {{annotated link|List of augmented reality software}}
- {{annotated link|Location-based service}}
- {{annotated link|Optical head-mounted display}}
- {{annotated link|Simulated reality}}
- {{annotated link|Visuo-haptic mixed reality}}
- {{annotated link|Wearable computer}}
{{div col end}}
References
{{Reflist}}
External links
{{Commons category-inline}}
{{Extended reality|state=collapsed}}
{{Authority control}}
Category:Applications of computer vision