Virtual keyboard

On a desktop computer, a virtual keyboard might provide an alternative input mechanism for users with disabilities who cannot use a conventional keyboard, for multi-lingual users who switch frequently between different character sets or alphabets, which may be confusing over time, or for users who are lacking a traditional keyboard.

Virtual keyboards may utilize the following:

• Virtual keyboards with touchscreen layouts or sensors{{patent|US|4725694|application|Auer Carol M, Castagno Daniel L/AT&T: "Computer interface device" priority date 13.05.1986}}
• Character variants, punctuation, and other special characters accessible through a menu, key/mouse combinations, or double/triple/long presses on sensors
• Number pad feature to facilitate typing numbers.[https://www.mobiletechreview.com/phones/Samsung-Omnia-i900-page2.htm Samsung Omnia i900, Reviewed August 21, 2008 by Lisa Gade: "There's even a virtual number pad for entering text T9-style, for those who are accustomed to texting on traditional mobile phones."]
• Optically projected keyboard layouts or similar arrangements of "keys" or sensing areas{{patent|DE|19734511|application|B. Kämmerer, C, Maggioni, H. Röttger/SIEMENS AG: "Kommunikationseinrichtung" filing date 08.08.1997}}{{patent|WO|0003348|C. Maggioni, B. Kämmerer/SIEMENS AG: "Projection Device / Vorrichtung zur Projektion" priority date 10.07.1998}}
• Optically detected human hand and finger motions{{patent |EP |0554492 |Hans E. Korth: "Method and device for optical input of commands or data" filing date 07.02.1992}}
• Multiple language sets that don't require a settings change

Various JavaScript virtual keyboards have been created on web browsers, allowing users to type their own languages on foreign keyboards. Multitouch screens allow the creation of virtual chorded keyboards for tablet computers,{{cite web|last=Chien-Hsu Chen|title=Design and Evaluation of Chord Keyboard for Tablet Computer Manipulation|url=https://www.researchgate.net/publication/242210707|publisher=National Cheng Kung University}} touchscreens, touchpads, and wired gloves.{{cite web|url=http://www.gizmag.com/asetniop-chorded-keyboard-concept/24477/|title=ASETNIOP concept designed to make touch typing on a touchscreen a mite easier|date=9 October 2012 |publisher=Gizmag.com}}{{cite web|url=http://www.todaysiphone.com/2012/10/asetniop-invisible-keyboard-aims-to-get-you-touch-typing-on-the-ipad/|title=ASETNIOP invisible keyboard aims to get you touch-typing on the iPad|date=19 October 2022 |publisher=todaysiphone.com}}

Virtual keyboards are commonly used as an on-screen input method in devices with no physical keyboard where there is no room for one, such as a pocket computer, personal digital assistant (PDA), tablet computer, or touchscreen-equipped mobile phone. Text is commonly inputted either by tapping a virtual keyboard or finger-tracing.{{cite web|url=https://www.engadget.com/2008/09/11/swype-finger-tracing-text-entry-seems-fast-hypnotic-and-magica/|title=Swype finger-tracing text entry seems fast, hypnotic, and magical|website=Engadget|date=11 September 2008 |access-date=2016-03-24}} Virtual keyboards are also featured in emulation software for systems that have fewer buttons than a computer keyboard would have.

File:Keyboard on a pocket-PC.jpg PDA]]

The four main approaches to enter text into a PDA were: virtual keyboards operated by a stylus, external USB keyboards, handwritten keyboards, and stroke recognition. Microsoft's mobile operating system approach was to simulate a completely functional keyboard, resulting in an overloaded layout.{{Cite book|url=https://www.researchgate.net/publication/220795451|title=Mobile application for determination of users' text entry speed|last1=Georgiev|first1=Tsvetozar|last2=Georgieva|first2=Evgenia|date=2008-06-12}} Without support for multi-touch technology, PDA vitural keyboards had usability constraints.

When Apple presented the iPhone in 2007, not including a physical keyboard was seen as a detriment.{{Cite news|url=https://www.telegraph.co.uk/technology/2017/01/09/world-reacted-first-iphone-10-years-ago/|title=How the world reacted to the first iPhone 10 years ago|last=Titcomb|first=James|date=2017|work=The Telegraph|access-date=2018-03-16|language=en-GB|issn=0307-1235}} However, Apple brought the multi-touch technology into the device, overcoming the usability problems of PDAs.

The most common mobile operating systems, Android and iOS, give the developer community the possibility to individually develop custom virtual keyboards.

File:Android Keyboard Settings Menue.png

The Android SDK provides an "InputMethodService".{{cite web|url=https://developer.android.com/reference/android/inputmethodservice/InputMethodService.html|title=InputMethodService {{!}} Android Developers|website=developer.android.com|language=en|access-date=2018-03-19}} This service provides a standard implementation of an input method, enabling the Android development community to implement their own keyboard layouts. The InputMethodService ships with it on Keyboard View.{{Cite news|url=https://code.tutsplus.com/tutorials/create-a-custom-keyboard-on-android--cms-22615|title=Create a Custom Keyboard on Android|work=Code Envato Tuts+|access-date=2018-03-19}} While the InputMethodService can be used to customize key and gesture inputs, the Keyboard Class loads an XML description of a keyboard and stores the attributes of the keys.{{cite web|url=https://developer.android.com/reference/android/inputmethodservice/Keyboard.html|title=Keyboard {{!}} Android Developers|website=developer.android.com|language=en|access-date=2018-03-19}}

As a result, it is possible to install different keyboard versions on an Android device, and the keyboard is only an application, most frequently downloaded among them being Gboard and SwiftKey; a simple activation over the Android settings menu is possible.{{cite web|url=https://play.google.com/store/search?q=keyboard|title=keyboard - Google Play|language=en-US|access-date=2018-03-19}}

{{clear|right}}

File:Apple iOS UIInputViewController.jpg

Apple's iOS operating system allows the development of custom keyboards, however no access is given to the dictionary or general keyboard settings. iOS automatically switches between system and custom keyboards if the user enters text into the text input field.{{cite web|url=https://developer.apple.com/library/content/documentation/StringsTextFonts/Conceptual/TextAndWebiPhoneOS/KeyboardManagement/KeyboardManagement.html|title=Managing the Keyboard|website=developer.apple.com|language=en|access-date=2018-03-19}}{{cite web|url=https://developer.apple.com/documentation/uikit/uiinputviewcontroller|title=UIInputViewController - UIKit {{!}} Apple Developer Documentation|website=developer.apple.com|language=en|access-date=2018-03-19}}

The UIInputViewController is the primary view controller for a custom keyboard app extension. This controller provides different methods for the implementation of a custom keyboard, such as a user interface for a custom keyboard, obtaining a supplementary lexicon or changing the primary language of a custom keyboard.{{cite web|url=https://developer.apple.com/library/content/documentation/General/Conceptual/ExtensibilityPG/CustomKeyboard.html|title=App Extension Programming Guide: Custom Keyboard|website=developer.apple.com|language=en|access-date=2018-03-19}}

Microsoft Windows provides the virtual keyboard through Common Text Framework service.

Diverse scientific papers at the beginning of the 2000s showed even before the invention of smartphones, that predicting words, based on what the user is typing, assisted in increasing the typing speed.{{Cite journal|last=Tom Stocky, Alexander Faaborg, Henry Lieberman|date=2004|title=A Commonsense Approach to Predictive Text Entry}}{{Cite journal|last=Dunlop, Mark D. and Crossan, Andrew|date=2000|title=Predictive text entry methods for mobile phones}} At the beginning of development of this keyboard feature, prediction was mainly based on static dictionaries. Google implemented the predicting method in 2013 in Android 4.4. This development was mainly driven by third party keyboard providers, such as SwiftKey and Swype.{{Cite news|url=https://www.macworld.co.uk/how-to/iphone/using-quicktype-swype-in-ios-8-3534355/|title=How to use new QuickType and Swype keyboard options in iOS 8|last=Hattersley|first=Lucy|work=Macworld UK|access-date=2018-03-19|language=en-GB}} In 2014 Apple presented iOS 8{{Cite news|url=https://www.cnet.com/how-to/using-ios-8s-quicktype-keyboard/|title=Using iOS 8's QuickType keyboard|date=2014-09-19|work=CNET|access-date=2018-03-19|language=en}} which includes a new predictive typing feature called Quick Type, which displays word predictions above the keyboard as the user types.

Haptic feedback provides for tactile confirmation that a key has been successfully triggered i.e. the user hears and feels a "click" as a key is pressed. Utilizing hysteresis, the feel of a physical key can be emulated to an even greater degree. In this case, there is an initial "click" that is heard and felt as the virtual key is pressed down, but then as finger pressure is reduced once the key is triggered, there is a further "unclick" sound and sensation as if a physical key is respringing back to its original unclicked state. This behaviour is explained in Aleks Oniszczak & Scott Mackenzie's 2004 paper "A Comparison of Two Input Methods for Keypads on Mobile Devices" which first introduced haptic feedback with hysteresis on a virtual keyboard.{{cite web|url=https://www.yorku.ca/mack/nordichi2004.pdf|title=CHI: A Comparison of Two Input Methods for Keypads on Mobile Devices|website=developer.apple.com|last=Oniszczak|first=Aleks|language=en}}

Keyboards are needed in different digital areas. smartphones and devices that create virtual worlds, for example, virtual reality or augmented reality glasses, need to provide text input possibilities.

An optical virtual keyboard was invented and patented by IBM engineers in 1992.{{patent|EP|0554492|Hans E. Korth: "Method and device for optical input of commands or data" filing date 07.02.1992}} It optically detects and analyses human hand and finger motions and interprets them as operations on a physically non-existent input device, such as a surface having painted keys. This allows it to emulate unlimited types of manually operated input devices including mouse or keyboard. All mechanical input units can be replaced by such virtual devices, optimized for the current application and for the user's physiology maintaining the speed, simplicity, and unambiguity of manual data input.

One example of this technology is the "Selfie Type" - a keyboard technology for mobile phones made by Samsung Electronics. It was intended to use the front-facing camera (the selfie camera) to track the user's fingers, enabling the user to type on an "invisible keyboard" on a table or another surface in front of the phone.{{Cite news|url=https://www.bbc.com/news/av/technology-51057261/ces-2020-samsung-s-invisible-keyboard-for-smartphones|title=The invisible keyboard for smartphones|work=BBC News |language=en|access-date=2020-01-13}}{{Cite web|url=http://theindependent.sg/samsung-introduces-selfietype-an-ai-powered-virtual-keyboard/|title=Samsung introduces SelfieType, an AI-powered virtual keyboard – The Independent News|date=11 January 2020 |language=en-US|access-date=2020-01-13}} It was introduced at the Consumer Electronics Show 2020{{Cite web|url=https://news.samsung.com/global/samsung-electronics-to-showcase-successful-c-lab-inside-projects-and-c-lab-outside-start-ups-at-ces-2020|title=Samsung Electronics to Showcase Successful 'C-Lab Inside' Projects and 'C-Lab Outside' Start-ups at CES 2020|website=news.samsung.com|language=en|access-date=2020-01-13}}{{Cite web|url=https://www.androidauthority.com/samsung-selfie-type-demo-video-1071955/|title=Here's how Samsung's crazy invisible keyboard tech will work|date=2020-01-06|website=Android Authority|language=en-US|access-date=2020-01-13}}{{Cite web|url=https://www.digitaltrends.com/computing/samsung-selfie-type-invisible-keyboard-ces-2020/|title=Samsung Selfie Type|last=|first=|date=6 January 2020|website=www.digitaltrends.com|archive-url=|archive-date=|access-date=2020-01-13}} and was expected to be launched in the same year but never did.

The basic idea of a virtual keyboard in an augmented reality environment is to give the user a text input possibility. A common approach is to render a flat keyboard into augmented reality, e.g. using the Unity TouchScreenKeyboard. The Microsoft HoloLens enables the user to point at letters on the keyboard by moving his head.{{cite web|url=https://developer.microsoft.com/en-us/windows/mixed-reality/keyboard_input_in_unity|title=Keyboard input in Unity|website=developer.microsoft.com|language=en-us|access-date=2018-03-16}}

Another approach was researched by the Korean KJIST U-VR Lab in 2003. Their suggestion was to use wearables to track the finger motion to replace a physical keyboard with virtual ones. They also tried to give audiovisual feedback to the user, when a key got hit. The basic idea was to give the user a more natural way to enter text, based on what he is used to.{{Cite book|url=https://www.researchgate.net/publication/220984297|title=ARKB: 3D vision-based Augmented Reality Keyboard.|last1=Lee|first1=Minkyung|last2=Woo|first2=Woontack|date=2003-01-01}}

The Magic Leap 1 from Magic Leap implements a virtual keyboard with augmented reality.{{Cite web |title=Text Entry |date=23 January 2025 |url=https://ml1-developer.magicleap.com/en-us/learn/guides/design-text-entry}}

The challenge, as in augmented reality, is to give the user the possibility to enter text in a completely virtual environment. Most augmented reality systems don't track the hands of the user. So many available systems provide the possibility to point at letters.{{Cite journal|last=Poupyrev|first=Ivan|title=Virtual Notepad: Handwriting in Immersive VR|url=http://www8.cs.umu.se/kurser/TDBD12/HT02/papers/virtual%20notepad.pdf}}

In September 2016, Google released a virtual keyboard app for their Daydream{{Citation|title=Daydream Keyboard|date=2018-02-21|url=https://play.google.com/store/apps/details?id=com.google.android.vr.inputmethod|publisher=Google LLC|access-date=2018-03-16}} virtual reality headset. To enter text, the user points at letters using the controller.{{Cite news|url=https://haptic.al/daydream-keyboard-8ae6b1058b3b|title=Google launches virtual keyboard for Daydream-ready phones|date=2016-10-19|work=Haptical|access-date=2018-03-16}}

In February 2017, Logitech presented an experimental approach to bring their keyboards into the virtual environment. The Vive Tracker and the Logitech G gaming keyboard track finger movement without wearing a glove. Fifty kits were sent to exclusive developers, enabling them, in combination with Logitech's BRIDGE developers kit, to test and experiment with the new technology.{{cite web|url=https://blog.vive.com/us/2017/11/02/introducing-the-logitech-bridge-sdk/|title=Introducing the Logitech BRIDGE SDK - VIVE Blog|website=blog.vive.com|access-date=2018-03-16}}{{cite web|url=https://techcrunch.com/2017/11/02/logitech-is-experimenting-with-a-keyboard-built-for-virtual-reality/|title=Logitech is experimenting with a keyboard built for virtual reality – TechCrunch|website=techcrunch.com|date=2 November 2017 |language=en-US|access-date=2018-03-16}}

Virtual keyboards may be used in some cases to reduce the risk of keystroke logging.{{cite web|title=Virtual keyboard|url=https://www.psbonline.co.in/BankAwayRetail/(S(3izwzr554kvmhxbgks11su55))/web/L001/retail/jsp/user/vir_keyb.html}} For example, Westpac's online banking service uses a virtual keyboard for password entry, as does TreasuryDirect (see picture). It is more difficult for malware to monitor the display and mouse to obtain the data entered via the virtual keyboard than it is to monitor real keystrokes. However, it is possible, for example by recording screenshots at regular intervals or upon each mouse click.Lake Software: [http://cnt.lakefolks.com/keylogger-no.htm Does not defeat keyloggers!]{{Citation

| last = Smith

| first = David A.

| title = Outsmarting Keyloggers

| publisher = PC Magazine

| date = 2006-06-21

| url = https://www.pcmag.com/article2/0,2817,1978513,00.asp

| access-date = 2009-11-16}}

Image:TreasuryDirectLoginScreen.PNG

Virtual keyboards may prevent keystroke inference attacks but can also act as keyloggers through telemetry and can leak sensitive information through text suggestions.{{Cite web |last=Carman |first=Ashley |date=2016-07-29 |title=SwiftKey bug leaked emails and other personal information |url=https://www.theverge.com/2016/7/29/12326152/swiftkey-bug-backup-sync-down-error-prediction |access-date=2025-03-02 |website=The Verge |language=en-US}}

The use of an on-screen keyboard on which the user "types" with mouse clicks can increase the risk of password disclosure by shoulder surfing, because:

• An observer can typically watch the screen more easily (and less suspiciously) than the keyboard, and see which characters the mouse moves to.
• Some implementations of the on-screen keyboard may give visual feedback of the "key" clicked, e.g. by changing its color briefly. This makes it much easier for an observer to read the data from the screen. In the worst case, the implementation may leave the focus on the most recently clicked "key" until the next virtual key is clicked, thus allowing the observer time to read each character even after the mouse starts moving to the next character.
• A user may not be able to "point and click" as fast as they could type on a keyboard, thus making it easier for the observer.

• Caldera SoftKeyboards (1997)
• Ease of Access
• Finger Touching Cell Phone
• Input method
• Mouse keys
• Multi-touch

{{reflist|colwidth=30em}}

{{Commons category|Virtual keyboards}}

Category:Assistive technology

Category:Computer keyboard types

Category:Pointing-device text input

Category:Touch user interfaces