{"id":34178,"date":"2024-10-03T21:20:04","date_gmt":"2024-10-03T10:20:04","guid":{"rendered":"https:\/\/www.kaspersky.com.au\/blog\/?p=34178"},"modified":"2024-10-04T21:22:13","modified_gmt":"2024-10-04T10:22:13","slug":"gazeploit-how-to-steal-passwords-apple-vision-pro","status":"publish","type":"post","link":"https:\/\/www.kaspersky.com.au\/blog\/gazeploit-how-to-steal-passwords-apple-vision-pro\/34178\/","title":{"rendered":"How to snoop on Apple Vision Pro user passwords"},"content":{"rendered":"

In September 2024, a team of researchers from both the University of Florida and Texas Tech University presented a paper<\/a> detailing a rather sophisticated method for intercepting text entered by users of the Apple Vision Pro mixed reality (MR) headset.<\/p>\n

The researchers dubbed this method GAZEploit. In this post, we\u2019ll explore how the attack works, the extent of the threat to owners of Apple VR\/AR devices, and how best to protect your passwords and other sensitive information<\/a>.<\/p>\n

How text input works in Apple visionOS<\/h2>\n

First, a bit about how text is input in visionOS \u2014 the operating system powering Apple Vision Pro. One of the most impressive innovations of Apple\u2019s MR headset is its highly effective use of eye tracking<\/a>.<\/p>\n

Gaze direction serves as the primary method of user interaction with the visionOS interface. The tracking is so precise that it works even for the smallest interface elements \u2014 including the virtual keyboard.<\/p>\n

\"Virtual<\/a>

visionOS uses a virtual keyboard and eye tracking to input text. Source<\/a><\/p><\/div>\n

Although visionOS offers voice control, the virtual keyboard<\/a> remains the primary text input method. For sensitive information such as passwords, visionOS provides protection against prying eyes: in screen-sharing mode, both the keyboard and the entered password are automatically hidden.<\/p>\n

\"Privacy<\/a>

During screen sharing, visionOS automatically hides passwords entered by Vision Pro users. Source<\/a><\/p><\/div>\n

Another key feature of Apple\u2019s MR headset lies in its approach to video calls. Since the device sits directly on the user\u2019s face, the standard front-camera option is no good for transmitting the user\u2019s video image. On the other hand, using a separate external camera for video calls would be very un-Apple-like; plus, video-conference participants wearing headsets would look rather odd.<\/p>\n

So Apple came up with a highly original technology that features a so-called virtual camera. Based on a 3D face scan, Vision Pro creates a digital avatar of the user (Apple calls it a Persona<\/a>), which is what actually takes part in the video call. You can use your Persona in FaceTime and other video-conferencing apps.<\/p>\n

https:\/\/media.kasperskydaily.com\/wp-content\/uploads\/sites\/102\/2024\/10\/04212038\/how-to-steal-passwords-apple-vision-pro-3.mp4<\/a><\/video><\/div>\n
\n

By using lots of biometric data, the Persona digital avatar in visionOS looks truly lifelike. Source<\/a><\/p>\n<\/div>\n

The headset\u2019s sensors track the user\u2019s face in real-time, allowing the avatar to mimic head movements, lip movements, facial expressions, and so on.<\/p>\n

GAZEploit: How to snoop on Apple Vision Pro user input<\/h2>\n

For the GAZEploit researchers, the seminal feature of the Persona digital avatar is the use of data fed from the Vision Pro\u2019s highly precise sensors to replicate the user\u2019s eye movements with absolute pinpoint accuracy. And it was here that the team discovered a vulnerability enabling interception of input text.<\/p>\n

\"Schematic<\/a>

Here\u2019s how GAZEploit works in principle \u2014 allowing an attacker to intercept text entered by an Apple Vision Pro user. Source<\/a><\/p><\/div>\n

The attack\u2019s core concept is quite simple: although the system carefully hides passwords entered during video calls, by tracking the user\u2019s eye movements, mirrored by their digital avatar, a threat actor can reconstruct the characters entered on the virtual keyboard, or, rather, keyboards, as visionOS has three: passcode (PIN) keyboard, default QWERTY keyboard, and number and special character keyboard. This complicates the recognition process, since an outside observer doesn\u2019t know which keyboard is in use.<\/p>\n

\"visionOS<\/a>

visionOS actually has three different virtual keyboards: (\u0430) for passcodes, (b) for letters, and (c) for numbers and special characters. Source<\/a><\/p><\/div>\n

However, neural networks effectively automate the GAZEploit attack. The first stage of the attack uses a neural network to identify text-input sessions. Eye movement patterns during use of the virtual keyboard differ significantly from normal patterns: blink rates decrease, and gaze direction becomes more structured.<\/p>\n

\"Recognizing<\/a>

First, the neural network identifies when text is being entered on the virtual keyboard. Source<\/a><\/p><\/div>\n

At the second stage, the neural network analyzes gaze stability changes to identify eye-based selection of characters, and uses characteristic patterns to pinpoint virtual key presses. Then, based on gaze direction, the system calculates which key the user was looking at.<\/p>\n

\"Recognizing<\/a>

Next, the neural network recognizes individual virtual keystrokes and the characters being entered. Source<\/a><\/p><\/div>\n

How accurately GAZEploit recognizes input data<\/h2>\n

In actual fact, it\u2019s all a bit more complicated than the graph above suggests. Calculations based on the avatar\u2019s eye position generate a heatmap of probable points on the virtual keyboard where the user\u2019s gaze might have landed during text entry.<\/p>\n

\"Example<\/a>

Mapped gaze directions for keystroke inference of the demo attack: (a) adaptive virtual keyboard mapping, (b) predicted first guess keystrokes, (c) actual keystrokes. The accuracy isn\u2019t perfect, but it\u2019s no bad. Source<\/a><\/p><\/div>\n

Then, the researchers\u2019 model converts the collected information into a list of K<\/em> virtual keys that were most likely \u201cpressed\u201d by the user. The model also provides for various data-entry scenarios (password, email address\/link, PIN, arbitrary message), taking into account the specifics of each.<\/p>\n

What\u2019s more, the neural network uses a dictionary and additional techniques to improve interpretation. For example, due to its size, the spacebar is often a top-five candidate \u2014 producing many false positives that need filtering. The backspace key requires special attention: if the keystroke guess is correct, it means the previous character was deleted, but if it\u2019s wrong, then two characters may get mistakenly discarded.<\/p>\n

\"GAZEploit<\/a>

GAZEploit suggests the top-five most likely characters. Source<\/a><\/p><\/div>\n

The researchers\u2019 detailed error analysis shows that GAZEploit often confuses adjacent keys. At maximum precision (K<\/em>=1), roughly one-third of entered characters are identified correctly. However, for groups of five most likely characters (K<\/em>=5), depending on the specific scenario, the accuracy is already 73\u201392%.<\/p>\n

\"Overall<\/a>

The accuracy of GAZEploit recognition in various scenarios. Source<\/a><\/p><\/div>\n

How dangerous the GAZEploit attack is in practical terms<\/h2>\n

In practice, such accuracy means that potential attackers are unlikely to obtain the target password in ready-to-go form; but they can dramatically \u2014 by many orders of magnitude, in fact \u2014 reduce the number of attempts needed to brute-force it.<\/p>\n

The researchers claim that for a six-digit PIN, it\u2019ll only take 32 attempts to cover a quarter of all the most likely combinations. For a random eight-character alphanumeric password, the number of attempts is slashed from hundreds of trillions to hundreds of thousands (from 2.2\u00d71014<\/sup> to 3.9\u00d7105<\/sup>, to be precise), which makes password cracking feasible even with a prehistoric Pentium CPU.<\/p>\n

In light of this, GAZEploit could pose a serious enough threat and find practical application in high-profile targeted attacks. Fortunately, the vulnerability has already been patched: in the latest versions of visionOS, Persona is suspended when the virtual keyboard is in use.<\/p>\n

Apple could conceivably protect users from such attacks in a more elegant way \u2014 by sprinkling some random distortions in the precise biometric data driving the digital avatar\u2019s eye movements.<\/p>\n

Regardless, Apple Vision Pro owners should update their devices to the latest version of visionOS \u2014 and breathe easily. One last thing, we advise them \u2014 and everyone else \u2014 to exercise caution when entering passwords during video calls: avoid it if you can, always use the strongest (long and random) character combinations possible, and use a password manager<\/a> to create and store them.<\/p>\n\n","protected":false},"excerpt":{"rendered":"

Recent research describes a method for snooping on what Apple Vision Pro users enter on the virtual keyboard.<\/p>\n","protected":false},"author":2726,"featured_media":34181,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1788,1789,2646],"tags":[3086,14,1116,111,359,282,187,43,768,321,422,3278,1117,268],"class_list":{"0":"post-34178","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-privacy","8":"category-technology","9":"category-threats","10":"tag-account-hijacking","11":"tag-apple","12":"tag-ar","13":"tag-attacks","14":"tag-authentication","15":"tag-cybersecurity","16":"tag-passwords","17":"tag-privacy","18":"tag-surveillance","19":"tag-technology","20":"tag-threats","21":"tag-video-conferencing","22":"tag-vr","23":"tag-vulnerabilities"},"hreflang":[{"hreflang":"en-au","url":"https:\/\/www.kaspersky.com.au\/blog\/gazeploit-how-to-steal-passwords-apple-vision-pro\/34178\/"},{"hreflang":"en-in","url":"https:\/\/www.kaspersky.co.in\/blog\/gazeploit-how-to-steal-passwords-apple-vision-pro\/28101\/"},{"hreflang":"en-ae","url":"https:\/\/me-en.kaspersky.com\/blog\/gazeploit-how-to-steal-passwords-apple-vision-pro\/23362\/"},{"hreflang":"en-us","url":"https:\/\/usa.kaspersky.com\/blog\/gazeploit-how-to-steal-passwords-apple-vision-pro\/30584\/"},{"hreflang":"en-gb","url":"https:\/\/www.kaspersky.co.uk\/blog\/gazeploit-how-to-steal-passwords-apple-vision-pro\/28254\/"},{"hreflang":"ru","url":"https:\/\/www.kaspersky.ru\/blog\/gazeploit-how-to-steal-passwords-apple-vision-pro\/38314\/"},{"hreflang":"x-default","url":"https:\/\/www.kaspersky.com\/blog\/gazeploit-how-to-steal-passwords-apple-vision-pro\/52267\/"},{"hreflang":"ru-kz","url":"https:\/\/blog.kaspersky.kz\/gazeploit-how-to-steal-passwords-apple-vision-pro\/28350\/"},{"hreflang":"en-za","url":"https:\/\/www.kaspersky.co.za\/blog\/gazeploit-how-to-steal-passwords-apple-vision-pro\/33834\/"}],"acf":[],"banners":"","maintag":{"url":"https:\/\/www.kaspersky.com.au\/blog\/tag\/passwords\/","name":"passwords"},"_links":{"self":[{"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/posts\/34178","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/users\/2726"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/comments?post=34178"}],"version-history":[{"count":3,"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/posts\/34178\/revisions"}],"predecessor-version":[{"id":34184,"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/posts\/34178\/revisions\/34184"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/media\/34181"}],"wp:attachment":[{"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/media?parent=34178"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/categories?post=34178"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/tags?post=34178"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}