{"id":12073,"date":"2016-07-01T09:00:43","date_gmt":"2016-07-01T13:00:43","guid":{"rendered":"https:\/\/www.kaspersky.com.au\/blog\/?p=12073"},"modified":"2019-09-17T21:12:31","modified_gmt":"2019-09-17T10:12:31","slug":"how-facial-recognition-works","status":"publish","type":"post","link":"https:\/\/www.kaspersky.com.au\/blog\/how-facial-recognition-works\/12073\/","title":{"rendered":"Man vs. machine: facial recognition"},"content":{"rendered":"<p><span class=\"embed-youtube\" style=\"text-align:center; display: block;\"><iframe class=\"youtube-player\" type=\"text\/html\" width=\"640\" height=\"390\" src=\"https:\/\/www.youtube.com\/embed\/Y1lnrGIbweY?version=3&amp;rel=1&amp;fs=1&amp;showsearch=0&amp;showinfo=1&amp;iv_load_policy=1&amp;wmode=transparent\" frameborder=\"0\" allowfullscreen=\"true\"><\/iframe><\/span><\/p>\n<p>In 2010 owners of the largest facial database in the world \u2014 Facebook \u2014 <a href=\"http:\/\/edition.cnn.com\/2010\/TECH\/social.media\/07\/02\/facebook.recognition\/\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">learned<\/a> how to distinguish a portrait from a landscape: the social network searched for faces in photos and tagged these areas. Sometimes it made mistakes. Four years later Facebook <a href=\"http:\/\/www.theverge.com\/2014\/7\/7\/5878069\/why-facebook-is-beating-the-fbi-at-facial-recognition\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">could tell with 97% accuracy<\/a> who was depicted in a photo: one person or two different people.<\/p>\n<p><span class=\"embed-youtube\" style=\"text-align:center; display: block;\"><iframe class=\"youtube-player\" type=\"text\/html\" width=\"640\" height=\"390\" src=\"https:\/\/www.youtube.com\/embed\/l4Rn38_vrLQ?version=3&amp;rel=1&amp;fs=1&amp;showsearch=0&amp;showinfo=1&amp;iv_load_policy=1&amp;wmode=transparent\" frameborder=\"0\" allowfullscreen=\"true\"><\/iframe><\/span><\/p>\n<p>This a major advance for Facebook but its algorithm still loses out to human brain in three percent of such cases. If somebody asks us to recognize a familiar person in bad-resolution photos <a href=\"http:\/\/web.mit.edu\/sinhalab\/\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">we\u2019ll do it better than computers<\/a>. Even if these images were taken from an unusual angle.<\/p>\n<p>This is uncommon as usually computers are more accurate than human beings. Why are we better at solving such challenges and how computers try to do the same?<\/p>\n<h3>Our brains went through a serious training<\/h3>\n<p>It <a href=\"http:\/\/www.sciencemag.org\/news\/2012\/10\/identifying-brains-own-facial-recognition-system\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">has emerged<\/a> that a certain brain area is solely devoted to facial recognition. It\u2019s called the fusiform gyrus and it is a part of the temporal lobe and occipital lobe. Human beings learn to distinguish faces from the time of their birth \u2014 infants develop this skill in the first days of their life. At as early as four months, <a href=\"http:\/\/news.stanford.edu\/news\/2012\/december\/infants-process-faces-121112.html\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">babies\u2019 brains already distinguishes<\/a> one uncle from another \u2014 and aunties as well, of course.<\/p>\n<p>Eyes, cheekbones, nose, mouth and eyebrows are the key face features that help us to recognize each other. Skin is also important, especially its texture and color. It\u2019s noteworthy, that our brain tends to process a face as a whole \u2014 <a href=\"https:\/\/en.wikipedia.org\/wiki\/Face_perception#In_individuals_with_autism_spectrum_disorder\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">mostly it doesn\u2019t focus on individual features<\/a>. This is why we can easily recognize people even if they hide half of their face below a scarf or a piece of paper. However, if somebody makes a simple collage and joins faces of two famous people, viewers might need some time to understand who is who in a picture.<\/p>\n<div id=\"attachment_12075\" style=\"width: 1290px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/media.kasperskydaily.com\/wp-content\/uploads\/sites\/102\/2016\/05\/06022253\/joliepitt.gif\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-12075\" class=\"size-full wp-image-12075\" src=\"https:\/\/media.kasperskydaily.com\/wp-content\/uploads\/sites\/102\/2016\/05\/06022253\/joliepitt.gif\" alt=\"Man vs. machine: facial recognition\" width=\"1280\" height=\"840\"><\/a><p id=\"caption-attachment-12075\" class=\"wp-caption-text\">This is what you\u2019ll see if you combine Brad Pitt\u2019s and Angelina Jolie\u2019s portraits<\/p><\/div>\n<p>Since birth our brain stores faces. We gradually create a general template and use it for facial processing. If one could draw this template it might look like this:<\/p>\n<p>http:\/\/ic.pics.livejournal.com\/vls_smolich\/36547418\/256052\/256052_900.jpg<\/p>\n<p>Facial processing is going on at a time when our brain compares person\u2019s appearance with an internal template: if the person\u2019s nose is wider, lips are more plump, the skin tone is warm or cold, etc. Those of us who rarely travel say sometimes that people of other races look very similar. They think so because their templates are \u201csensitized\u201d to facial features, common for their surrounding.<\/p>\n<p>By the way, some <a href=\"https:\/\/peerj.com\/articles\/1115\/\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">animals can distinguish faces<\/a> as well like dogs and monkeys. Though sniffing gives them a lot of useful information, visual images also help these animals to recognize other living creatures. What\u2019s interesting, man\u2019s best friends \u2014 dogs \u2014 not only easily understand our mood by looking at our faces, they can also <a href=\"http:\/\/www.wikihow.com\/Teach-a-Dog-to-Smile\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">learn how to smile<\/a> as well.<\/p>\n<h3>How does a computer recognize faces?<\/h3>\n<p>What is the connection between human smiles and facial processing? These two are almost inseparable as any expression changes our faces beyond recognition, especially for computer algorithms.<\/p>\n<p>Software can compare two front-facing facial portraits and determine if they depict one and the same person. These solutions work quite like portrait-painters: they analyze so-called nodal points on human faces. These points are used to determine our individual faces; different methods find from 80 to 150 nodal points on a single face.<\/p>\n<p>For example, artists and software both measure distance between eyes, width of the nose, depth of the eye sockets, the shape of the cheekbones, the length of the jaw line and so on.<\/p>\n<p>http:\/\/jeldshi.muzkult.ru\/img\/upload\/770\/image_image_77893.jpg<\/p>\n<p>When you change the eye level or ask the model to turn their head, these measurements change. As many facial processing algorithms analyze the images only in two-dimensional space, point of sight is crucial for accurate recognition. Do you want to stay incognito? Hide your eyes and cheekbones behind sunglasses and cover your chin and mouth with a scarf to preserve anonymity. When <a href=\"https:\/\/www.kaspersky.com.au\/blog\/findface-experiment\/11916\/\" target=\"_blank\" rel=\"noopener noreferrer\">we tested the scandalous FindFace service<\/a>, it was able to recognize models only on front-facing portraits.<\/p>\n<p>https:\/\/www.kaspersky.ru\/blog\/files\/2016\/04\/rec.jpg<\/p>\n<p>This is how you can fool facial recognition services, which work with \u201cflat images.\u201d However, the morning sun never lasts a day and more progressive algorithms are already under way.<\/p>\n<h3>What\u2019s next?<\/h3>\n<p>Our brain trains to process faces as we grow up. The ability to distinguish between \u201cus\u201d and \u201cthem\u201d is one of the essential skills necessary for survival. <a href=\"https:\/\/www.ted.com\/talks\/jeremy_howard_the_wonderful_and_terrifying_implications_of_computers_that_can_learn?language=en\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">Modern computers can learn<\/a> like humans and program themselves. To improve results of machine facial processing, developers use self-learning algorithms, and feed them hundreds of human portraits as a schoolbook. It\u2019s not hard to find these images \u2014 there are a lot of them online, on social medias, photo hosting sites, photo stocks and other web-resources.<\/p>\n<blockquote class=\"twitter-tweet\" data-width=\"500\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\">You can\u2019t replace your face, says facial recognition \u2013 <a href=\"https:\/\/t.co\/tW6vdmxPWE\" target=\"_blank\" rel=\"noopener nofollow\">https:\/\/t.co\/tW6vdmxPWE<\/a> <a href=\"https:\/\/t.co\/dKXmOVdJ33\" target=\"_blank\" rel=\"noopener nofollow\">pic.twitter.com\/dKXmOVdJ33<\/a><\/p>\n<p>\u2014 Kaspersky (@kaspersky) <a href=\"https:\/\/twitter.com\/kaspersky\/status\/723502848114307072?ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener nofollow\">April 22, 2016<\/a><\/p><\/blockquote>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<p>Facial-based identification became more efficient when <a href=\"https:\/\/en.wikipedia.org\/wiki\/Three-dimensional_face_recognition\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">algorithms started working with 3-D models<\/a>. Projecting a grid onto the face and integrating video capture of human head software understands how this person looks from different angles. By the way, templates in human brains are also three-dimensional. Though this technology is still under development one can already find several proprietary solutions on the market.<\/p>\n<p>Mimic studies also gain traction. Realistic rendering of emotions is a gold mine for video gaming industry and a great number of companies work hard to make their characters more and more convincing. Important steps in this direction are already taken. The same technology will be of great service to facial recognition software \u2014 when these solutions wade through human mimics, they will know that this funny smiley in photo is probably pulled by that young girl on the street.<\/p>\n<p><span class=\"embed-youtube\" style=\"text-align:center; display: block;\"><iframe class=\"youtube-player\" type=\"text\/html\" width=\"640\" height=\"390\" src=\"https:\/\/www.youtube.com\/embed\/9ymDg2NI584?version=3&amp;rel=1&amp;fs=1&amp;showsearch=0&amp;showinfo=1&amp;iv_load_policy=1&amp;wmode=transparent\" frameborder=\"0\" allowfullscreen=\"true\"><\/iframe><\/span><\/p>\n<p>Apart from 3D models, developers work on other angles, for example, Identix company <a target=\"_blank\" rel=\"noopener noreferrer\">created<\/a> a biometrics technology for face recognition called FaceIt Argus. It analyzes the uniqueness of skin texture: lines, pores, scars and other things like that. FaceIt Argus creators claim that their development can identify differences between identical twins, which is not yet possible using facial recognition software alone.<\/p>\n<p>This system is said to be insensitive to changes in facial expression (like blinking, frowning or smiling) and has the ability to compensate for mustache or beard growth and the appearance of eyeglasses. Accurate identification can be increased by 20 to 25 percent if FaceIt Argus is used together with other facial processing systems. On the other hand, this technology fails if you use low-resolution images taken at low light.<\/p>\n<p>Anyway, to cover that eventuality there is other technology. Computer scientists at the Karlsruhe Institute of Technology (Germany) have developed the new technique, which <a href=\"http:\/\/www.dailymail.co.uk\/sciencetech\/article-3178864\/Nowhere-hide-Facial-recognition-technology-identify-people-complete-DARKNESS-reading-thermal-signatures.html\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">recognizes infrared portraits<\/a> of people, taken at poor lighting or even in total darkness.<\/p>\n<p>This technology analyzes human thermal signatures and matches their mid or far-infrared images with ordinary photos with 80% accuracy at maximum. The bigger number of images is available, the more successfully the algorithm works. When only one visible image is available, the accuracy drops to 55%.<\/p>\n<p><span class=\"embed-youtube\" style=\"text-align:center; display: block;\"><iframe class=\"youtube-player\" type=\"text\/html\" width=\"640\" height=\"390\" src=\"https:\/\/www.youtube.com\/embed\/a_W_ZmyYlj0?version=3&amp;rel=1&amp;fs=1&amp;showsearch=0&amp;showinfo=1&amp;iv_load_policy=1&amp;wmode=transparent\" frameborder=\"0\" allowfullscreen=\"true\"><\/iframe><\/span><\/p>\n<p>Making such a match is not as easy as it might seem at first glance: the thing is that there\u2019s no linear correlations between faces in regular and infrared light. The image, which is build on the base of thermal emissions, looks quite different from a regular portrait taken in daylight.<\/p>\n<p>The intensity of thermal emissions depends a lot on skin and environment temperatures and even on person\u2019s mood. Besides, usually infrared images have lower resolution than regular photos, which only makes the task more difficult.<\/p>\n<p>http:\/\/i.dailymail.co.uk\/i\/pix\/2015\/07\/29\/16\/2AEFE49A00000578-3178864-Computer_scientists_have_developed_a_technology_that_can_recogni-m-29_1438184452186.jpg<\/p>\n<p>To solve this problem, scientists turned to machine learning algorithm and \u201cfed\u201d their system with 1586 photos of 82 people.<\/p>\n<h3>It\u2019s everywhere!<\/h3>\n<p>Nowadays, facial recognition technologies are used almost all over the world. Recently Uber <a href=\"https:\/\/www.techinasia.com\/uber-china-facial-recognition\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">rolled out<\/a> similar solution in China to keep control of its own taxi drivers. NEC and Microsoft <a href=\"http:\/\/www.biometricupdate.com\/201604\/nec-combines-facial-recognition-technologies-with-azure-iot-technology\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">combine facial procession and IoT<\/a> to let marketing specialists get to know their clients better, and better and even better. At the same time, <a href=\"https:\/\/www.kaspersky.com.au\/blog\/findface-deanon\/11921\/\" target=\"_blank\" rel=\"noopener noreferrer\">trolls from Russian 2ch.ru forum use a facial recognition service to attack porn actresses<\/a> online.<\/p>\n<blockquote class=\"twitter-tweet\" data-width=\"500\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\"><a href=\"https:\/\/twitter.com\/hashtag\/Trolls?src=hash&amp;ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener nofollow\">#Trolls<\/a> expose <a href=\"https:\/\/twitter.com\/hashtag\/porn?src=hash&amp;ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener nofollow\">#porn<\/a> stars social networking accounts <a href=\"https:\/\/t.co\/2mY8kh0JlJ\" target=\"_blank\" rel=\"noopener nofollow\">https:\/\/t.co\/2mY8kh0JlJ<\/a> <a href=\"https:\/\/twitter.com\/hashtag\/socialmedia?src=hash&amp;ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener nofollow\">#socialmedia<\/a> <a href=\"https:\/\/t.co\/2tejy4TDZ9\" target=\"_blank\" rel=\"noopener nofollow\">pic.twitter.com\/2tejy4TDZ9<\/a><\/p>\n<p>\u2014 Kaspersky (@kaspersky) <a href=\"https:\/\/twitter.com\/kaspersky\/status\/723590897028440064?ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener nofollow\">April 22, 2016<\/a><\/p><\/blockquote>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<p>The development of facial recognition technology will make us rethink everything we know about privacy. It will not happen today or even in a year, but it\u2019s already high time to get ready. After all, you can\u2019t replace you face, can you?<\/p>\n<p>If you wonder what can be the outcome of technology invasion of privacy, we recommend you to watch the British mini-series \u201cBlack mirror,\u201d especially the \u201c<a href=\"https:\/\/en.wikipedia.org\/wiki\/Fifteen_Million_Merits_(Black_Mirror)\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">Fifteen Million Merits<\/a>\u201d episode.<\/p>\n<p><span class=\"embed-youtube\" style=\"text-align:center; display: block;\"><iframe class=\"youtube-player\" type=\"text\/html\" width=\"640\" height=\"390\" src=\"https:\/\/www.youtube.com\/embed\/zLZHdK6l55I?version=3&amp;rel=1&amp;fs=1&amp;showsearch=0&amp;showinfo=1&amp;iv_load_policy=1&amp;wmode=transparent\" frameborder=\"0\" allowfullscreen=\"true\"><\/iframe><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Accurate identification of people\u2019s faces is a very human process but computers are gaining on our processing. A look at what\u2019s going on now and what we\u2019ll see soon.<\/p>\n","protected":false},"author":522,"featured_media":12074,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[5,1788,2647,1789],"tags":[1506,1232,20,301,1565,880,43,1083],"class_list":{"0":"post-12073","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-news","8":"category-privacy","9":"category-special-projects","10":"category-technology","11":"tag-algorithms","12":"tag-biometrics","13":"tag-facebook","14":"tag-facial-recognition","15":"tag-findface","16":"tag-future","17":"tag-privacy","18":"tag-technologies"},"hreflang":[{"hreflang":"en-au","url":"https:\/\/www.kaspersky.com.au\/blog\/how-facial-recognition-works\/12073\/"},{"hreflang":"en-us","url":"https:\/\/usa.kaspersky.com\/blog\/how-facial-recognition-works\/7357\/"},{"hreflang":"en-gb","url":"https:\/\/www.kaspersky.co.uk\/blog\/how-facial-recognition-works\/7382\/"},{"hreflang":"es-mx","url":"https:\/\/latam.kaspersky.com\/blog\/how-facial-recognition-works\/7316\/"},{"hreflang":"es","url":"https:\/\/www.kaspersky.es\/blog\/how-facial-recognition-works\/8564\/"},{"hreflang":"it","url":"https:\/\/www.kaspersky.it\/blog\/how-facial-recognition-works\/8512\/"},{"hreflang":"tr","url":"https:\/\/www.kaspersky.com.tr\/blog\/how-facial-recognition-works\/2302\/"},{"hreflang":"x-default","url":"https:\/\/www.kaspersky.com\/blog\/how-facial-recognition-works\/12073\/"},{"hreflang":"pt-br","url":"https:\/\/www.kaspersky.com.br\/blog\/how-facial-recognition-works\/6402\/"},{"hreflang":"de","url":"https:\/\/www.kaspersky.de\/blog\/how-facial-recognition-works\/8090\/"},{"hreflang":"ja","url":"https:\/\/blog.kaspersky.co.jp\/how-facial-recognition-works\/11385\/"},{"hreflang":"en-za","url":"https:\/\/www.kaspersky.co.za\/blog\/how-facial-recognition-works\/12073\/"}],"acf":[],"banners":"","maintag":{"url":"https:\/\/www.kaspersky.com.au\/blog\/tag\/algorithms\/","name":"algorithms"},"_links":{"self":[{"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/posts\/12073","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/users\/522"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/comments?post=12073"}],"version-history":[{"count":3,"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/posts\/12073\/revisions"}],"predecessor-version":[{"id":23434,"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/posts\/12073\/revisions\/23434"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/media\/12074"}],"wp:attachment":[{"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/media?parent=12073"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/categories?post=12073"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kaspersky.com.au\/blog\/wp-json\/wp\/v2\/tags?post=12073"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}