Facial Recognition Using 3D Mug Shots – Future of Forensic Surveillance

Facial Recognition
Turning a 2D photograph in 3D for Facial Recognition

Facial recognition is an essential element of video forensics. Our human visual cognition system is attuned to natural facial recognition, in fact we easily recognize human faces even in tough visual environments, such as bad light conditions or different pose variations. Many technologies have also been developed that allow computers to recognize faces. Although many of these facial recognition systems have been used for a long time they are typically based on 2D image facial recognition. However, 2D image facial recognition systems pose many challenges which are directed related to its data variability. When a surveillance camera records a face, it’s usually at a strange angle – not the standard straight on image captured in a standard ID, such as a Driver’s License or passport photos. A 2D image based facial recognition system is an inadequate tool for matching faces captured from different angles. Other challenges to facial recognition may include challenges due to pose variations, bad lighting conditions, occlusions and facial expressions.

Facial Recognition

To overcome these challenges scientists have been working aggressively to develop computer vision systems that can process and analyze 3D faces exactly as the human vision does. Also called the 3D mug shot, this is an interesting technology which is recently being used by police officers in Tokyo. It creates a map of a face that can be used to match surveillance images — even at strange angles. From April 2016, Tokyo’s 102 Metropolitan Police Department Stations will place 3D cameras that will capture faces and record unprecedented facial information for their 3D identification stations. Besides being used for forensic and surveillance purposes, these systems can also be used in biometric machines, improve human–computer interaction (HCI), facial surgery, video communications and 3D animation.

Basic concept of 3D Facial Recognition

Facial Recognition
Computers can now adapt facial recognition software to compare against different facial expressions

The 3D facial recognition programs attempt to recover facial 3D shapes from cameras and reproduce their actions. Furthermore, the software also attempts to recover these facial shapes under multiple pose and light variations. The fields of computer vision and computer graphics are closely related to this technology of facial recognition. It requires high tech knowledge of capturing and processing human geometry. Such programs should be capable of using techniques for 3D reconstruction of geometric shapes. Way back in 2010 computerized tests were made on about 1.6 million mug shots to pick someone from these mug shots. The advanced algorithms achieved this with an accuracy of 92 percent. Tests were also run on photos of people who were not looking directly at the camera. With such advanced technologies, video forensic experts can use forensic video analysis and convert low quality surveillance images into powerful evidence with unprecedented accuracy.

Geometric and Topological Aspects of the Human Face

Facial Recognition
Resistance is Futile

Some notable geometric and topological features of a human face are considered as distinguishing features of any human face. In effect they pose both a challenge and an opportunity to the field of 3D face recognition. The following points discuss the influence of these aspects and the challenges they pose to 3D face recognition:

  1. Changes in Human face: The human face can change as a result of factors such as age, weight loss, weight gain and facial expressions. As distinguishing 3D shape variations of human face among different individuals are statistically small, these changes pose serious challenges in the field of 3D facial recognition. Besides changing the geometry, some changes such as mouth opening can result in topological changes to 3D facial structure as well. However, following three aspects of human face have helped overcome these challenges and also helped in the development of rigid approaches for 3D face recognitions systems:
    1. The anatomical structure of the face remains unchanged, especially in the case of changes related to human expressions.
    2. Some facial regions such as the nose and forehead are less affected by change in expression. These regions are also called the semirigid regions.
    3. Depending on the facial expression, some other facial regions besides the semirigid regions, will be less deformed.

Fiducial points in a Human face: Some detectable fiducial points in a human face are eye corners, the mid point between the eyes, the tip of the nose and its two lower corners, the furthest chin point, and mouth corners. These fiducial points are often used to establish point-to-point correspondence between two or more facial scans. They also act as standard locations from where other local features are extracted. This helps in the process of facial feature matching and recognition. However this can pose a challenge in case of deformed surfaces. In such cases the issues can be resolved by first establishing relation between these fiducial points.