Human Ear Recognition - Computer Science and Engineering

IDENTITY SCIENCES

Human Ear Recognition

Arun Ross

West Virginia University

Ayman Abaza

West Virginia High Tech Consortium Foundation

Ear recognition technology is a potentially valuable tool in the
biometric arsenal.

I n a world where social inter- In spite of tremendous biometric Bertillon, a French police officer
action is increasingly digital advances, identifying noncooperative who pioneered the use of physical
in nature (Facebook, Google+, individuals in public spaces and other measurements to identify criminals.
Skype) and financial transac- unconstrained environments remains Bertillon combined qualitative and
tions are routinely conducted over a challenging problem. Only partial quantitative descriptions of various
the Internet (online banking), reliably or corrupted biometric information body parts, including the ear, in what
establishing an individual’s identity might be available—for example, a he called anthropometry (Identifica-
is of paramount importance. Sev- surveillance video might capture only tion anthropométrique: instructions
eral law enforcement and military a portion of an individual’s face. signalétiques, 1885).
applications also need a depend-
able method to identify people—for To improve human recognition, In 1906, R. Imhofer, a doctor in
example, to determine if an encoun- biometric researchers are exploring Prague, studied a set of 500 ears and
tered individual is a potential threat the use of ancillary characteristics noted that he could clearly distin-
or criminal suspect. such as scars, marks, tattoos, height, guish between them based on only
and body shape in conjunction with four features (“Die Bedeutung der
The limitations of traditional primary features like the face. The Ohrmuschel für die Feststellung der
modes of authentication based on ear is one such promising “soft” Identität,” Archiv für die Kriminologie,
ID cards and passwords have led to biometric. vol. 26, pp. 150-163).
the development of sophisticated
biometric systems that establish The external ear flap, known as More than 50 years later, a team of
human identity using an individual’s the pinna, has several morphologi- researchers visually assessed 206 sets
physical or behavioral attributes, cal components as Figure 1a shows. of ear photographs of newborn babies
such as fingerprints, face, iris, hand While its structure is relatively and concluded that the morphological
geometry, voice, or gait. Biometric simple, it varies significantly across constancy of the ear could be used
systems are now being incorporated individuals. Figure 1b shows exam- to establish a newborn’s identity (C.
in various applications ranging ples of these variations, which, along Fields et al., “The Ear of the Newborn
from personal laptop access to with the ear’s size, color, and texture as an Identification Constant,”
international border control. The can serve as a distinguishing charac- Obstetrics and Gynecology, July 1960,
US-VISIT program, for example, teristic. Changes in facial expression pp. 98-102).
employs fingerprint recognition to and age do not significantly impact
determine if a traveler to the US is on the ear’s appearance, although the Between 1948 and 1962, Alfred
a government watch list. Similarly, effect of gravity and ear accessories Iannarelli collected ear photographs
the United Arab Emirates uses the Iris can perturb the length of the ear lobe. of thousands of individuals and
Expellee Tracking System to identify extracted 12 different geometric
and apprehend deported individuals EARLY RESEARCH measurements of the ear based on the
who attempt to reenter the country crus of helix (The Iannarelli System of
using false travel documents. The ear’s potential for use in Ear Identification, Foundation Press,
human identification was recognized 1964), as Figure 2 shows. Iannarelli
as early as the 1880s by Alphonse claimed that this set of measurements

0018-9162/11/$26.00 © 2011 IEEE Published by the IEEE Computer Society NOVEMBER 2011 79

r11ids.indd 79 10/25/11 1:08 PM

IDENTITY SCIENCES

(a) (b)

Figure 1. External anatomy of the ear. (a) The external flap, referred to as the pinna, has several morophological components: (1) helix
rim, (2) lobule, (3) antihelix, (4) concha, (5) tragus, (6) antitragus, (7) crus of helix, (8) triangular fossa, and (9) incisure intertragica. (b)
The pinna’s structure varies across individuals. Examples of right (top row) and left (bottom row) ear images.

Figure 2. The Iannarelli identification Ear recognition involves four steps. AUTOMATED EAR
system entails the extraction of 12 Ear detection. The first step is RECOGNITION
geometric measurements of the ear to localize the ear’s position in an
based on the crus of helix. image. The system typically uses a Mark Burge and Wilhelm Burger
rectangular boundary to indicate reported the first attempt to automate
was reasonably unique across the ear’s spatial extent in the side the ear recognition process in 1997
individuals. profile of a face image. Ear detection (“Ear Biometrics for Computer Vision,”
is critical because errors at this stage Proc. 21st Workshop Austrian Assoc.
EAR BIOMETRICS can undermine the system’s utility. for Pattern Recognition, 1997, pp. 275-
Feature extraction. While the 282). They used a mathematical graph
An ear biometric system can be system can directly use the seg- model to represent and match the
viewed as a typical pattern recogni- mented ear during the matching curves and edges in a 2D ear image.
tion system that reduces an input stage, most systems extract a
image to a set of features and then salient set of features to represent Two years later, Belén Moreno,
compares this against the feature the ear. Feature extraction reduces Ángel Sanchez, and José Vélez
sets of other images to determine the segmented ear to a mathemati- described a fully automated ear
its identity. Ear recognition can be cal model—for example, a feature recognition system based on various
accomplished using either a 2D digital vector—that summarizes the dis- features such as ear shape and
image of the ear or a 3D point cloud criminatory information present in wrinkles (“On the Use of Outer Ear
that captures the ear’s surface. the ear image. Images for Personal Identification
Matching. The system compares in Security Applications,” Proc. 33rd
the features extracted from the Ann. Int’l Carnahan Conf. Security
input ear image to those stored Technology, IEEE, 1999, pp. 469-476).
in the database to establish the
ear’s identity. In its simplest form, Since then, researchers have pro-
matching generates scores indicating posed numerous feature extraction
the similarity to other ear images. and matching schemes, based on
Decision. The system uses the computer vision and image processing
match scores to render a final deci- algorithms, for ear recognition. These
sion. In verification mode, a “yes” range from simple appearance-based
indicates a genuine match and a “no” methods such as principal component
an impostor. In identification mode, analysis and independent compo-
the output is a list of potential match- nent analysis to more sophisticated
ing identities ranked by match score. techniques based on scale-invariant
feature transforms, local binary pat-
terns, wavelet transforms, and force
fields. (D.J. Hurley, M.S. Nixon, and J.N.
Carter, “Force Field Feature Extraction

80 computer

r11ids.indd 80 10/25/11 1:08 PM

Figure 3. Occlusion due to accessories and hair can lower or inhibit ear recognition system performance.

for Ear Biometrics,” Computer Vision in the biometric arsenal. For exam- been widely accepted in court due
and Image Understanding, June 2005, ple, forensic examiners reviewing to a lack of scientific consensus as to
pp. 491-512). surveillance videotapes in the Neth- their individuality.
erlands used the ear biometric to
In 2005, Hui Chen and Bir Bhanu identify suspects in gas station rob- C urrently, there are no com-
presented a 3D ear recognition beries who had covered their faces, mercially available ear rec-
system that exploited the depth and but not their ears, (A.J. Hoogstrate, ognition systems. However,
structure of the ear’s morphological H.V.D. Heuvel, and E. Huyben, “Ear the future holds tremendous potential
components (“Contour Matching for Identification Based on Surveillance for incorporating ear images with face
3D Ear Recognition,” Proc. 7th IEEE Camera Images,” Science & Justice, images in a multibiometric configura-
Workshop Applications of Computer July 2001, pp. 167-172). tion, even as researchers continue to
Vision [WACV 05], IEEE, pp. 123-128). refine the technology. For example,
To improve matching accuracy, assigning an ear image to one of sev-
IMPROVING MATCHING researchers are exploring the possi- eral predefined categories could allow
ACCURACY bility of combining images of the ear for rapid retrieval of candidate identi-
and the face. Even if the ear cannot ties from a large database. In addition,
As Figure 3 shows, occlusion due be used to verify human identity in the use of ear thermograms could
to hair and accessories can lower a given situation, it could exclude an help mitigate the problem of occlu-
or inhibit ear recognition system identity from being considered as a sion due to hair and accessories. As
performance. Changes in external potential match if it is sufficiently dif- the technology matures, both forensic
lighting and variations in facial pose ferent from the input probe image. and biometric domains will benefit
with respect to the camera can also from this biometric.
have a negative impact. EARPRINTS
Arun Ross is an associate professor
In addition, the recognition accu- The use of 2D or 3D ear images in the Lane Department of Computer
racy of ear recognition algorithms for human recognition differs from Science and Electrical Engineering at
has predominantly been evaluated the use of earprints: marks left by West Virginia University. Contact him
using ear images acquired under secretions from the outer ear when at [email protected].
ideal conditions, such as an indoor someone presses up against a wall,
environment with highly controlled door, or window. Earprints have been Ayman Abaza is a senior scientist in
lighting. This has generated criticism introduced as physical evidence in the Advanced Technology Group at the
that the matching accuracy of these several criminal cases in the US and West Virginia High Technology Foun-
algorithms, as reported in the litera- other countries, although some con- dation. Contact him at aabaza@wvhtf.
ture, could be overly optimistic. victions that relied on earprints have org.
been overturned. Earprints haven’t
Nevertheless, ear recognition tech- Editor: Karl Ricanek Jr., director of the Face
nology is a potentially valuable tool Aging Group at the University of North
Carolina Wilmington; [email protected]
build your career
Selected CS articles and columns
IN COMPUTING are available for free at
http://ComputingNow.computer.org.
www.computer.org/buildyourcareer

NOVEMBER 2011 81

r11ids.indd 81 10/25/11 1:08 PM