Biometric authentication device, biometric authentication method, and recording medium

This application is a Continuation of U.S. application Ser. No. 17/270,303 filed on Feb. 22, 2021, which is a National Stage Entry of PCT/JP2019/032729 filed on Aug. 22, 2019, which claims priority from Japanese Patent Application 2018-162229 filed on Aug. 31, 2018, the contents of all of which are incorporated herein by reference, in their entirety.

The example embodiments relate to a biometric authentication device and the like for authenticating a user.

A personal authentication technology using biometric information on a user, such as a fingerprint, iris, face, voiceprint, and ear, has the advantage of being less likely to be leaked or stolen and not to be forgotten or lost than a password or key. For this reason, various biometric authentication approaches for authenticating users have been developed in recent years, including voiceprint (voice) authentication (hereinafter referred to as voice authentication) and otoacoustic authentication (hereinafter referred to as ear authentication).

In voice authentication, the user's voice is used for authentication. For example, in an instrument such as a smart speaker, a user performs an operation through voice interaction with an artificial intelligence (AI) agent equipped in the instrument. Voice authentication is used for purposes such as logging in to a service provided by the instrument or providing a more suitable response to an individual user from the instrument (personalizing) (see PTL 1).

In ear authentication, reverberation characteristics in an ear of a user are used for authentication. For example, a hearable device such as headphones or earphones is equipped with a microphone, and extracts the reverberation characteristics of a sound in the ear to authenticate a user. The microphone of the hearable device can also be used to simultaneously perform the above voice authentication (see PTL 2).

When the above voice authentication and ear authentication are performed independently, the authentication is not precise in some cases. For example, when the background noise is large, the accuracy of the voice authentication is lowered, and when the earphone is not sufficiently adhered to the ear, the accuracy of the ear authentication is lowered. If authentication fails, the user cannot immediately use an instrument (or function) accessible on the assumption that authentication succeeds. This could bring about fatal consequences for users who need to use the instrument immediately (for example, emergency medical personnel). Meanwhile, when voice authentication and ear authentication are simply combined, although the accuracy becomes higher, the time taken by authentication is given as the sum of times required for these approaches of authentication, and the user has to wait for a longer time.

The disclosure has been made in view of the above disadvantages, and one object of the disclosure is to provide a biometric authentication device and the like capable of performing highly accurate, quick authentication in combination of voice authentication and ear authentication.

In view of the above disadvantages, a biometric authentication device according to a first aspect of the disclosure includes:

A biometric authentication method according to a second aspect of the disclosure includes:

A biometric authentication program according to a third aspect of the disclosure includes:

The program may be stored on a non-transitory computer-readable storage medium.

According to the disclosure, a biometric authentication device and the like capable of performing highly accurate, quick authentication in combination of voice authentication and ear authentication can be provided.

Hereinafter, several example embodiments will be described in detail with reference to the drawings.

(Biometric Authentication Device)

A biometric authentication deviceaccording to a first example embodiment will be described with reference to. The biometric authentication deviceis equipped with an electronic circuit incorporating a voice authentication process, an ear authentication process, and a process of integrating results of both of the processes. Alternatively, a program for executing these processes may be implemented by being installed in a communication instrument (such as a smartphone) owned by a user. As illustrated in, the biometric authentication deviceis connected to an earphonebuilt with a speakerand a microphonevia a wired cable or wireless communication. The speakeris connected to a replay unitof the biometric authentication device. The microphoneis connected to a recording unitof the biometric authentication device.

As illustrated in, the biometric authentication devicemay be equipped on the earphonebuilt with the speakerand the microphone.

The speakeroutputs a sound to a user side. In ear authentication, the microphoneinputs a reverberation sound from an ear side of the user while the user is wearing the earphone. In voice authentication, the microphoneinputs a speaking of the user while the user is wearing the earphone. When it is difficult to pick up the user's voice during voice authentication, the user may move the earphonenear the mouth or the like. The microphonesmay be prepared separately for voice authentication and ear authentication. For example, using an earphone on a headset side, a microphone for ear authentication may be equipped inside the earphone, and a microphone alone such as an earphone microphone may be fixed near the mouth of the user as a microphone for voice authentication.

The communication instrument (not illustrated) owned by the user may be equipped with an AI interaction program. The AI interaction program can be operated in cooperation with various applications, and is capable of acquiring various types of information and executing various application functions through interaction using voice. The voice on the AI side is delivered to the user via the speakerbuilt in the earphone, and the voice on the user side is delivered to the AI interaction program via the microphonebuilt in the earphone. The above-mentioned information acquisition and function execution includes information and functions that are only permitted to be accessed by an authorized user. Examples of the above include functions that require strong security to protect personal information, such as connecting to an email account and a bank account. When requested by the user to access such a function, the AI interaction program activates the biometric authentication deviceto manage the access.

As illustrated in, the biometric authentication deviceaccording to the first example embodiment includes an interaction control unit, the replay unit, the recording unit, an ear authentication unit, a voice authentication unit, and an authentication integration unit.

The interaction control unitcontrols in such a way that, after the replay unitoutputs a sound into the user's ear, the recording unitinputs (records) the user's voice. The interaction control unitincludes an ear authentication notification unit, a voice authentication notification unit, an ear authentication instruction unit, a voice authentication instruction unit, and an authentication result storage unit.

The ear authentication notification unitnotifies the user that ear authentication (or both of ear authentication and voice authentication) will begin. Specifically, the ear authentication notification unitcauses the replay unitto reproduce a voice (which can be a recorded voice) for letting the user know that the authentication is to start.

The voice authentication notification unitnotifies the user that voice authentication will begin. Specifically, the voice authentication notification unitcauses the replay unitto reproduce a voice (which can be a recorded voice) for letting the user know that authentication is to start.

The ear authentication instruction unitinstructs the ear authentication unitto start ear authentication.

The voice authentication instruction unitinstructs the voice authentication unitto start voice authentication. The voice authentication instruction unitor the voice authentication unitmay generate a one-time password that can be used only for a predetermined time. The generated one-time password is delivered to the user by voice via the earphone, or is delivered to the user by being displayed on a display or the like of the user's communication instrument (not illustrated).

The authentication result storage unitstores verification scores notified from the ear authentication unitand the voice authentication unit.

The replay unitreproduces (outputs) a recorded sound. The replay unitmay include a storage unit for storing a sound to be reproduced.

The recording unitrecords a sound that is input. The recording unitmay include a storage unit for storing a recorded sound.

The ear authentication unitincludes a signal generation unit, an acoustic characteristic extraction unit, an ear verification unit, and a feature storage unit.

The signal generation unitgenerates a sound (such as a probe signal) to be output from the replay unit. This sound is output to acquire a reverberation sound in the ear of the user to be authenticated.

The acoustic characteristic extraction unitacquires the reverberation sound in the ear of the user to be authenticated, via the recording unit, and extracts acoustic characteristics relevant to the ear shape of the user from the acquired reverberation sound.

The ear verification unitverifies whether acoustic characteristics of the user to be authenticated, which are stored in advance in the feature storage unit, coincide with the extracted acoustic characteristics. The result of the verification is output as a verification score (ear authentication score).

The feature storage unitstores the acoustic characteristics of the user to be authenticated, which have been acquired in advance.

The voice authentication unitincludes a speaker feature extraction unit, a voice verification unit, and a feature storage unit.

The speaker feature extraction unitextracts a talker feature from the voice of the user recorded via the recording unit.

The voice verification unitverifies whether a talker feature of the user to be authenticated, which is stored in advance in the feature storage unit, coincides with the extracted talker feature. The result of the verification is output as a verification score (voice authentication score).

The feature storage unitstores the talker feature of the user to be authenticated, which has been acquired in advance.

The authentication integration unitcalculates and outputs an authentication integration result, based on the ear authentication score and the voice authentication score.

The authentication integration result to be output is passed to a program that manages access to the secured instrument or function, such as the aforementioned AI interaction program. If the authentication integration result is equal to or more than a predetermined value, the AI interaction program permits access to a function desired by the user and terminates the biometric authentication device.

(Operation of Biometric Authentication Device)

The operation of an authentication process of the biometric authentication deviceaccording to the first example embodiment will be described below with reference to the sequence chart illustrated in.

First, in step S, when the interaction control unitreceives the fact that biometric authentication is required due to a request or the like from the user, the ear authentication notification unitof the interaction control unitinstructs the replay unitto reproduce a voice for notifying the user of the start of ear authentication and voice authentication. The replay unitreproduces a voice (for example, “ear authentication is starting and then voice authentication will follow”) for notifying the user of the start of ear authentication and voice authentication, in line with the instruction. The notification may be made not only by voice but also on a screen. For example, when the user owns a communication instrument (not illustrated) provided with a display unit, “authentication is starting” may be displayed on the provided display unit.

In step S, the ear authentication instruction unitinstructs the ear authentication unitto start ear authentication. Upon receiving this instruction, the signal generation unitof the ear authentication unitgenerates a probe signal and sends the generated probe signal to the replay unit.

In step S, the replay unittransmits the probe signal to the speaker, and the speakerreproduces the transmitted probe signal. The probe signal may be an audible sound such as a beep sound or a melody, or may be a white noise. The probe signal may be a non-audible sound such as ultrasound.

In step S, the recording unitrecords a reverberation sound of the reproduced probe signal in the user's ear via the microphone, and sends the recorded data to the ear authentication unit. The ear verification unitof the ear authentication unitperforms an ear authentication process using the recorded data. In the ear authentication process, the ear verification unitestimates a transfer function in the ear using the probe signal and the reverberation sound. The ear verification unitmay further compress the transfer function into a low-dimensional feature by principal component analysis or the like, or may convert the transfer function into a feature such as a mel-frequency cepstral coefficient (MFCC). The output thus obtained is employed as an ear acoustic feature. The ear verification unitcompares the obtained ear acoustic feature with another ear acoustic feature exactly of the concerned user extracted in advance by the same procedure at the time of user registration, and calculates an ear acoustic score. A Euclidean distance or a cosine distance is conceivable as a distance scale used for score calculation.

In step S, the ear verification unitsends the verification score and the like as results of the ear authentication process to the interaction control unit. The interaction control unittemporarily stores the received verification score of the ear authentication in the authentication result storage unit.

In step S, the voice authentication notification unitof the interaction control unitinstructs the replay unitto reproduce a voice for notifying the user of the start of voice authentication and a prompt for the voice authentication. At this time, since the user has already recognized in step Sthat the authentication process is being executed, it is unnecessary to make a notification directly indicating the start of execution of voice authentication, such as “voice authentication is starting”, but it is sufficient to give only a specific prompt such as “please say your password”. The replay unitreproduces a voice for notifying the user of the start of voice authentication in line with the instruction.

As the password, a matter that is normally known only to the user (for example, “please say your mother's maiden name”) may be used. Words used in voice authentication may be notified by voice (for example, “please say today's date”). Besides, the password may be displayed on the display unit (not illustrated) of the user's communication instrument in such a way that the user is notified of the password (for example, “please say the password: ABC” is displayed). The voice authentication instruction unitmay generate a one-time password every time a predetermined time elapses, and cause the replay unitto broadcast the generated password and deliver the generated password to the user via the earphoneby sound, or cause the display unit (not illustrated) of the user's communication instrument to display the generated password. With this method, the degree of safety of the password can be enhanced.

In step S, the voice authentication instruction unitof the interaction control unitinstructs the voice authentication unitto start voice authentication. Upon receiving the instruction, the voice authentication unitinstructs the recording unitto start recording the user's speaking (password).

In step S, the recording unitrecords the voice of the user who has heard the voice authentication start notification and started an utterance via the microphone, and sends the recorded data to the voice authentication unit.