Howling suppression method and apparatus, and in-ear earphones and storage medium

The present disclosure claims the priority to the Chinese Patent Disclosure No. 202110982814.X, entitled “HOWLING SUPPRESSION METHOD AND APPARATUS, AND IN-EAR EARPHONES AND STORAGE MEDIUM” filed with China Patent Office on Aug. 25, 2021, the entire contents of which are incorporated into the present disclosure by reference.

The present disclosure relates to a technical field of audio processing, and more particularly, to a howling suppression method and apparatus, and in-ear earphones and a storage medium.

As we age, the range of frequencies that human ears can hear may decrease. The “World Hearing Report” released by the WHO states that one fifth of the world's people are experiencing hearing impairment, and hearing loss affects more than 1.5 billion people worldwide, of which 430 million people have moderate or higher hearing loss in the ear having better hearing, accordingly, the WHO recommends that ear and hearing care should be available to all. Due to market incentives, an auxiliary hearing function of TWS (True Wireless Stereo) earphones is becoming more and more important. Due to the hearing loss of hearing-impaired people in specific frequency bands, the auxiliary hearing function of TWS earphones is implemented by gain amplification in the missing frequency band for hearing-impaired patients. However, when implementing the auxiliary hearing function, since a distance between the microphone and the speaker is too close, howling phenomenon occurs.

In related art, adaptive filters are mainly used to suppress howling. However, when howling occurs in a higher frequency band, howling cannot be suppressed by simply use an adaptive filter, so as to impair the user's hearing.

Therefore, how to improve the suppression effect of howling events and reduce the damage to the user's hearing caused by audio playing are technical problems need to be solved.

An object of the present disclosure is to provide a howling suppression method and apparatus, and an in-ear earphone and storage medium, which can improve the suppression effect of howling events and reduce the damage to the user's hearing caused by audio playing.

In order to achieve the above object, the present disclosure provides a howling suppression method, the howling suppression method includes:

Optionally, performing a filtering process on the audio signal the amplitude gain reduction of which is reduced, to obtain the target audio signal includes:

Optionally, determining, according to an audio signal collected by a microphone, whether a howling event occurs includes:

Optionally, determining the howling type corresponding to the howling event includes:

Optionally, the method further includes:

Optionally, reducing the amplitude gain of the frequency band to which the howling event belongs includes:

Optionally, the method further includes:

The present disclosure also provides a howling suppression apparatus, the howling suppression apparatus includes:

The present disclosure also provides a storage medium, on which a computer program is stored, wherein when the computer program is executed, steps of the howling suppression method are implemented.

The present disclosure provides an in-ear earphone including a microphone, a speaker, a memory and a processor, wherein a computer program is stored in the memory, and when the computer program in the memory is executed by the processor, steps of the above howling suppression method are implemented.

The present disclosure provides a howling suppression method, including the following steps: determining, according to an audio signal collected by a microphone, whether a howling event occurs; if a howling event occurs, determining the howling type corresponding to the howling event; if the howling type is impulse wave howling, reducing the amplitude gain of a frequency band to which the howling event belongs, and performing a filtering process on the audio signal the amplitude gain reduction of which is reduced, to obtain a target audio signal; and controlling a speaker to play the target audio signal.

According to the present disclosure, if it is determined that a howling event occurs, determine a howling type corresponding to the howling event, so as to determine a frequency band to which the howling event belongs according to the howling type. If the howling type is impulse wave howling, it means that the energy of the howling event is high, and at this time, the howling cannot be completely suppressed by only using a filter. In the present disclosure, amplitude gains of the frequency band to which the howling event belongs is reduced, and then a filtering process on the audio signal the amplitude gain reduction of which is reduced is performed, to obtain a target audio signal. There are fewer feedback signals that can cause howling in the target audio signal, thus using a speaker to play the target audio signal can improve the suppression effect of howling event, reduce the damage to the user's hearing caused by audio playback. In addition, the present disclosure provides a howling suppression apparatus, a storage medium, and an in-ear earphone, to which the howling suppression method is applied, but will not be repeated herein.

Solutions of embodiments of the present disclosure will be clearly and completely described below in order to make the purpose, solutions and advantages of the embodiments of the present disclosure apparent. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present disclosure.

Referring to,is a flow chart of a howling suppression method according to an embodiment of the present disclosure.

Specifically, the method includes the following steps:

S: obtaining an audio signal collected by a microphone.

Here, the embodiment may be applied to hearing aids, earphones and other devices equipped with a microphone (e.g., a feedback microphone) and a speaker. When a distance between the microphone and the speaker is too close or a sound output by the speaker is too loud, the microphone collects an output signal of the speaker, so that the output signal of the speaker is amplified again, resulting in a howling event.

S: determining, according to the audio signal collected by the microphone, whether a howling event occurs; if a howling event occurs, enter S; if not, end the process.

After obtaining g the audio signal collected by the microphone, whether a howling event occurs may be determined based on an amplitude of the audio signal. That is, when there is a signal with an amplitude greater than a preset amplitude in the audio signal, it is determined that a howling event is detected.

S: determining a howling type corresponding to the howling event.

Here, the howling types may include impulse wave howling and non-impulse wave howling. The energy of the impulse wave howling is higher than the energy of the non-impulse wave howling. In the present disclosure, the impulse wave howling and the non-impulse wave howling may be divided according to the ability of an adaptive filter to process howling. For example, the highest energy of howling that the adaptive filter can suppress is A, the howling type of a howling event with energy greater than A is impulse wave howling, and the howling type of a howling event with energy less than or equal to A is non-impulse wave howling.

S: if the howling type is impulse wave howling, reducing an amplitude gain of a frequency band to which the howling event belongs, and performing a filtering process on the audio signal the amplitude gain reduction of which is reduced, to obtain a target audio signal.

Here, the howling event occurs at the current moment is caused by a shock wave, and the howling event cannot be suppressed by only using an adaptive filter. In the embodiment, the audio signal first is gained and then processed. Specifically, an amplitude of the frequency band to which the howling event belongs is gained, and then filtering process is performed on the audio signal the amplitude gain reduction of which is reduced, to obtain the target audio signal.

S: controlling a speaker to play the target audio signal.

According to the embodiment, if it is determined that a howling event occurs, determine a howling type corresponding to the howling event, so as to determine the frequency band to which the howling event belongs according to the howling type. If the howling type is impulse wave howling, it means that the energy of the howling event is high, and at this time, the howling cannot be completely suppressed by using only a filter. In the embodiment, amplitude gains of the frequency band to which the howling event belongs is reduced, and then a filtering process on the audio signal the amplitude gain reduction of which is reduced is performed, to obtain a target audio signal. There are fewer feedback signals that can cause howling in the target audio signal, and thus using a speaker to play the target audio signal can improve the suppression effect of howling event, reduce the damage to the user's hearing caused by audio playback.

As an embodiment, for impulse wave howling, an adaptive filter and a nonlinear post-processing filter may be cooperated to process the audio signal, to obtain the target audio signal. Referring to,is a schematic diagram of an audio signal filtering process method according to an embodiment of the present disclosure. This embodiment is a further introduction of Sin the embodiment of, i.e., the filtering process on the audio signal the amplitude gain reduction of which is reduced, and includes:

S: determining a speaker reference signal collected by a feedback microphone.

Here, in the embodiment, a speaker reference signal collected by a feedback microphone is used as a reference to remove or reduce nonlinear distortion howling.

S: inputting the speaker reference signal and the audio signal the amplitude gain reduction of which is reduced into an adaptive filter, to obtain a first filtering result.

S: inputting the first filtering result and the audio signal the amplitude gain reduction of which is reduced into a nonlinear post-processing filter, to obtain a second filtering result.

S: superimposing the first filtering result and the second filtering result to obtain a superposed result, and subtracting the superposed result from the audio signal, to obtain the target audio signal.

In the embodiment, an adaptive filter is used to eliminate linear echo, and a nonlinear post-processing filter is used to eliminate nonlinear echo. The first filtering result and the second filtering result are superimposed to obtain the target audio signal in which the linear echo and nonlinear echo are eliminated.

As an alternative embodiment, after determining the howling type according to the frequency band to which the howling event belongs, if the howling type is non-impulse wave howling, an adaptive filter is used to filter the collected audio signal, to obtain the target audio signal.

As an alternative embodiment, since the howling event refers to a situation where the energy (amplitude) of an output signal of the speaker in a high-frequency band is too high, the audio signal collected by the microphone includes the output signal of the speaker and an output signal of non-speaker (such as user voice, environmental noise, etc.). In the embodiment, whether a howling event occurs may be determined according to the amplitude of the audio signal collected by the microphone.

Accordingly, the embodiment can also determine the howling type using the speaker reference signal. It may include the steps of: extracting feature vectors from the audio signal; inputting the feature vectors into a howling type determining model; and determining the howling type corresponding to the howling event according to an output result of the howling type determining model.

As an embodiment, the above embodiment can reduce the amplitude gain of the frequency band to which the howling event belongs by way of: determining the gain change amount according to the span and energy of the frequency band to which the howling event belongs; and reducing the amplitude gain of the frequency band to which the howling event belongs according to the gain change amount.

Further, after reducing the amplitude gain of the frequency band to which the howling event belongs according to the gain change amount, the method may include: extending a preset duration and determining whether a howling event occurs at the current moment; if a howling event occurs, reducing the gain change amount by a preset value to obtain a new gain change amount, and reducing the amplitude gain of the frequency band to which the howling event belongs according to the new gain change amount.

Hereinafter, the process described in the above embodiments will be described below through a howling suppression solution for in-ear earphones in practical applications.is a schematic diagram showing a howling suppression principle according to an embodiment of the present disclosure. In this embodiment, a speaker signal collected by a feedback microphone is used as a reference, and a gain of the howling part can be reduced according to the howling type to maintain the stability of the adaptive filter. As illustrated in, a sound signal is collected by the microphone and passes through the speaker after speech pre-processing, the speech pre-processing includes noise reduction and dynamic range control processing, the signal passes through a feedback microphone of the speaker and a sound played by the speaker is collected. At this time, since a distance between the microphone and the speaker is short, the sound amplified by the speaker is picked up again by the microphone, and the picked-up signal is transmitted to the speaker again by the microphone. After multiple cycles, the signal is infinitely superimposed and amplified, and the amplitude is infinitely superimposed at a certain frequency point, resulting in howling.

Assume that the signal collected by the microphone is X, a pure speech signal is S, and a feedback signal is N, a relationship between the above signals is: X=S+N. The pure speech signal refers to a speech signal without any noise or feedback. The feedback signal refers to a signal output by the speaker and picked up by the microphone. The feedback signal can be determined based on a feedback path and the speaker reference signal, that is: N=F*Y, wherein F represents the feedback path, and Y represents the speaker signal. The feedback path refers to a propagation path of sound. For example, a sound is emitted by the speaker and is picked up by the microphone, the feedback path is a path where the sound from the speaker travels to the microphone. The speaker signal is used as a reference signal when processed in the filter, and accordingly, the speaker signal is also referred to as a speaker reference signal.

In order to solve the problem of howling, it is necessary to accurately estimate the feedback path and speaker signal. Estimating the feedback path means obtaining an estimation of the feedback path based on an update of the adaptive filter. The formula for feedback path estimation is: F′ (t)=F′ (t−1)+a((Y*Y)/((Y*E)+d)); wherein a represents a preset coefficient, E represents an error signal at the previous moment, t represents time, F′ represents the estimated feedback path information, and F represents the real feedback path information. In the embodiment, the feedback path information estimated by the above formula can be used as the real feedback path information, E(t−1)=X−F′ (t−1)*Y (t−1). To accurately estimate the coefficient, the reference signal Y, which is the sound emitted by the speaker, should be accurately obtained. The sound of the speaker may experience non-linear distortion after passing through the path or being gained again, as a result, directly using the speaker signal as a reference is quite different from the actual one. In the embodiment, a feedback microphone can be used to collect the sound of the speaker as a reference, which can effectively remove or reduce nonlinear components. Since the speaker signal may have non-linear distortion after amplification and output, this part of the non-linear distortion cannot be precisely estimated, but the speaker signal of the feedback microphone is actually the sound played by the speaker.

The update principle of the adaptive filter is that the update is stable when the signal is stable. If an impulse wave howling occurs, it will unable to converge when updating the adaptive filter, such that when there is an impulse wave howling caused by sounds such as a horn, the embodiment can accurately estimate the feedback path by first reducing the gain and then updating the coefficient of the adaptive filter.

The howling type can be classified based on excitation signals or normal speech signals, for example, a sound of horn, a working sound of microwave oven, and a speech signal may be included. In the embodiment, howling type determining models of neural networks such as Deep Neural Networks (DNN), Recurrent Neural Network (RNN), and Convolutional Neural Networks (CNN) may be used to perform howling determining. The classified signal is processed, if it is a sound of microwave oven or a whistle sound, frequency gain control is performed first, and then filtering is performed according to the adaptive filter to remove the feedback signal. However, since the speaker signal is an amplified signal after processing, the signal inevitably undergoes nonlinear changes, and NLP is added to filter the signal, which solves an error caused by nonlinearity and eliminates the instability of high-frequency gain.

Referring to,is a flow chart of sound signal processing for suppressing howling according to an embodiment of the present disclosure. The flow chart includes the following steps:

The microphone collects an audio signal X(n), and performs speech pre-processing operations such as noise reduction and dynamic range control on the audio signal X(n). In, s (n) represents a pure signal without feedback signal, and vF(n) represents the feedback signal.