Audio device and computer readable program

This application isa broadening reissue of U.S. Pat. No. 11,315,561, issued Apr. 26, 2022, which issued from U.S. Non-Provisional Application Ser. No. 16/638,540, filed Feb. 12, 2020, which isthe National Stage of International Application No. PCT/JP2018/009883, filed Mar. 14, 2018, which claims the benefit of Japanese Patent Application No. 2017-156673, filed Aug. 14, 2017. The contents of these prior applications are incorporated by reference herein in their entirety.

The present invention relates to an audio device having a function of receiving a voice operation.

In Patent Literature 1, there is disclosed an audio device configured to wirelessly download audio data from a media server or the like to reproduce the audio data in accordance with an operation received from a user through a remote controller.

In Patent Literature 2, there is disclosed a telephone set with a voice reply function, which is configured to allow a user to visually see a voice recognition result of a voice input to a microphone by varying a display form of a light emitting diode (LED).

[PTL 1] U.S. Pat. No. 7,987,294 A

[PTL 2] JP 2000-106588 A

In recent years, it has been demanded that such an audio device as described in Patent Literature 1 have a voice operation reception function of performing voice recognition processing on a voice input to a microphone to receive a voice operation, to thereby allow various operations to be performed without the use of the remote controller. In this case, in a case where a voice operation is received while audio data is being reproduced, when a reply to a voice recognition result of the input voice is output by voice, a voice indicating the voice recognition result is consequently mixed into a reproduced sound of the audio data, which affects an audio reproduction environment. It is therefore preferred that the voice recognition result be indicated through use of an LED as in the case of the telephone set with a voice reply function described in Patent Literature 2.

However, in the telephone set with a voice reply function described in Patent Literature 2, the display form of the LED is set to anyone of a turned-off state, a blinking state, and a turned-on state, to thereby merely notify the user of a case in which it has been determined that there is no voice reply (turned-off state), a case in which it has been determined that the input voice is not a predetermined voice (blinking state), and a case in which it has been determined that the input voice is the predetermined voice (turned-on state). Thus, even when this technology is applied to an audio device, a state of the voice recognition processing cannot be notified to the user in more detail. Meanwhile, through use of a plurality of LEDs or through use of a liquid crystal display (LCD) instead of the LEDs, a state of the voice recognition processing can be notified to the user in more detail, but cost is increased accordingly. Further, a display unit for notifying of a state of the voice recognition processing occupies a large proportion of the front surface of the audio device, which imposes a limitation on the design of the audio device.

The present invention has been made in view of the above-mentioned circumstances, and has an object to provide an audio device having a voice operation reception function, which is capable of notifying of a detailed state of voice recognition processing without affecting an audio reproduction environment, and is low in cost and excellent in the degree of freedom in design.

In order to solve the above-mentioned problem, in one embodiment of the present invention, a plurality of lighting patterns in each of which brightness is changed on a time axis are provided, and pulse width modulation (PWM) control is performed on an LED based on the plurality of lighting patterns, to thereby express a state of the voice recognition processing. In this case, the lighting pattern may have an intensity of brightness synchronized with intonation and accent of a voice message representing the state of the voice recognition processing.

For example, according to one embodiment of the present invention, there is provided an audio device, which has a voice operation reception function of receiving an operation through a voice input to a microphone, the audio device including: a light emitting diode (LED); LED control means for performing pulse width modulation (PWM) control on the LED; and lighting pattern storage means for storing, for each state of voice recognition processing, a lighting pattern in which brightness is changed on a time axis, in which the LED control means is configured to perform PWM control on the LED in accordance with a lighting pattern that is stored in the lighting pattern storage means in association with a state of voice recognition processing performed on the voice input to the microphone.

According to the present invention, the plurality of lighting patterns in which the brightness is changed on the time axis are provided, and the PWM control is performed on the LED through use of the plurality of lighting patterns, to thereby express the state of the voice recognition processing. Accordingly, it is possible to express the detailed state of the voice recognition processing with only one LED. Thus, according to the present invention, it is possible to provide the audio device having the voice operation reception function, which is capable of notifying of the detailed state of the voice recognition processing without affecting an audio reproduction environment, and is low in cost and excellent in the degree of freedom in design.

Now, one embodiment of the present invention is described with reference to the accompanying drawings.

1 FIG. is a diagram for illustrating a schematic configuration of a wireless audio system in this embodiment.

1 FIG. 4 1 4 2 3 4 1 As illustrated in, the wireless audio system in this embodiment includes a media serverand a wireless speaker, which is connected to the media servervia an access pointand a network, such as a WAN or a LAN. The media serverprovides audio data to the wireless speaker.

1 11 10 11 1 4 10 1 12 10 12 11 The wireless speakerincludes a microphoneon its front surface, on which a speakeris arranged, and performs voice recognition processing on a voice input to the microphoneto receive a voice operation from a user. Then, the wireless speakerfollows the voice operation received from the user to download audio data from the media serverand reproduce the audio data, and outputs a reproduced sound of the audio data from the speaker. Further, the wireless speakerincludes a full-color LEDon its front surface, on which the speakeris arranged, and uses the LEDto notify the user of a state of the voice recognition processing performed on the voice input to the microphone.

2 FIG. 1 is a diagram for illustrating a schematic functional configuration of the wireless speaker.

2 FIG. 1 10 11 12 13 14 15 16 17 18 19 As illustrated in, the wireless speakerincludes the speaker, the microphone, the full-color LED, a network interface unit, an audio reproduction unit, a voice recognition dictionary data storage unit, a voice recognition processing unit, a lighting pattern storage unit, an LED control unit, and a main control unit.

13 4 2 3 The network interface unitis an interface for communicating to/from the media servervia the access pointand the network.

14 19 10 The audio reproduction unitreproduces audio data received from the main control unit, and outputs a reproduced sound of the audio data from the speaker.

15 The voice recognition dictionary data storage unitstores, for each language, such as Japanese and English, voice recognition dictionary data to be used in voice recognition processing.

16 15 11 The voice recognition processing unitrefers to the voice recognition dictionary data storage unitto perform voice recognition processing on a voice input to the microphone.

17 12 The lighting pattern storage unitstores, for each language, such as Japanese and English, a lighting pattern of the LEDin which brightness is changed on a time axis, in association with a state of the voice recognition processing. The lighting pattern is to notify the user of a corresponding state of the voice recognition processing, and an intensity of brightness on the time axis is synchronized with intonation and accent of a voice message representing a state of the voice recognition processing in a corresponding language.

3 FIG.(A) 3 FIG.(E) toare each a graph for schematically showing an example of the lighting pattern.

170 171 Cases in which the voice message is Japanese are illustrated as examples. In the figures, a vertical axisindicates an intensity of brightness (W), and a horizontal axisindicates time (t).

3 FIG.(A) 172 11 172 is an illustration of an example of a lighting patterna for notifying the user of the fact that a recognition result of voice recognition processing performed on a voice input to the microphoneis to request start of reception of a voice operation (e.g., “kaishi (Start)”) and the state of the voice recognition processing is set to “voice operation reception is started” in response to this. In the lighting patterna, the intensity of brightness on the time axis is changed in synchronization with the intonation and accent of the voice message “ryōkai shimashita (OK)”.

3 FIG.(B) 172 11 172 is an illustration of an example of a lighting patternb for notifying, when a voice is input to the microphoneafter the start of reception of the voice operation, the user of the fact that the state of the voice recognition processing is set to “voice is input” in response to this. In the lighting patternb, the intensity of brightness on the time axis is changed in synchronization with the intonation and accent of the voice message “jushin shimashita (Your voice is received)”.

3 FIG.(C) 172 11 172 is an illustration of an example of a lighting patternc for notifying, when voice recognition is being performed on the voice input to the microphone, the user of the fact that the state of the voice recognition processing is set to “voice is being recognized” in response to this. In the lighting patternc, the intensity of brightness on the time axis is changed in synchronization with the intonation and accent of the voice message “ninshikichū desu (Now recognizing)”.

3 FIG.(D) 172 11 172 is an illustration of an example of a lighting patternd for notifying, when the voice recognition processing performed on the voice input to the microphonehas been successful and a command has successfully been received from the user, the user of the fact that the state of the voice recognition processing is set to “recognition is successful” in response to this. In the lighting patternd, the intensity of brightness on the time axis is changed in synchronization with the intonation and accent of the voice message “jikkō shimasu (Your command is executed)”.

3 FIG.(E) 172 11 172 is an illustration of an example of a lighting patterne for notifying, when the voice recognition processing performed on the voice input to the microphonehas failed, the user of the fact that the state of the voice recognition processing is set to “recognition has failed” in response to this. In the lighting patterne, the intensity of brightness on the time axis is changed in synchronization with the intonation and accent of the voice message “saido onegai shimasu (Please say it again)”.

3 FIG.(A) 3 FIG.(E) Into, the exemplary lighting patterns to be used when a voice message is Japanese are illustrated, but also in a lighting pattern to be used when a voice message is another language, for example, English, the intensity of brightness on the time axis is changed in synchronization with intonation and accent as in the case of Japanese.

18 17 12 19 The LED control unitrefers to the lighting pattern storage unitto perform PWM control on the LEDso that its brightness is changed on the time axis in accordance with a lighting pattern specified by the main control unit.

19 13 14 16 18 The main control unitcentrally controls the network interface unit, the audio reproduction unit, the voice recognition processing unit, and the LED control unit.

1 Next, an operation of the wireless speakerconfigured as described above is described.

4 FIG. 5 FIG. 1 andare diagrams for illustrating a flow of the operation of the wireless speaker.

11 1 16 15 2 16 19 First, when a voice is input from the user through the microphone(YES in Step S), the voice recognition processing unitrefers to the voice recognition dictionary data of each language stored in the voice recognition dictionary data storage unitto perform voice recognition processing on the input voice (Step S). Then, the voice recognition processing unitpasses a recognition result of the voice recognition processing to the main control unit.

19 3 19 19 3 1 In response to this, the main control unitdetermines whether or not a recognition result of the voice recognition processing is to request start of reception of a voice operation (Step S). For example, in a case where the input voice is Japanese, when the recognized voice is “kaishi (Start)”, the main control unitdetermines that the voice is to request start of reception of the voice operation, and in a case where the input voice is English, when the recognized voice is “Start”, the main control unitdetermines that the voice is to request start of reception of the voice operation. When the recognition result of the voice recognition processing is not to request start of reception of the voice operation (NO in Step S), the flow returns to Step S.

3 19 19 16 16 4 Meanwhile, when the recognition result of the voice recognition processing is to request start of reception of the voice operation (YES in Step S), the main control unitsets a language forming a sentence represented by the recognition result of the voice recognition processing, as a language to be recognized. Then, the main control unitinstructs the voice recognition processing unitto narrow down the voice recognition dictionary data to that of the language to be recognized. In response to this, the voice recognition processing unitnarrows down the voice recognition dictionary data to be used in the subsequent voice recognition processing to the voice recognition dictionary data of the language to be recognized (Step S).

19 18 18 12 18 17 18 12 5 Next, the main control unitnotifies the LED control unitof the language to be recognized, the state “voice operation reception is started” of the voice recognition processing, and a light emission color “first light emission color (e.g., blue)” to instruct the LED control unitto turn on the LED. In response to this, the LED control unitidentifies, from the lighting pattern storage unit, a lighting pattern that is associated with the language to be recognized and the state “voice operation reception is started” of the voice recognition processing. Then, the LED control unitperforms PWM control on the LEDin the first light emission color so that its brightness is changed in accordance with the identified lighting pattern (Step S). With this control, the user is notified of the fact that the reception of the voice operation is enabled.

19 18 18 12 18 12 6 After that, the main control unitnotifies the LED control unitof a light emission color “second light emission color (e.g., white)” to instruct the LED control unitto turn on the LED. In response to this, the LED control unitturns on the LEDin the second light emission color (Step S). With this control, the user is notified of the fact that input of a voice is being waited for.

11 7 16 19 19 18 18 12 18 17 18 12 10 Next, when a voice is input from the user through the microphone(YES in Step S), the voice recognition processing unitnotifies the main control unitof the fact that the voice has been input. The main control unitnotifies the LED control unitof the language to be recognized, the state “voice is input” of the voice recognition processing, and the light emission color “first light emission color” to instruct the LED control unitto turn on the LED. In response to this, the LED control unitidentifies, from the lighting pattern storage unit, a lighting pattern that is associated with the language to be recognized and the state “voice is input” of the voice recognition processing. Then, the LED control unitperforms PWM control on the LEDin the first light emission color so that its brightness is changed in accordance with the identified lighting pattern (Step S). With this control, the user is notified of the fact that the input of the voice has been detected.

16 15 11 16 19 19 18 18 12 18 17 18 12 12 After that, the voice recognition processing unitrefers to the voice recognition dictionary data of the language to be recognized stored in the voice recognition dictionary data storage unitto start voice recognition processing on the input voice (Step S). Further, the voice recognition processing unitnotifies the main control unitof the fact that the voice recognition processing is being performed. The main control unitnotifies the LED control unitof the language to be recognized, the state “voice is being recognized” of the voice recognition processing, and the light emission color “first light emission color” to instruct the LED control unitto turn on the LED. In response to this, the LED control unitidentifies, from the lighting pattern storage unit, a lighting pattern that is associated with the language to be recognized and the state “voice is being recognized” of the voice recognition processing. Then, the LED control unitperforms PWM control on the LEDin the first light emission color so that its brightness is changed in accordance with the identified lighting pattern (Step S). With this control, the user is notified of the fact that the voice recognition processing is being performed on the input voice.

16 19 16 19 16 13 19 18 18 12 18 17 18 12 16 6 Next, the voice recognition processing unitpasses a recognition result of the voice recognition processing to the main control unit. In this case, when the voice recognition processing has failed, the voice recognition processing unitnotifies the main control unitof this fact. In response to this, when the voice recognition processing performed in the voice recognition processing unithas failed (NO in Step S), the main control unitnotifies the LED control unitof the language to be recognized, the state “recognition has failed” of the voice recognition processing, and a light emission color “third light emission color (e.g., red)” to instruct the LED control unitto turn on the LED. In response to this, the LED control unitidentifies, from the lighting pattern storage unit, a lighting pattern that is associated with the language to be recognized and the state “recognition has failed” of the voice recognition processing. Then, the LED control unitperforms PWM control on the LEDin the third light emission color so that its brightness is changed in accordance with the identified lighting pattern (Step S). With this control, the user is notified of the fact that a voice is required to be input again because the reception of the voice operation has failed. After that, the flow returns to Step S.

16 13 19 18 18 12 18 17 18 12 14 Meanwhile, when the voice recognition processing performed in the voice recognition processing unithas been successful (YES in Step S), the main control unitnotifies the LED control unitof the language to be recognized, the state “recognition is successful” of the voice recognition processing, and the light emission color “first light emission color” to instruct the LED control unitto turn on the LED. In response to this, the LED control unitidentifies, from the lighting pattern storage unit, a lighting pattern that is associated with the language to be recognized and the state “recognition is successful” of the voice recognition processing. Then, the LED control unitperforms PWM control on the LEDin the first light emission color so that its brightness is changed in accordance with the identified lighting pattern (Step S). With this control, the user is notified of the fact that the reception of the voice operation has been successful.

19 16 15 19 4 13 4 19 14 4 19 14 19 14 19 14 6 After that, the main control unitexecutes a command indicated by a processing result of the voice recognition processing received from the voice recognition processing unit(Step S). For example, when the processing result of the voice recognition processing is a music name, the main control unitaccesses the media servervia the network interface unitto download audio data having the music name from the media server. When the processing result of the voice recognition processing is to request start of reproduction (for example, “saisei” in the case of Japanese, and “Play” in the case of English), the main control unitinstructs the audio reproduction unitto start reproduction of the audio data downloaded from the media server. When the processing result of the voice recognition processing is to request stop of reproduction (for example, “teishi” in the case of Japanese, and “Stop” in the case of English), the main control unitinstructs the audio reproduction unitto stop reproduction of the audio data. When the processing result of the voice recognition processing is to request an increase of the volume of a reproduced sound (for example, “onryō zōdai” in the case of Japanese, and “Volume up” in the case of English), the main control unitinstructs the audio reproduction unitto increase the volume of the reproduced sound. When the processing result of the voice recognition processing is to request a decrease of the volume of a reproduced sound (for example, “onryō genshō” in the case of Japanese, and “Volume down” in the case of English), the main control unitinstructs the audio reproduction unitto decrease the volume of the reproduced sound. After that, the flow returns to Step S.

16 11 7 8 19 18 12 18 12 9 1 Meanwhile, when no voice is input to the voice recognition processing unitthrough the microphone(NO in Step S), and a time-out occurs due to the elapse of a predetermined period of time (YES in Step S), the main control unitinstructs the LED control unitto turn off the LED. In response to this, the LED control unitturns off the LED(Step S). After that, the flow returns to Step S.

One embodiment of the present invention is described above.

12 12 1 1 In this embodiment, the plurality of lighting patterns in which the brightness is changed on the time axis are provided, and the PWM control is performed on the LEDthrough use of the plurality of lighting patterns, to thereby express the state of the voice recognition processing. Accordingly, it is possible to express the detailed state of the voice recognition processing with only one LEDarranged on the front surface of the wireless speaker. Therefore, according to this embodiment, it is possible to provide the wireless speakerhaving the voice operation reception function, which is capable of notifying of the detailed state of the voice recognition processing without affecting the audio reproduction environment, and is low in cost and excellent in the degree of freedom in design.

12 Further, in this embodiment, for each state of the voice recognition processing, the lighting pattern having the intensity of brightness synchronized with intonation and accent of the voice message representing the state of the voice recognition processing is used. Accordingly, the user can understand, through the LEDsubjected to PWM control in accordance with a lighting pattern, the intonation and accent of the voice message corresponding to the lighting pattern, and hence the user can intuitively grasp the state of the voice recognition processing.

12 12 12 12 Further, in this embodiment, when the state of the voice recognition processing is “waiting for input of voice”, the LEDis turned on in the second light emission color, and when the state of the voice recognition processing is “recognition has failed”, the LEDis turned on in the third light emission color and in a corresponding lighting pattern. Meanwhile, when the state of the voice recognition processing is a state other than “waiting for input of voice” and “recognition has failed”, that is, any one of “voice operation reception is started”, “voice is input”, “voice is being recognized”, and “recognition is successful”, the LEDis turned on in the first light emission color and in a corresponding lighting pattern. Therefore, according to this embodiment, the user can also grasp the state of the voice recognition processing from the light emission color of the LED.

11 11 Further, in this embodiment, when a recognition result of the voice recognition processing performed on a voice input to the microphoneindicates a request to start reception of the voice operation, voice recognition dictionary data to be used in the voice recognition processing performed on a voice subsequently input to the microphoneis narrowed down to voice recognition dictionary data of a language (language to be recognized) forming a sentence represented by the recognition result. Therefore, according to this embodiment, it is possible to efficiently perform the voice recognition processing by reducing the time required for the voice recognition processing.

The present invention is not limited to the embodiment described above, and various modifications may be made thereto within the scope of the gist of the present invention.

12 12 For example, in the embodiment described above, when the PWM control is performed on the LEDin accordance with a lighting pattern, the light emission color of the LEDmay be changed in accordance with the intensity of brightness. With this configuration, the user can more easily understand the intonation and accent of a voice message corresponding to the lighting pattern, and hence the user can more intuitively grasp the state of the voice recognition processing.

13 12 14 5 FIG. 5 FIG. Further, in the embodiment described above, when the voice recognition processing performed on the input voice has been successful (YES in Step Sof), the PWM: control is performed on the LEDin accordance with a lighting pattern corresponding to the language to be recognized and the state “recognition is successful” of the voice recognition processing (Step Sof). The lighting pattern has the intensity of brightness synchronized with the intonation and accent of a voice message representing the state “recognition is successful” of the voice recognition processing in the language to be recognized, but a command obtained as a result of the voice recognition may be included in the voice message to notify the user of the fact that the command obtained as a result of the voice recognition is to be executed.

12 12 12 12 12 For example, when a command obtained as a result of the voice recognition is a music name, the PWM control is performed on the LEDthrough use of a lighting pattern having the intensity of brightness synchronized with intonation and accent of voice data (e.g., “It is downloaded”) representing the downloading of audio data having the music name. When a command obtained as a result of the voice recognition is to start reproduction, the PWM control is performed on the LEDthrough use of a lighting pattern having the intensity of brightness synchronized with intonation and accent of voice data (e.g., “It is played”) representing the start of reproduction. When a command obtained as a result of voice recognition is to stop reproduction, the PWM control is performed on the LEDthrough use of a lighting pattern having the intensity of brightness synchronized with intonation and accent of voice data (e.g., “It is stopped”) representing the stop of reproduction. When a command obtained as a result of the voice recognition is to increase the volume of a reproduced sound, the PWM control is performed on the LEDthrough use of a lighting pattern having the intensity of brightness synchronized with intonation and accent of voice data (e.g., “Volume is turned up”) representing the increase of the volume. When a command obtained as a result of the voice recognition is to decrease the volume of a reproduced sound, the PWM control is performed on the LEDthrough use of a lighting pattern having the intensity of brightness synchronized with intonation and accent of voice data (e.g., “Volume is turned down”) representing the decrease of the volume.

1 Further, in the embodiment described above, the case in which the wireless speakerperforms voice recognition processing is taken as an example for description. However, the present invention is not limited thereto.

3 11 19 13 19 15 16 1 For example, voice recognition processing may be performed through use of an external server (voice recognition server) connected to the network. Specifically, when a voice is input to the microphone, the main control unittransmits voice data on the input voice to the external server via the network interface unitto request the external server to perform voice recognition processing on the voice data. Then, the main control unitacquires a processing result of the voice recognition processing from the external server. In this case, the voice recognition dictionary data storage unitand the voice recognition processing unitcan be omitted from the wireless speaker.

1 1 1 As another example, voice recognition processing may be performed through use of a voice input terminal provided separately from the wireless speaker. Specifically, the voice input terminal performs voice recognition processing on a voice input thereto to receive a voice operation from a user. Then, the voice input terminal transmits to the wireless speakera control command corresponding to the received voice operation. With this control, the wireless speakerperforms an operation corresponding to the voice operation received by the voice input terminal from the user.

11 11 11 Further, in the embodiment described above, when a recognition result of voice recognition processing performed on a voice input to the microphoneindicates a request to start reception of a voice operation, voice recognition dictionary data to be used in the voice recognition processing performed on a voice subsequently input to the microphoneis narrowed down to voice recognition dictionary data of a language (language to be recognized) forming a sentence represented by the recognition result. However, the present invention is not limited thereto. Even in voice recognition processing performed on a voice input to the microphoneafter the request to start reception of the voice operation is made, the voice recognition processing may be performed through use of pieces of voice recognition dictionary data of all available languages without narrowing down the voice recognition dictionary data.

1 2 FIG. Further, in the embodiment described above, the functional configuration of the wireless speakerillustrated inmay be implemented by hardware through use of an integrated logic IC, for example, an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA), or may be implemented by software through use of a computer, for example, a digital signal processor (DSP). Alternatively, the functional configuration may be implemented by a general computer, for example, a personal computer including a central processing unit (CPU), a memory, a flash memory, a hard disk drive or another such auxiliary storage device, a wireless LAN adaptor or another such communication device, a speaker, and a microphone, causing the CPU to load a predetermined program into the memory from the auxiliary storage device and execute the program.

1 4 10 Further, in the embodiment described above, as the audio device, the wireless speakerconfigured to download audio data from the media serverand reproduce the audio data to output the audio data as a sound from the speakeris taken as an example for description. However, the present invention is not limited thereto. The present invention is widely applicable to audio devices having a voice operation reception function.

1 2 3 4 10 11 12 13 14 15 16 17 18 19 : wireless speaker,: access point,: network,: media server,: speaker,: microphone,: LED,: network interface unit,: audio reproduction unit,: voice recognition dictionary data storage unit,: voice recognition processing unit,: lighting pattern storage unit,: LED control unit,main control unit