A combination device (305) according to the present invention includes: a detection unit (501) that detects active coded bitstreams that are effective coded bitstreams from a plurality of coded bitstreams (116) within a predetermined time period; a first combining unit (504) that combines, from a plurality of downmix sub-streams (115) included in the coded bitstreams (116), only downmix sub-streams (115) included in the active coded bitstreams so as to generate a combined downmix sub-stream (121); and a second combining unit (506) that combines, from a plurality of parameter sub-streams (113) included in the coded bitstreams (116), only parameter sub-streams (113) included In the active coded bitstreams so as to generate a combined parameter sub-stream (122).
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A combination device that combines a plurality of coded bitstreams transmitted from a plurality of sites, the plurality of coded bitstreams each including a downmix sub-stream and a parameter sub-stream, the downmix sub-stream being generated by down-mixing a plurality of input audio signals and having a total number of signals that is less than a total number of signals in the plurality of input audio signals, and the parameter sub-stream being to be used to decode the downmix sub-stream into the plurality of input audio signals which have the total number of signals that is more than the total number of signals in the downmix sub-stream, said combination device comprising: a detection unit configured to detect an active coded bitstream from the plurality of coded bitstreams within a predetermined time period, the active coded bitstream being a coded bitstream that has a sound volume that is larger than a predetermined threshold value; a first combining unit configured to combine, from among a plurality of downmix sub-streams, only plural downmix sub-streams included in plural active coded bitstreams, so as to generate a combined downmix sub-stream; a second combining unit configured to combine, from among a plurality of parameter sub-streams, only plural parameter sub-streams included in the plural active coded bitstreams, so as to generate a combined parameter sub-stream; and a transmission unit configured to transmit, to the plurality of sites, a combined bit stream that includes the combined downmix sub-stream and the combined parameter sub-stream; wherein said second combining unit includes a parameter basis unifying unit configured to convert different parameter presentation bases of the plural parameter sub-streams to a single unified parameter presentation basis, and generate plural unified parameters based on the single unified parameter presentation basis, when the plural parameter sub-streams are expressed by the different parameter presentation bases, and wherein said second combining unit is configured to combine the plural unified parameters so as to generate the combined parameter sub-stream.
A device combines audio streams from multiple locations. Each location sends a coded bitstream containing a downmix (a simplified audio version with fewer signals than the original) and parameter data (used to reconstruct the original audio from the downmix). The device detects which bitstreams are "active" based on sound volume exceeding a threshold. It combines only the downmixes from active streams into a combined downmix and combines only the parameter data from active streams into a combined parameter stream. The device transmits a combined bitstream containing these combined streams back to all locations. If the parameter data uses different formats, the device converts them to a unified format before combining.
2. The combination device according to claim 1 , wherein said first combining unit includes: a decoding unit configured to decode, from among the plurality of downmix sub-streams, only the plural downmix sub-streams included in the plural active coded bitstreams, so as to generate plural decoded downmix sub-streams; an adding unit configured to add the plural decoded downmix sub-streams together so as to generate at least one intermediate combined downmix sub-stream; and a coding unit configured to code the at least one intermediate combined downmix sub-stream so as to generate at least one combined downmix sub-stream.
The combination device described previously combines downmix streams by first decoding only the downmix sub-streams from active bitstreams. These decoded downmix streams are then added together to create an intermediate combined downmix stream. Finally, this intermediate stream is coded (re-encoded) to generate the final combined downmix stream included in the combined bitstream sent back to all sites.
3. The combination device according to claim 1 , wherein said first combining unit is configured to combine, for each of the plurality of sites, a set of downmix sub-streams transmitted from the plurality of sites so as to generate a combined downmix sub-stream to be transmitted to the site for which the set of downmix sub-streams is combined, the set of downmix sub-streams including the downmix sub-streams included in the plural active coded bitstreams transmitted from the plurality of sites other than the site for which the set of downmix sub-streams is combined, wherein said second combining unit is configured to combine, for each of the plurality of sites, a set of parameter sub-streams transmitted from the plurality of sites so as to generate a combined parameter sub-stream to be transmitted to the site for which the set of parameter sub-streams is combined, the set of parameter sub-streams including the parameter sub-streams included in the plural active coded bitstreams transmitted from the plurality of sites other than the site for which the set of parameter sub-streams is combined, wherein said transmission unit is configured to transmit, to each of the plurality of sites, a combined bitstream that includes (i) the combined downmix sub-stream which is generated for the site by the first combining unit and (ii) the combined parameter sub-stream which is generated for the site by the second combining unit, and wherein when a number of inactive coded bitstreams is two or more, the inactive coded bitstreams being coded bitstreams other than the plurality of active coded bitstreams in the plurality of coded bitstreams, (1) said first combining unit is configured to combine the plural downmix sub-streams included in all of the plural active coded bitstreams so as to generate a common combined downmix sub-stream, (2) said second combining unit is configured to combine the plural parameter sub-streams included all of the plural active coded bitstreams so as to generate a common combined parameter sub-stream, and (3) said transmission unit is configured to transmit a common combined bitstream that includes the common combined downmix sub-stream and the common combined parameter sub-stream, to sites which transmit the inactive coded bitstreams that are two or more.
The combination device creates separate combined audio streams for each location by combining only the audio streams sent from *other* locations. For each site, it combines downmixes from active bitstreams from all other sites into a combined downmix for *that* site, and it combines parameter data from active bitstreams from all other sites into a combined parameter stream for *that* site. These combined streams are sent back to the specific site. If multiple sites are inactive, a common combined downmix and parameter stream, created using active bitstreams, is sent to the inactive sites.
4. The combination device according to claim 1 , wherein, when a number of the plural active coded bitstreams is two, said transmission unit is configured to (i) transmit, without a combining process, a first coded bitstream that is one of the plural active coded bitstreams to a site transmitting a second coded bitstream that is other one of the plural active coded bitstreams, and to (ii) transmit the second coded bitstream to a site transmitting the first coded bitstream without a combining process.
In the previous combination device setup, if only two locations are transmitting active audio streams, the device simply forwards each location's audio stream to the other location without any combination process. It transmits the first active coded bitstream to the site transmitting the second active coded bitstream, and vice versa.
5. The combination device according to claim 1 , wherein, when a number of the plural active coded bitstreams is one, said transmission unit is configured to transmit, without a combining process, the active coded bitstream to the plurality of sites except a site transmitting the active coded bitstream.
In the previous combination device setup, if only one location is transmitting an active audio stream, the device forwards that active audio stream to all other locations except for the originating location, without any combination process.
6. The combination device according to claim 1 , wherein said detection unit is configured to detect the active coded bitstream based on information included in each of the plurality of parameter sub-streams.
The combination device determines whether an audio stream is active (exceeding a volume threshold) by analyzing information contained within the parameter sub-stream of each audio stream.
7. The combination device according to claim 1 , wherein said first combining unit is configured to combine the plural downmix sub-streams included in all of the plural active coded bitstreams so as to generate a single combined downmix sub-stream, wherein said second combining unit is configured to combine the plural parameter sub-streams included in all of the plural active coded bitstreams so as to generate a single combined parameter sub-stream, and wherein said transmission unit is configured to transmit, to each of the plurality of sites, a single combined bitstream that includes the single combined downmix sub-stream and the single combined parameter sub-stream.
The combination device combines all downmix sub-streams from all active locations into a single combined downmix sub-stream, and all parameter sub-streams from all active locations into a single combined parameter sub-stream. This single combined bitstream, containing these combined streams, is then transmitted to *every* location.
8. The combination device according to claim 7 , further comprising an additional information generation unit configured to generate, for each of active sites which have transmitted the plural active coded bitstreams, additional information to be used to specify a signal component from signal components in the single combined bitstream, the signal component corresponding a corresponding one of the plural active coded bitstreams which has been transmitted from the each of active sites, wherein said transmission unit is configured to transmit the additional information to the each of active sites.
The device as described previously, which sends a single combined bitstream to every location, includes a unit that generates additional information for each active location. This information identifies the portion of the combined bitstream (the signal component) that corresponds to *that* location's original audio stream. The device then transmits this additional information back to each active location, enabling them to extract their original audio.
9. The combination device according to claim 8 , wherein said additional information generation unit is configured to generate the additional information for each of the active sites, the additional information being to be used to specify a parameter from parameters included in the single combined parameter sub-stream, the parameter corresponding to a corresponding one of the plural parameter sub-streams which has been transmitted from the each of the active sites.
In the previous device, the additional information generated for each active site specifically identifies a parameter within the combined parameter sub-stream that corresponds to the original parameter sub-stream sent from that site.
10. The combination device according to claim 1 , further comprising a parameter basis selection unit configured to select the single unified parameter presentation basis from a plurality of parameter presentation bases, based on a current bit rate available for transmission from said combination device to the plurality of sites.
The combination device selects the unified parameter presentation basis (the format for parameter data) from several options, based on the available bit rate for transmitting data from the device to the various locations. It chooses a format that best fits the available bandwidth.
11. The combination device according to claim 1 , further comprising a parameter basis selection unit configured to select the single unified parameter presentation basis from a plurality of parameter presentation bases, based on a bit cost indicating the number of bits of the combined parameter sub-stream.
The combination device selects the unified parameter presentation basis based on the bit cost (number of bits required) of the resulting combined parameter sub-stream. It selects the format that minimizes the size of the combined parameter data.
12. The combination device according to claim 2 , wherein each of the plural downmix sub-stream is generated by down-mixing the plurality of input audio signals, transformed into a spectrum domain, and coded, wherein said decoding unit is configured to decode the plural downmix sub-streams to generate the plural decoded downmix sub-streams in the spectrum domain, and wherein said adding unit is configured to add the plural decoded downmix sub-streams in the spectrum domain together so as to generate the at least one intermediate combined downmix sub-stream.
In the downmix audio combination device, each downmix sub-stream is generated by transforming the original audio signals into the frequency domain (spectrum), downmixing them, and then coding the downmixed spectrum. The decoding unit decodes these downmix sub-streams back into the frequency domain. The adding unit then adds these decoded frequency-domain downmixes together to create the intermediate combined downmix sub-stream, which is also in the frequency domain.
13. The combination device according to claim 12 , wherein said first combining unit further includes a scaling unit configured to scale the at least one intermediate combined downmix sub-stream so that spectrum power of the plural decoded downmix sub-streams is preserved in the at least one intermediate combined downmix sub-stream, and wherein said coding unit is configured to code the at least one intermediate combined downmix sub-stream scaled by said scaling unit so as to generate the combined downmix sub-stream.
Building on the frequency-domain audio combiner above, the device includes a scaling unit that adjusts the intermediate combined downmix sub-stream to preserve the original spectrum power of the individual decoded downmix sub-streams. This scaled stream is then coded (re-encoded) to generate the final combined downmix stream.
14. The combination device according to claim 12 , wherein said second combining unit includes: an inverse quantization unit configured to inversely quantize the plural parameter sub-streams so as to generate plural inversely-quantized parameters; a parameter combining unit configured to combine the plural inversely-quantized parameters so as to generate a combined parameter; a parameter update unit configured to update a part of parameters included in the combined parameter so as to generate a updated parameter; and a quantization unit configured to quantize (a) a parameter except the part of parameters included in the combined parameter and (b) the updated parameter so as to generate the combined parameter sub-stream.
In the frequency-domain audio combination device, the parameter combining process involves several steps. First, an inverse quantization unit converts the parameter sub-streams into inversely-quantized parameters. These parameters are then combined to create a combined parameter. A parameter update unit then updates a portion of the combined parameter. Finally, a quantization unit quantizes both the updated portion and the remaining parameters to generate the final combined parameter sub-stream.
15. A telecommunication system comprising: a plurality of sites each including a coding device that generates a coded bitstream that includes a downmix sub-stream and a parameter sub-stream, the downmix sub-stream being generated by down-mixing a plurality of input audio signals, and the parameter sub-stream being to be used to reconstruct the plurality of input audio signals from the downmix sub-stream; and the combination device according to claim 1 which combines a plurality coded bitstreams including the coded bitstream which are transmitted from said plurality of sites so as to generate a combined bitstream, and transmits the combined bitstream to each of said plurality of sites, wherein each of said plurality of sites further includes a decoding device that decodes the combined bitstream to generate output audio signals.
A telecommunication system includes multiple locations, each with a coding device that generates a coded bitstream including a downmix and parameter data. The system also includes a combination device, as previously described, that combines these bitstreams and sends the combined stream back to each location. Each location also includes a decoding device that decodes the combined bitstream to generate the final output audio signals.
16. A telecommunication system comprising: a plurality of sites each including a coding device that generates a coded bitstream that includes a downmix sub-stream and a parameter sub-stream, the downmix sub-stream being generated by down-mixing a plurality of input audio signals, and the parameter sub-stream being to be used to reconstruct the plurality of input audio signals from the downmix sub-stream; and the combination device according to claim 8 which combines a plurality of coded bitstreams including the coded bitstream which are transmitted from said plurality of sites so as to generate a combined bitstream, and transmits the combined bitstream to each of said plurality of sites, wherein each of said plurality of sites further includes a decoding device that decodes the combined bitstream to generate output audio signals, and wherein said decoding device generates, based on the additional information, the output audio signals from which a signal component is removed from signal components in the single combined bitstream, the signal component corresponding to the coded bitstream transmitted from a corresponding one of said plurality of sites which includes said decoding device.
This telecommunication system is similar to the previous one, but uses the combination device that generates additional information to identify the source of each signal component. Each location's decoding device uses this additional information to *remove* its own original audio signal component from the combined audio stream, effectively preventing feedback. The decoding device generates output audio signals based on the combined bitstream with the signal component corresponding to the coded bitstream transmitted from the same site removed.
17. A combining method of combining a plurality of coded bitstreams transmitted from a plurality of sites, the plurality of coded bitstreams each including a downmix sub-stream and a parameter sub-stream, the downmix sub-stream being generated by downmixing a plurality of input audio signals and having a total number of signals that is less than a total number of signals in the plurality of input audio signals, and the parameter sub-stream being to be used to decode the downmix sub-stream into the plurality of input audio signals which have the total number of signals that is more than the total number of signals in the downmix sub-stream, said combining method comprising: detecting an active coded bitstream from the plurality of coded bitstreams within a predetermined time period, the active coded bitstream being a coded bitstream that has a sound volume that is larger than a predetermined threshold value; combining, from among a plurality of downmix sub-streams, only plural downmix sub-streams included in plural active coded bitstreams including the active bitstream, so as to generate a combined downmix sub-stream; combining, from among a plurality of parameter sub-streams, only plural parameter sub-streams included in the plural active coded bitstreams, so as to generate a combined parameter sub-stream; and transmitting, to each of the plurality of sites, a combined bit stream that includes the combined downmix sub-stream and the combined parameter sub-stream, wherein said combining only the plural downmix sub-streams includes converting different parameter presentation bases of the plural parameter sub-streams to a single unified parameter presentation basis, and generating plural unified parameters based on the single unified parameter presentation basis, when the plural parameter sub-streams are expressed by the different parameter presentation bases, and wherein said combining only the plural downmix sub-streams combines the plural unified parameters so as to generate the combined parameter sub-stream.
This invention relates to audio signal processing, specifically methods for combining multiple coded bitstreams from different sites to reconstruct a multi-channel audio output. The problem addressed is efficiently merging audio signals from multiple sources while maintaining audio quality and reducing computational complexity. Each coded bitstream includes a downmix sub-stream (a compressed version of multiple input audio signals) and a parameter sub-stream (metadata used to decode the downmix back into the original multi-channel signals). The method detects active bitstreams (those with sound volumes above a threshold) within a set time period. Only the downmix and parameter sub-streams from these active bitstreams are combined. If the parameter sub-streams use different formats, they are converted to a unified format before merging. The combined downmix and parameter sub-streams are then transmitted back to the original sites. This approach ensures that only relevant audio signals are processed, reducing redundancy and improving efficiency in multi-site audio conferencing or broadcasting systems.
18. A non-transitory computer-readable recording medium for use in a computer, the recording medium having a computer program recorded thereon for causing the computer to execute the combining method according to claim 17 .
A non-transitory computer-readable medium stores a program that, when executed, performs the audio combining method as previously described, including active stream detection, downmix and parameter stream combination, format unification, and transmission of the combined stream.
19. An integrated circuit that combines a plurality of coded bitstreams transmitted from a plurality of sites, the plurality of coded bitstreams each including a downmix sub-stream and a parameter sub-stream, the downmix sub-stream being generated by downmixing a plurality of input audio signals and having a total number of signals that is less than a total number of signals in the plurality of input audio signals, and the parameter sub-stream being to be used to decode the downmix sub-stream into the plurality of input audio signals which have the total number of signals that is more than the total number of signals in the downmix sub-stream, said integrated circuit comprising: a detection unit configured to detect an active coded bitstream from the plurality of coded bitstreams within a predetermined time period, the active coded bitstream being a coded bitstream that has a sound volume that is larger than a predetermined threshold value; a first combining unit configured to combine, from among a plurality of downmix sub-streams, only plural downmix sub-streams included in plural active coded bitstreams including the active bitstream, so as to generate a combined downmix sub-stream; a second combining unit configured to combine, from among a plurality of parameter sub-streams, only plural parameter sub-streams included in the plural active coded bitstreams, so as to generate a combined parameter sub-stream; and a transmission unit configured to transmit, to each of the plurality of sites, a combined bit stream that includes the combined downmix sub-stream and the combined parameter sub-stream, wherein said second combining unit includes a parameter basis unifying unit configured to convert different parameter presentation bases of the plural parameter sub-streams to a single unified parameter presentation basis, and generate plural unified parameters based on the single unified parameter presentation basis, when the plural parameter sub-streams are expressed by the different parameter presentation bases, and wherein said second combining unit is configured to combine the plural unified parameters so as to generate the combined parameter sub-stream.
An integrated circuit (chip) combines audio streams from multiple locations. Each location sends a coded bitstream containing a downmix and parameter data. The chip detects active streams based on volume, combines downmixes from only active streams into a combined downmix, and combines parameter data from only active streams into a combined parameter stream. The combined streams are sent back to all locations. If parameter data formats differ, the chip converts them to a unified format before combining.
20. The combination device according to claim 1 , wherein each of the different parameter presentation bases is a pattern of dividing a parameter tile.
In the audio combination device, the different parameter presentation bases (formats) mentioned previously each represent a different pattern for dividing a parameter tile, essentially organizing the parameter data in different ways.
21. The combination device according to claim 1 , wherein said parameter basis unifying unit is configured (i) to determine whether or not the plural parameter sub-streams are expressed by the different parameter presentation bases, and (ii) when the plural parameter sub-streams are expressed by the different parameter presentation bases, to convert the different parameter presentation bases of the plural parameter sub-streams into the single unified parameter presentation basis to unify the different parameter presentation bases.
In the audio combination device, the parameter basis unifying unit first determines if the parameter sub-streams from different sites use different formats. If they do, the unit converts these different formats to a single, unified format to ensure compatibility before combining the parameter data.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
February 4, 2010
August 6, 2013
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.