Encoding an information signal

Notice: More than one reissue application has been filed on Jul. 24, 2023 for the reissue of U.S. Pat. No. 8,126,721, including pending subject reissue application Ser. No. 18/225,619, allowed reissue patent application Ser. No. 18/225,488, and pending reissue application Ser. Nos. 18/225,505, 18/225,527, 18/225,553, and 18/225,568.

This application, Ser. No. 18/225,619, filed Jul. 24, 2023, is a reissue application of issued U.S. Pat. No. 8,126,721, issued Feb. 28, 2012, whichclaims priority from Provisional U.S. patent application Ser. No. 60/862,033, which was filed on Oct. 18, 2006, and is incorporated herein its entirety by reference.

This application claims priority from Provisional U.S. patent application Ser. No. 60/862,033, which was filed on Oct. 18, 2006, and is incorporated herein in its entirety by reference.

The present invention relates to information signal encoding such as audio encoding, and, in that context, in particular to SBR (spectral band replication) encoding.

In applications having a very small bit rate available, it is known, in the context of encoding audio signals, to use an SBR technique for encoding. Only the low-frequency portion is encoded fully, i.e. at an adequate temporal and spectral resolution. For the high-frequency portion, only the spectral envelope, or the envelope of the spectral temporal curve of the audio signal, is detected and encoded. On the decoder side, the low-frequency portion is retrieved from the encoded signal and is subsequently used to reconstruct, or “replicate”, the high-frequency portion therefrom. However, to adapt the energy of the high-frequency portion, which has thus been preliminarily reconstructed, to the actual energy within the high-frequency portion of the original audio signal, the spectral envelope transmitted is used, on the decoder side, for spectral weighting of the high-frequency portion reconstructed preliminarily.

For the above effort to be worthwhile, it is important, of course, that the number of bits used for transmitting the spectral envelopes be as small as possible. It is therefore desirable for the temporal grid within which the spectral envelope is encoded to be as coarse as possible. On the other hand, however, too coarse a grid leads to audible artefacts, which is notable, in particular, with transients, i.e. at locations where the high-frequency portions will predominate rather than, as usual, the low-frequency portions, or where there is at least a rapid increase in the amplitude of the high-frequency portions. In audio signals, such transients correspond, for example, to the beginnings of a note, such as actuation of a piano string or the like. If the grid is too coarse over the time period of a transient, this may lead to audible artefacts in the decoder-side reconstruction of the entire audio signal. For, as one knows, on the decoder side, the high-frequency signal is reconstructed from the low-frequency portion in that, within the grid area, the spectral energy of the decoded low-frequency portion is normalized and then adapted to the spectral envelope transmitted by means of weighting. In other words, spectral weighting is simply performed within the grid area so as to reproduce the high-frequency portion from the low-frequency portion. However, if the grid area around the transient is too large, a lot of energy will be located, within this grid area, in addition to the energy of the transient, in the background and/or chord portion in the low-frequency portion which is used for reproducing the high-frequency portion. Said low-frequency portion is co-amplified by the weighting factor, even though this does not result in a good estimation of the high-frequency portion. Across the entire grid area, this will lead to an audible artefact which, in addition, will set in even before the actual transient. This problem may also be referred to as “pre-echo”.

The problem could be solved when the grid area around the transient is fine enough so that the transient/background ratio of the part of the low-frequency portion within this grid area is improved. Small grid areas or small grid boundary distances, however, are obstacles on the way to the above-outlined desire for a low bit consumption for encoding the spectral envelopes.

In the ISO/IEC 14496-3 standard—simply referred to as “the standard” below—an SBR encoding is described in the context of the AAC encoder. The AAC encoder encodes the low-frequency portion in a frame-by-frame manner. For each such SBR frame, the above-specified time and frequency resolution is defined at which the spectral envelope of the high-frequency portion is encoded in this frame. To address the problem that transients may also fall on SBR frame boundaries, the standard allows that the temporal grid may temporarily be defined such that the grid boundaries do not necessarily coincide with the frame boundaries. Rather, in this standard, the encoder transmits, per frame, a syntax element bs_frame_class to the decoder, said syntax element indicating per frame whether the temporal grid of the spectral envelope gridding for the respective frame is defined precisely between the two frame boundaries or between boundaries which are offset from the frame boundaries, specifically at the front and/or at the back. Overall, there are four different classes of SBR frames, i.e. FIXFIX, FIXVAR, VARFIX and VARVAR. The syntax used by the encoder in the standard to define the grid per SBR frame is depicted in a pseudo code representation in. In particular, in the representation of, those syntax elements which are actually encoded and/or transmitted by the encoder are printed in bold type in, the number of the bits used for transmission and/or encoding being indicated in the second column from the right in the respective row. As may be seen, the syntax element bs_frame_class which has just been mentioned is initially transmitted for each SBR frame. As a function thereof, further syntax elements will follow which, as will be illustrated, define the temporal resolution and/or gridding. If, for example, the 2-bits syntax element bs_frame_class indicates that the SBR frame in question is a FIXFIX SBR frame, the syntax element tmp which defines the number of grid areas in this SBR frame, and/or which defines the number of envelopes, as 2will be transmitted as the second syntax element. The syntax element bs_amp_res, which is used for the quantization step size for encoding the spectral envelope in the current SBR frame, is automatically adjusted as a function of bs_num_env, and is not encoded or transmitted. Finally, for a FIXFIX frame, a bit is transmitted for determining the frequency resolution of the grid bs_freq_res. FIXFIX frames are defined precisely for one frame, i.e. the grid boundaries coincide with the frame boundaries as defined by the AAC encoder.

This is different for the other three classes. For FIXVAR, VARFIX and VARVAR frames, syntax elements bs_var_bord_1 and/or bs_bar_bod_0 are transmitted to indicate the number of time slots, i.e. the time units wherein the filter bank for spectral decomposition of the audio signal operates, by which are offset relative to the normal frame boundaries. As a function thereof, syntax elements bs_num_rel_1 and an associated tmp and/or bs_num_rel_0 and an associated tmp are also transmitted so as to define a number of grid areas, or envelopes, and the size thereof from the offset frame boundary. Finally, a syntax element bs_pointer is also transmitted within the variable SBR frames, said syntax element pointing to one of the defined envelopes and serving to define one or two noise envelopes for determining the noise portion within the frame as a function of the spectral envelope gridding, which, however, shall not be explained in detail below in order to simplify the representation. Finally, the respective frequency resolution is determined, namely by a respective one-bit syntax element bs_freq_res per envelope, for all grid areas and/or envelopes in the respective variable frames.

represents, by way of example, a FIXFIX frame wherein the syntax element tmp is 1, so that the number of envelopes is bs_num_env 2=2. Init shall be assumed that the time axis extends from the left to the right in a horizontal manner. An SBR frame, i.e. one of the frames in which the AAC encoder encodes the low-frequency portion, is indicated by reference numeralsin. As can be seen, the SBR framehas a length of 16 QMF slots, the QMF slots being, as has been mentioned, the time slots in which units the analysis filter bank operates, the QMF slots being indicated by boxin. In FIXFIX frames, the envelopes, or grid areas,a andb, i.e. two in number here, have the same length within the SBR frames, so that a time grid and/or envelope boundaryis defined precisely in the center of the SBR frame. In this manner the exemplary FIXFIX frame ofdefines that a spectral distribution for the grid area, or the envelope,a, and a further one for envelope, is temporally determined from the spectral values of the analysis filter bank. The envelopes, or grid areas,a andb thus specify the grid in which the spectral envelope is encoded and/or transmitted.

By comparison,shows a VARVAR frame. SBR frameand associated QMF slotsare indicated again. For this SBR frame, however, syntax elements bs_var_bord_0 and/or bs_var_bord_1 have defined that the envelopesa′,b′ andc′ associated therewith are not to start at the SBR frame starta and/or to end at the SBR frame endb. Rather, one may see fromthat the previous SBR frame (not to be seen in) has already been extended two QMF slots beyond the SBR frame starta of the current SBR frame, so that the last envelopeof the preceding SBR frame still extends into the current SBR frame. The last envelopec′ of the current frame also extends beyond the SBR frame end of the current SBR frame, namely, by way of example, also by two QMF slots here. In addition, one can also see here, by way of example, that the syntax elements of the VARVAR frame bs_num_rel_0 and bs_num_rel_1 are adjusted to 1, respectively, with the additional information that the envelopes thus defined have a length of four QMF slots at the start and at the end of the SBR frame, i.e.a′ andb′ in accordance with tmp=1, so as to extend from the frame boundaries into the SBR frameby this number of slots. The remaining space of the SBR framewill then be occupied by the remaining envelope, in this case the third envelopeb′.

By having T in one of the QMF slots,indicates, by way of example, the reason why a VARVAR frame has been defined here, namely because the transient position T is located close to the SBR frame endb, and because there probably was a transient (not to be seen) also in the SBR frame preceding the current one.

The standardized version in accordance with ISO/ICE 14496-3 thus involves overlapping of two successive SBR frames. This enables setting the envelope boundaries in a variable manner, irrespective of the actual SBR frame boundaries in accordance with the waveform. Transients may thus be enveloped by envelopes of their own, and their energy may be cut off from the remaining signal. However, an overlap also involves an additional system delay, as was illustrated above. In particular, four frame classes are used for signaling in the standard. In the FIXFIX class, the boundaries of the SBR envelopes coincide with the boundaries of the core frame, as is shown in. The FIXFIX class is used when no transient is present in this frame. The number of envelopes specifies their equidistant distribution within the frame. The FIXVAR class is provided when there is a transient in the current frame. Here, the respective set of envelopes thus starts at the SBR frame boundary and ends, in a variable manner, in the SBR transmission area. The VARFIX class is provided for the event that a transient is not located in the current, but in the previous frame. The sequence of envelopes from the last frame here is continued by a new set of envelopes which ends at the SBR frame boundary. The VARVAR class is provided for the case that a transient is present both in the last frame and in the current frame. Here, a variable sequence of envelopes is continued by a further variable sequence. As has been described above, the boundaries of the variable envelopes are transmitted in relation to one another.

Even though the number of QMF slots by which the boundaries may be offset relative to the fixed frame boundaries by means of the syntax elements bs_var_bord_0 and bs_var_bord_1, this possibility results in a delay on the decoder side due to the occurrence of envelopes which extend beyond SBR frame boundaries and thus necessitate the formation and/or averaging of spectral signal energies across SBR frame boundaries. However, this time delay is not tolerable in some applications, such as in applications in the field of telephony or other live applications which rely on the time delay caused by the encoding and decoding to be small. Even though the occurrence of pre-echoes is thus prevented, the solution is not suitable for applications necessitating a short delay time. In addition, the number of bits needed for transmitting the SBR frames in the above-described standard is relatively high.

According to an embodiment, an encoder may have a low-frequency portion encoder for encoding a low-frequency portion of an information signal in units of frames of the information signal; a localizer for localizing transients within the information signal; an associator for, as a function of the localization, associating a respective reconstruction mode from among at least two possible reconstruction modes with the frames of the information signal, and, for frames which have associated therewith a first one of the at least two possible reconstruction modes, associating a respective transient position indication with these frames; and a generator for generating a representation of a spectral envelope of a high-frequency portion of the information signal in a temporal grid which depends on reconstruction modes associated with the frames, such that frames which have the first one of the at least two possible reconstruction modes associated therewith, the frame boundaries of these frames coincide with grid boundaries of the grid, and the grid boundaries of the grid within these frames depend on the transient position indication; and a combiner for combining the encoded low-frequency portion, the representation of the spectral envelope and information on the associated reconstruction modes and the transient position indications into an encoded information signal.

According to another embodiment, a decoder may have an extractor for extracting, from the encoded information signal, an encoded low-frequency portion of an information signal, a representation of a spectral envelope of a high-frequency portion of the information signal, information on reconstruction modes associated with frames of the information signal and corresponding with one, respectively, of at least two reconstruction modes, and transient position indications associated with frames, in each case, which have a first one of the at least two reconstruction modes associated with them; a low-frequency portion decoder for decoding the encoded low-frequency portion of the information signal in units of frames of the information signal; a provider for providing a preliminary high-frequency portion signal on the basis of the decoded low-frequency portion; and an adaptor for spectrally adapting the preliminary high-frequency portion signal to the spectral envelopes by means of spectral weighting of the preliminary high-frequency portion signal as a function of the representation of the spectral envelopes in a temporal grid which depends on the reconstruction modes associated with the frames, such that for frames having the first one of the at least two possible reconstruction modes associated with them, the frame boundaries of these frames coincide with grid boundaries of the grid, and the grid boundaries of the grid within these frames depend on the transient position indication.

According to another embodiment, an encoded information signal may have an encoded low-frequency portion of an information signal; a representation of a spectral envelope of a high-frequency portion of an information signal; and of information on reconstruction modes which are associated with frames of the information signal and each correspond to one of at least two reconstruction modes, and transient position indications each associated with frames which have a first one of the at least two reconstruction modes associated with them, such that the information signal may be obtained from the encoded information signal by: decoding the encoded low-frequency portion of the information signal in units of frames of the information signal; providing a preliminary high-frequency portion signal on the basis of the decoded low-frequency portion; and spectrally adapting the preliminary high-frequency portion signal to the spectral envelopes by spectrally weighting the preliminary high-frequency portion signal as a function of the representation of the spectral envelopes in a temporal grid which depends on the reconstruction modes associated with the frames, such that for frames which have the first one of the at least two possible reconstruction modes associated with them, the frame boundaries of these frames coincide with grid boundaries of the grid, and the grid boundaries of the grid within these frames depend on the transient position indication.

According to another embodiment, a method of encoding may have the steps of encoding a low-frequency portion of an information signal in units of frames of the information signal; localizing transients within the information signal; associating, as a function of the localization, a respective reconstruction mode from among at least two possible reconstruction modes with the frames of the information signal, and, for frames which have associated therewith a first one of the at least two possible reconstruction modes, associating a respective transient position indication with these frames; and generating a representation of a spectral envelope of a high-frequency portion of the information signal in a temporal grid which depends on the reconstruction modes associated with the frames, such that frames which have the first one of the at least two possible reconstruction modes associated therewith, the frame boundaries of these frames coincide with grid boundaries of the grid, and the grid boundaries of the grid within these frames depend on the transient position indication; and combining the encoded low-frequency portion, the representation of the spectral envelope and information on the associated reconstruction modes and the transient position indications into an encoded information signal.

According to another embodiment, a method of decoding may have the steps of extracting, from the encoded information signal, an encoded low-frequency portion of an information signal, a representation of a spectral envelope of a high-frequency portion of the information signal and information on reconstruction modes associated with frames of the information signal and corresponding with one, respectively, of at least two reconstruction modes, and transient position indications associated with frames, in each case, which have a first one of the at least two reconstruction modes associated with them; decoding the encoded low-frequency portion of the information signal in units of frames of the information signal; providing a preliminary high-frequency portion signal on the basis of the decoded low-frequency portion; and spectrally adapting the preliminary high-frequency portion signal to the spectral envelopes by means of spectral weighting of the preliminary high-frequency portion signal as a function of the representation of the spectral envelopes in a temporal grid which depends on the reconstruction modes associated with the frames, such that for frames having the first one of the at least two possible reconstruction modes associated with them, the frame boundaries of these frames coincide with grid boundaries of the grid, and the grid boundaries of the grid within these frames depend on the transient position indication.

A finding of the present invention is that the transient problem may be sufficiently addressed, and for this purpose, a further delay on the decoding side may be reduced, if a new SBR frame class is employed wherein the frame boundaries are not offset, i.e. the grid boundaries are still synchronized with the frame boundaries, but wherein a transient position indication is additionally used as a syntax element so as to be used, on the encoder and/or decoder sides, within the frames of this new frame class for determining the grid boundaries within these frames.

In accordance with one embodiment of the present invention, the transient position indication is used such that a relatively short grid area, referred to as transient envelope below, will be defined around the transient position, whereas only one envelope will extend, in the remaining part before and/or behind it, in the frame, from the transient envelope to the start and/or the end of the frame. The number of bits to be transmitted and/or to be encoded for the new class of frames is thus also very small. On the other hand, transients and/or pre-echo problems associated therewith may be sufficiently addressed. Variable SBR frames, such as FIXVAR, VARFIX and VARVAR, will then no longer be needed, so that delays for compensating envelopes which extend beyond SBR frame boundaries will no longer be necessary. In accordance with an embodiment of the present invention, only two frame classes thus will now be admissible, namely a FIXFIX class and this class which has just been described and which will be referred to as LD_TRAN class below.

In accordance with a further embodiment of the present invention, it is not the case that one or several spectral envelopes and/or spectral energy values are transmitted and/or inserted into the encoded information signal for each grid area within the frames of the LD_TRAN class. Specifically, this is not even done when the transient envelope specified in its position within the frame by the transient position indication is located close to the frame boundary which is leading in terms of time, so that the envelope of this LD_TRAN frame, said envelope being located between the frame boundary which is leading in terms of time and the transient envelope, will extend only over a short time period, which is not justified from the point of view of encoding efficiency, since, as one knows, the brevity of this envelope is not due to a transient, but rather to the accidental temporal proximity of the frame boundary and the transient. In accordance with this alternative embodiment, the spectral energy value(s) and the respective frequency resolution of the previous envelope are taken over, therefore, for this envelope concerned, just like the noise portion, for example. Thus, transmission may be omitted, which is why the compression rate is increased. Conversely, losses in terms of audibility are only small, since there is not transient problem at this point. In addition, no delay will occur on the decoder side, since utilization for high-frequency reconstruction is directly possible for all envelopes involved, i.e. envelopes from a previous frame, transient envelope and intervening envelope.

In accordance with a further embodiment, the problems of an unintentionally large amount of data in the occurrence of a transient at the end of an LD_TRAN frame are addressed in that an agreement is reached between the encoder and the decoder as to how far the transient envelope which is located at the trailing frame boundary of the current LD_TRAN frame is to virtually project into the subsequent frame. The decision is made, for example, by means of accessing the tables in the encoder and the decoder alike. In accordance with the agreement, the first envelope of the subsequent frame, such as the single envelope of a FIXFIX frame, is shortened so as to begin only at the end of the virtual extended envelope. The encoder calculates the spectral energy value(s) for the virtual envelope over the entire time period of this virtual envelope, but transmits the result, as it seems, only for the transient envelope, possibly in a manner which is reduced as a function of the ratio of the temporal portion of the virtual envelope in the leading and trailing frames. On the decoder side, the spectral energy value(s) of the transient envelope located at the end are used both for high-frequency reconstruction in this transient envelope and, separate therefrom, for high-frequency reconstruction in the initial extension area in the subsequent frames, in that one and/or several spectral energy value(s) for this area are derived from that, or those, of the transient envelope. “Oversampling” of transients located at frame boundaries is thereby avoided.

In accordance with a further aspect of the present invention, a finding of the present invention is that the transient problems described in the introduction to the description may be sufficiently addressed, and a delay on the decoder side may be reduced, if an envelope and/or grid area division is indeed used, according to which envelopes may indeed extend across frame boundaries so as to overlap with two adjacent frames, but if these envelopes are again subdivided by the decoder at the frame boundary, and the high-frequency reconstruction is performed at the grid which is subdivided in this manner and coincides with the frame boundaries. For the partial grid areas, thus obtained, of the overlap grid areas a spectral energy value, or a plurality of spectral energy values, is/are obtained, respectively, on the decoder side, from the one or the plurality of spectral energy value(s) as have been transmitted for the envelope extending across the frame boundary.

In accordance with a further aspect of the present invention, a finding of the present invention is that a delay on the decoding side may be obtained by reducing the frame size and/or the number of the samples contained therein, and that the effect of the increased bit rate associated therewith may be reduced if a new flag is introduced, and/or a transient absence indication is introduced, for frames having reconstruction modes according to which the grid boundaries coincide with the frame boundaries of these frames, such as FIXFIX frames, and/or for the respective reconstruction mode. Specifically, if there is no transient present in such a shorter frame, and if no other transient is present in the vicinity of the frame, so that the information signal is stationary at this point, the transient absence indication may be used not to introduce, for the first grid area of such a frame, any value describing the spectral envelope into the encoded information signal, but to derive, or obtain, same on the decoder side, rather from the value(s) representing the spectral envelope, said values being provided in the encoded information signal for the last grid area and/or the last envelope of the temporally preceding frame. In this manner, shortening of the frames with a reduced effect on the bit rate is possible, which shortening enables shorter delay time, on the one hand, and enables the transient problems because of the smaller frame units, on the other hand.

shows the architecture of an encoder in accordance with an embodiment of the present invention. The encoder ofis, by way of example, an audio encoder generally indicated by reference numeral. It includes an inputfor the audio signal to be encoded, and an outputfor the encoded audio signal. It shall be assumed below that the audio signal in inputis a sampled audio signal, such as a PCM-encoded signal. However, the encoder ofmay also be implemented differently.

The encoder offurther includes a down-sampler105and an audio encoderwhich are connected, in the order mentioned, between the inputand a first input of a formatter, the output of which, in turn, is connected to the outputof the encoder. Due to the connection of the portions105and, an encoding of the down-sampled audio signalresults at the output of the audio encoder, said encoding, in turn, corresponding to an encoding of the low-frequency portion of the audio signal. The audio encoderis an encoder which operates in a frame-by-frame manner in the sense that the encoder result present at the output of the audio encodercan only be decoded in units of these frames. By way of example, it shall be assumed below that the audio encoderis an encoder in conformity with AAC-LD in accordance with the standard of ISO/IEC 14496-3.

An analysis filter bank, an envelope data calculatoras well as an envelope data encoderare connected, in the order mentioned, between the inputand a further input of the formatter. In addition, the encoderincludes an SBR frame controllerwhich has a transient detectorconnected between its input and the input. Outputs of the SBR frame controllerare connected both to an input of the envelope data calculatorand to a further input of the formatter.

Now that the architecture of the encoder ofhas been described above, its mode of operation will be described below. As has already been mentioned, an encoded version of the low-frequency portion of the audio signalarrives at the first input of formatterin that the audio encoderencodes the down-sampled version of the audio signal, wherein, e.g., only every other sample of the original audio signal is forwarded. The analysis filter bankgenerates a spectral decomposition of the audio signalwith a certain temporal resolution. It shall be assumed, by way of example, that the analysis filter bankis a QMF filter bank (QMF=quadrature mirror filter). The analysis filter bankgenerates M subband values per QMF time slot, the QMF time slots each including 64 audio samples, for example. To reduce the data rate, the envelope data calculatorforms, from the spectral information of the analysis filter bankwhich has high temporal and spectral resolutions, a representation of the spectral envelope of audio signalwith a suitably lower resolution, i.e. within a suitable time and frequency grid. In this context, the time and frequency grid is set by the SBR frame controllerper frame, i.e. per frame of the frames as are defined by the audio encoder. Again, the SBR frame controllerperforms this control as a function of detected and/or localized transients as are detected and/or localized by the transient detector. For detection transients and/or note commencement times, the transient detectorperforms a suitable statistical analysis of the audio signal. The analysis may be performed in the time domain or in the spectral domain. The transient detectormay evaluate, for example, the temporal envelope curve of the audio signal, such as the evaluation of the increase in the temporal envelope curve. As will be described in more detail below, the SBR frame controllerassociates each frame and/or SBR frame to one of two possible SBR frame classes, namely either to the FIXFIX class or to the LD_TRAN class. In particular, the SBR frame controllerassociates the FIXFIX class with each frame which contains no transient, whereas the frame controller associates the LD_TRAN class with each frame having a transient located therein. The envelope data calculatorsets the temporal grid in accordance with the SBR frame classes as have been associated with the frames by the SBR frame controller. Irrespective of the precise association, all frame boundaries will coincide with grid boundaries. Only the grid boundaries within the frames are influenced by the class association. As will be explained below in more detail, the SBR frame controller sets further syntax elements as a function of the frame class associated, and outputs these to the formatter. Even though not explicitly depicted in, the syntax elements may naturally also be subjected to an encoding operation.

Thus, the envelope data calculatoroutputs a representation of the spectral envelopes in a resolution which corresponds to the temporal and spectral grid predefined by the SBR frame controller, namely by one spectral value per grid area. These spectral values are encoded by the envelope data encoderand forwarded to the formatter. The envelope data encodermay possibly also be omitted. The formattercombines the information received into the encoded audio data streamand/or to the encoded audio signal, and outputs same at the output.

The mode of operation of the encoder ofwill be described in a little more detail below usingwith regard to temporal grid division which is set by the SBR frame controllerand used by the envelope data calculatorto determine, from the analysis filter bank output signal, the signal envelope in the predefined grid division.

initially shows, by means of a pseudo code, the syntax elements by means of which the SBR frame controllerpredefines the grid division which is to be used by the envelope data calculator. Just like in the case of, those syntax elements which are actually forwarded from the SBR frame controllerto the formatterfor encoding and/or for transmission are depicted in bold print in, the respective row in the columnindicating the number of bits used for representing the respective syntax element. As may be seen, a determination is initially made, by the syntax element bs_frame_class, for the SBR frame, whether the SBR frame is a FIXFIX frame or an LD_TRAN frame. Depending on the determination (), different syntax elements are then transmitted. In the case of the FIXFIX class (), the syntax element bs_num_env[ch] of the current SBR frame ch is initially set to 2by the 2-bit syntax element tmp (). Depending on the number bs_num_env[ch] the syntax element bs_amp_res is left at a value of 1 which has been preset by default, or is set to zero (), the syntax element bs_amp_res indicating the quantization accuracy with which the spectrally enveloping values which are obtained by the calculatorin the predefined gridding are forwarded to the formatterin a state in which they are encoded by the encoder. The grid areas and/or envelopes predefined in their numbers by bs_num_env[ch] are set—with regard to their frequency resolution, which is to be used in same by the envelope data calculatorto determine the spectral envelope within them—by a common () syntax element bs_freq_res[ch] which is forwarded () to the formatterwith a bit from the SBR frame controller.

The mode of operation of the envelope data calculatoris to be described again below with reference towhen the SBR frame controllerspecifies that the current SBR frameis a FIXFIXFIX frame. In this case, the envelope data calculatorequally subdivides the current frame, which consists—here by way of example—of N=16 analysis filter bank time slots, into grid areas and/or envelopesa andb, so that here both grid areas and/or both envelopesa,b have a length of N/bs_num_inv[ch] time slotsand take up as many time slots between the SBR frame boundariesa andb. In other words, with FIXFIX frames, the envelope data calculatorarranges the grid boundariesuniformly between the SBR frame boundariesa,b such that they are equidistantly distributed within these SBR frames. As has already been mentioned, the analysis filter bankoutputs subband spectral values per time slot. The envelope data calculatortemporally combines the subband values in an envelope-by-envelope manner and adds their square sums in order to obtain the subband energies in an envelope resolution. Depending on the syntax element bs_freq_res[ch], the envelope data calculatoralso combines, in a spectral direction, several subbands to reduce the frequency resolution. In this manner, the envelope data calculatoroutputs, per envelopea,b, a spectrally enveloping energy sampling at a frequency resolution which depends on bs_freq_res[ch]. These values are then encoded by the encoderwith a quantization which in turn depends on bs_amp_res.

So far, the preceding description related to the case where the SBR frame controllerassociated a specific frame with the FIXFIX class, which is the case if there are no transients in this frame, as was described above. The following description, however, relates to the other class, i.e. the LDN-TRAN class, which is associated with a frame if it has a transient located in it, as is indicated by the detector. Thus, if the syntax element bs_frame_class indicates that this frame is an LDN-TRAN frame (), the SBR frame controllerwill determine and transmit, with four bits, a syntax element bs_transient_position so as to indicate—in units of the time slots, for example relative to the frame starta or, alternatively, relative to the frame endb—the position of the transient as has been localized by the transient detector(). At present, four bits are sufficient for this purpose. An exemplary case is depicted in., in turn, shows the SBR frameincluding the 16 time slots. The sixth time slotfrom the SBR frame starta has a transient T located therein, which would correspond to bs_transient_position=5 (the first time slot is the time slot zero). As is indicated atin, the subsequent syntax for setting the grid of an LD_TRAN frame is dependent on bs_transient_position, which must be taken into account, on the decoder side, in the parsing performed by a respective demultiplexer. However, at, the mode of operation of the envelope data calculatorupon obtaining the syntax element bs_transient_Position from the SBR frame controllermay be illustrated, which is as follows. By means of the transient position indication, the calculatorlooks up bs_transient_position in a table, an example of which is shown in. As will be explained in more detail below with reference to the table of, the calculatorwill set, by means of the table, an envelope subdivision within the SBR frame in such a manner that a short transient envelope is arranged around transient position T, whereas one or two envelopesa andb occupy the remaining part of the SBR frame, namely the part from the transient envelopeto the SBR frame starta, and/or the part from the transient envelopeto the SBR frame endb.

The table shown inand used by the calculatornow includes five columns. The possible transient positions which, in the present example, extend from zero to 15 have been entered into the first column. The second column indicates the number of envelopes and/or grid areas,a and/orb which result at the respective transient position. As may be seen, the possible numbers are 2 or 3, depending on whether the transient position is located close to the SBR frame start or the SBR frame enda,b, only two envelopes being present in the latter case. The third column indicates the position of the first envelope boundary within the frame, i.e. the boundary of the first two adjacent envelopes in units of time slots, specifically the position of the start of the second envelope, the position=zero indicating the first time slot in the SBR frame. The fourth column accordingly indicates the position of the second envelope boundary, i.e. the boundary between the second and third envelopes, this indication naturally being defined only for those transient positions for which three envelopes are provided. Otherwise, the values entered are negligible in this column, which is indicated by “−” in. As may be seen by way of example in the table of, there is, for example, only the transient envelopeand the subsequent envelopeb in the event that the transient position T is located in one of the first two time slotsfrom the SBR frame starta. It is not until the transient position is located in the third time slot from the SBR frame starta that there are three envelopesa,,b, envelopea including the first two time slots, transient envelopeincluding the third and fourth time slots, and envelopeb including the remaining time slots, i.e. from the fifth one onwards. The last column in the table ofindicates, for each transient position possibility, which of the two or three envelopes corresponds to that which has the transient and/or the transient position located therein, this information obviously being redundant and thus not necessarily having to be set forth in a table. However, the information in the last column serves to specify—in a manner which will be described in more detail below—the boundary between two noise envelopes, within which the calculatordetermines a value which indicates the magnitude of the noisy portion within these noise envelopes. The manner in which the boundary between these noise envelopes and/or grid areas is determined by the calculatoris known on the decoder side, and is performed in the same manner on the decoder side, just like the table ofis also present on the decoder side, namely for parsing and for grid division.

Referring back to, the calculatormay thus determine the number of envelopes and/or grid areas in the LD_TRAN frames from Table 2 of, the SBR frame controller () indicating, for each one of these two or three envelopes, the frequency resolution by a respective 1-bit syntax element bs_freq_res[ch] per envelope (). The controlleralso transmits the syntax values bs_freq_res[ch], which set the frequency resolution, to the formatter().

Thus, the calculatorcalculates, for all LD_TRAN frames, spectral envelope energy values as temporal means over the duration of the individual envelopesa,,b, the calculator combining, in the frequency resolution, different numbers of subbands as a function of bs_freq_res of the respective envelope.

Referring back to FIG. 2, the calculator 112 may thus determine the number of envelopes and/or grid areas in the LD_TRAN frames from Table 2 of FIG. 3, the SBR frame controller (116) indicating, for each one of these two or three envelopes, the frequency resolution by a respective 1-bit syntax element bs freq res[ch] per envelope (219). The controller 116 also transmits the syntax values bs freq res[ch], which set the frequency resolution, to the formatter 108.

Thus, the calculator 112 calculates, for all LD_TRAN frames, spectral envelope energy values as temporal means over the duration of the individual envelopes 222a, 220, 222b, the calculator combining, in the frequency resolution, different numbers of subbands as a function of bs freq res of the respective envelope.

The above description mainly dealt with the mode of operation of the encoder with regard to calculating the signal energies for representing the spectral envelopes in the time/frequency grid as is specified by the SBR frame controller. Additionally, however, the encoder ofalso transmits, for each grid area of a noise grid, a noise value which indicates, for this temporal noise grid area, the magnitude of the noisy portion in the high-frequency portion of the audio signal. Using these noise values, an even better reproduction of the high-frequency portion from the decoded low-frequency portion may be performed on the decoder side, as will be described below. As may be seen from, the number bs_num_noise of the noise envelopes for LD_TRAN frames is two, whereas the number for FIXFIX frames with bs_num_env=1 may also be one.

The subdivision of the LD_TRANS SBR frames into the two noise envelopes, but also of the FIXFIX frames into the one or two noise envelopes, may be performed, for example, in the same manner as is described in chapter 4.6.18.3.3 in the above-mentioned standard, to which reference shall be made in this context, and which passage shall be included, in this respect, by reference in the description of the present application. In particular, for example, the boundary between the two noise envelopes is positioned, by the envelope data calculatorfor LD_TRAN frames, onto the same boundary as—if the envelopea exists—the envelope boundary between the envelopea and the transient envelopeand as—if the envelopedoes not exist—the envelope boundary between the transient envelopeand the envelopeb.

Before continuing with the description of a decoder which is able to decode the encoded audio signal at outputof encoderof, the interplay between the analysis filter bankand the envelope data calculatorshall be dealt with in more detail. By the box,depicts, by way of example, the individual subband values which are output by the analysis filter bank. Init is assumed that the time axis t again extends from the left to the right in a horizontal manner. A column of boxes in a vertical direction thus corresponds to the subband values as obtained by the analysis filter bankat a certain time slot, an axis f being intended to indicate that the frequency is to increase in the upward direction.shows, by way of example, 16 successive time slots belonging to an SBR frame. It is assumed, in, that the present frame is an LD_TRAN frame and that the transient position is the same as was indicated, by way of example, in. The resulting grid classification within the frameand/or the resulting envelopes are also illustrated in.also indicates the noise envelopes, specifically byand. Using the formation of the sum of squares, the envelope data calculatornow determines mean signal energies in the temporal and spectral grid, as is depicted inby the dashed lines. In the embodiment of, the envelope data calculatorthus determines, for the envelopea and the envelopeb, only half as many spectral energy values for representing the spectral envelope as for the transient envelope. However, as may also be seen, the spectral energy values for the representation of the spectral envelopes are formed only by means of the subband valueslocated in the higher-frequency subbandsto, whereas the low-frequency subbandstoare ignored, since the low-frequency portion is encoded, as is known, by the audio encoder. In this context, it shall be noted, as a precaution, that the number of the subbands here is only by way of example, of course, as is the bundling of the subbands within the individual envelopes to form groups of four or two, respectively, as is indicated in. To remain with the example of, a total of 32 spectral energy values are calculated by the envelope data calculatorin the example offor representing the spectral envelopes, the quantization accuracy of which is performed for encoding, again as a function of bs_amp_res, as was described above. In addition, the envelope data calculatordetermines a noise value for the noise envelopesand, respectively, on the basis of the subband values of the subbandstowithin the respective envelopeor, respectively.

Now that the encoder has been described above, the following will provide a description of a decoder in accordance with an embodiment of the present invention which is suited to decode the encoded audio signal at the output, said description below also addressing the advantages entailed by the LD_TRAN class described with regard to bit rate and delay.

The decoder of, which is generally indicated at, comprises a data inputfor receiving the encoded audio signal, and an outputfor outputting a decoded audio signal. The input of a demultiplexer, which possesses three outputs, is adjacent to the input. An audio decoder, an analysis filter bank, a subband adapter, a synthesis filter bankas well as an adderare connected, in the order mentioned, between a first one of these outputs and the output. The output of the audio decoderis also connected to a further input of the adder. As will be described below, a connection of the output of the analysis filter bankto a further input of the synthesis filter bankmay be provided instead of the adderwith its additional input. The output of the analysis filter bank, however, is also connected to an input of a gain value calculator, the output of which is connected to a further input of the subband adapter, and which also comprises second and third inputs, the second of which is connected to a further output of the demultiplexer, and the third input of which is connected, via an envelope data decoder, to the third output of the multiplexer.

The mode of operation of the decoderis as follows. The demultiplexersplits up the arriving encoded audio signal at the inputby means of parsing. Specifically, the demultiplexeroutputs the encoded signal relating to the low-frequency portion, as has been generated by the audio encoder, to the audio decoderconfigured such that it is able to obtain, from the information obtained, a decoded version of the low-frequency portion of the audio signal and to output it at its output. The decoderthus already has knowledge of the low-frequency portion of the audio signal to be decoded. However, the decoderdoes not obtain any direct information on the high-frequency portion. Rather, the output signal of the decoderalso serves, at the same time, as a preliminary high-frequency portion signal or at least as a master, or basis, for the reproduction of the high-frequency portion of the audio signal in the decoder. Portions,,,, andfrom the decoderserve to utilize this master to reproduce, or to reconstruct, the final high-frequency portion therefrom, this high-frequency portion thus reconstructed being combined, by the adder, again with the decoded low-frequency portion so to eventually obtain the decoded audio signal. In this context it shall be noted, for completeness, sake, that the decoded low-frequency signal from the decodercould also be subject to further preparatory treatments before it is input into the analysis filter bank, this not being shown, however, in.

In the analysis filter bank, the decoded low-frequency signal is again subject to a spectral dispersion with a fixed time resolution and a frequency resolution which essentially corresponds to that of the analysis filter bank of the encoder. Remaining with the example of, the analysis filter bankwould outputsubband values per time slot, for example, said subband values corresponding to the 32 low-frequency subbands (-in). It is possible that the subband values as are output by analysis filter bankare reinterpreted, as early as at the output of this filter bank, or before the input of the subband adapter, as the subband values of the high-frequency portion, i.e. are copied into the high-frequency portion, as it were. However, it is also possible that in the subband adapter, the low-frequency subband values obtained from the analysis filter bankinitially have high-frequency subband values added to them in that all or some of the low-frequency subband values are copied into the higher-frequency portion, such as the subband values of subbandsto, as are obtained from the analysis filter bank, into subbandsto.

In order to perform the adaptation to the spectral envelope as has been encoded, on the encoder side, into the encoded audio signal, the demultiplexerwill initially forward that part of the encoded audio signalwhich relates to the encoding of the representation of the spectral envelope, as has been generated by the encoderon the encoder side, to the envelope data decoder, which, in turn, will forward the decoded representation of this spectral envelope to the gain values calculator. In addition, the demultiplexeroutputs that part of the encoded audio signal which relates to the syntax elements for grid division, as have been introduced into the encoded audio signal by the SBR frame controller, to the gain values calculator. The gain values calculatornow associates the syntax elements ofwith the frames of the audio decoderin a manner which is as synchronized as that of the SBR frame controlleron the encoder side. For the exemplary frame contemplated in, for example, the gain values calculatorobtains, for each time/frequency domain of the dashed grid, an energy value from the envelope data decoder, which energy values together represent the spectral envelope.

In the same grid, the gain values calculatoralso calculates the energy in the preliminarily reproduced high-frequency portion so as to be able to normalize the reproduced high-frequency portion in this grid and to weight it with the respective energy values it has obtained from the envelope data decoder, whereby the preliminarily reproduced high-frequency portion is spectrally adjusted to the spectral envelope of the original audio signal. Here, the gain values calculator takes into account the noise values which also have been obtained from the envelope data decoderper noise envelope, so as to correct the weighting values for the individual subband values within this noise frame. Thus, what is forwarded at the output of the subband adapterare subbands comprising subband values which are adapted with corrected weighting values to the spectral envelope of the original signal in the high-frequency portion. The synthesis filter bankputs together the high-frequency portion thus reproduced in the time domain using these spectral values, whereupon the addercombines this high-frequency portion with the low-frequency portion from the audio decoderinto the final decoded audio signal at the output. As is indicated by the dashed line in, it is also possible, alternatively, for the synthesis filter bankto use, for synthesis, not only the high-frequency subbands as have been adapted by subband adapter, but to also use the low-frequency subbands as directly correspond to the output of the analysis filter bank. In this manner, the result of the synthesis filter bankwould directly correspond to the decoded output signal which could then be output at the output.

The above embodiments had in common that the SBR frames comprised an overlap region. In other words, the time division of the envelopes was adapted to the time division of the frames, so that no envelope overlaps two adjacent frames, for which purpose a respective signaling of the envelope time grid was conducted, specifically by means of LD_TRAN and FIXFIX classes. However, problems will arise if transients occur at the edges of the blocks or frames. In this case, a disproportionately large number of envelopes is needed to encode the spectral data including the spectral energy values, or the spectral envelope values, and the frequency resolution values. In other words, more bits are consumed than would be needed by the location of the transients. In principle, two such “unfavorable” cases may be distinguished, which are illustrated in.

The first unfavorable situation will occur when the transient, which is established by the transient detector, is located almost at a frame start of a frame, as is illustrated in.shows an exemplary case wherein a frameof the FIXFIX class, which comprises a single envelopewhich extends over all 16 QMF slots, precedes the frame, at the start of which a transient has been detected by the transient detector, which is why the framehas been associated, by the SBR frame controller, with an LD_TRAN class, with a transient position pointing to the third QMF slot of the frame, so that the frameis subdivided into three envelopes,, and, of which enveloperepresents the transient envelope, and the other envelopesandsurround same and extend to the frame boundariesb andc of the respective frame. Merely to avoid confusion, it shall be pointed out thatis based on the assumption that a different table than inhas been used.