Generation of Result Image Data from Origin Data Based on a Medical Imaging

The present patent document claims the benefit of German Patent Application No. 10 2024 203 947.0, filed Apr. 26, 2024, which is hereby incorporated by reference in its entirety.

The disclosure relates to a computer-implemented method for generation of result image data from origin data that is based on a medical imaging method, by an overall processing algorithm that includes a number of part algorithms. The disclosure also relates to a processing apparatus, to a computer program, and to a data medium.

In medical imaging, a series of image processing algorithms may be applied to the physically detected image, (e.g., to the image recorded by an x-ray detector, as part of a reconstruction of image data from measurement data, and/or after such a reconstruction), in order to prepare the image as well as possible for the intended usage purpose. Reconstruction algorithms for creation of slice images from projection recordings, denoising algorithms for reduction of quantum noise in x-ray images or also segmentation algorithms for emphasizing relevant structures are used as image processing algorithms for example.

Here, the image obtained from the purely physical processes, (which may also be referred to as the raw image and which may be considered as the image closest to reality), is modified in a wide variety of ways. With each processing stage, the processed image removes data from the raw image with regard to image values and/or image representation in a desired and, in some cases, additionally in an undesired way. The undesired modifications may include algorithmic artifacts, which are willingly taken into account for the desired effects. A known example of such artifacts are “halos,” e.g., over-bright areas in the area of edges during the modification of the frequency spectrum.

In particular, when at least one artificial intelligence image processing algorithm is used that attempts to interpret image contents, the results of the processing are not necessarily verifiable for an observer. During processing, assumptions are made about properties and contents of the image in order to generate the processed image. With such processing, familiar image properties and their relationships may become lost for the observer. Thus, an interpreting denoising algorithm might free a dose-related noisy image from noise to an extent that the impression of a high-dose image arises, in which the recognizable structures would actually have corresponded to the real information content of the original image. An interpreting algorithm might thus represent its interpretation as actually present information in this case, even if actually only insufficient image information might be available. Frequently with such processing, this also does not result in easily detectable artifacts, which may inform the observer how greatly the processed image deviates for the “truth,” e.g., from the original raw image.

The scope of the present disclosure is defined solely by the appended claims and is not affected to any degree by the statements within this summary. The present embodiments may obviate one or more of the drawbacks or limitations in the related art.

The underlying object of the disclosure is to support a user in the assessment of result image data resulting from an overall processing algorithm and to make it possible for the user to assess how greatly and/or in what way the result image data deviates for the processed origin image data.

The object is achieved by a computer-implemented method for generation of result image data from origin data that is based on a medical imaging method, by an overall processing algorithm that includes a number of part algorithms, wherein the part algorithm, depending on respective input data and at least one respective parameter setting, generates respective output data. The input data of the part algorithms is predetermined by the origin data predetermined or is established from the origin data. The result data is established depending on the output data of the part algorithms. The parameter setting for at least one part algorithm is predetermined depending on a control parameter of the overall processing algorithm, wherein the control parameter is adjustable by an operator input of the user continuously or in at least three stages between a first and a second limit value. The first limit value is assigned a first value of the parameter setting and the second limit value is assigned a second value of the parameter setting for the at least one part algorithm. Setting the first value of the parameter setting for the at least one part algorithm, at least for a subgroup of the possible origin data, leads to the output data of the at least one part algorithm or of the number of part algorithms deviating more greatly from input data or from reference data that would result from an application of a reference algorithm assigned to the at least one part algorithm or to the number of part algorithms to this input data than by setting the second value of the parameter setting.

The user, e.g., the observer of the result image data or of an image established from this, for example, of a slice image or of an artificial projection image, may be able to recognize to what extent the appraised image reflects the real information content of the imaging. The method gives them the opportunity here of easily changing this extent in order to make this assessment possible and in order to adapt the image to current requirements. T his is made possible by the user, by setting a single parameter, namely the control parameter, being able to vary the origin data, continuously or at least in a number of stages between various processing strengths. In particular, the control parameter is able to beset effectively continuously, e.g., in a plurality of stages such as in at least 10 stages, at least 20 stages, or at least 50 stages.

By setting the control parameter to the second limit value, the raw image or an image that results from minimal processing may be shown. By setting the control parameter to the first limit value, a maximum optimization of the image impression for the given task may result. Through a multi-stage or continuous transition between these degrees of processing, the effects of the various part algorithms in particular may be well verified by the user, so that erroneous image impressions may be removed. To do this in particular, as explained in greater detail below, various parts of the image processing in particular in various parameter ranges of the control parameter may be deactivated or attenuated.

In particular, it is possible that the control parameter and thus the strength of the change to the image are shown together with the result image data or the image established from the data, in order to make this information directly accessible to the user. The control parameter may be displayed directly as an operating element, for example, as a regulator, in order to make a dynamic adaptation possible as required.

The opportunity of setting the control parameter or the operating element explained also make it possible for the user to set an individual degree of image data processing dynamically and in line with requirements. For example, a doctor undertaking the treatment in an angiography for navigation of devices using fluoroscopy may allow a high degree of algorithmic interpretation in favor of a noise-free and clear representation but lower this for diagnostics using a higher x-ray dose and/or for exact placement of implants.

The origin data may directly involve measurement data or data already preprocessed, for example, three-dimensional image data already reconstructed. The origin data may be based on a magnetic resonance imaging, an x-ray imaging such as a computed tomography or a fluoroscopy, an ultrasound imaging, or on any other given medical imaging method. The imaging itself may not be part of the method described herein. The imaging may have already been concluded and the origin data may be taken from a database or similar. The imaging may alternatively be integrated into the method as an additional method act.

The part algorithms may be switched in series after one another so that a first of the part algorithms processes the origin data and the others of the part algorithms each further process the output data of the preceding part algorithm. It is also possible for at least parts of the part algorithms to be applied in parallel, e.g., in order to separately process various frequency ranges of the origin data or of an intermediate result. For further processing by at least one further of the part algorithms or for provision of the result image data, the output data of the part algorithms operating in parallel may be summed in a weighted manner, for example.

The second value of the parameter setting of the respective part algorithm may lead, at least for one subgroup of the part algorithms, to the output data of the respective part algorithm being identical to the input data of this part algorithm, whereby the part algorithms of this subgroup would be effectively skipped, with a setting of the control parameter to the second limit value. It is also possible for the second value of the parameter setting of the respective part algorithm to continue to change the input data, but to lead to a less strong processing of the respective input data than the first value of the parameter setting. For example, the strength of a filtering or a spectral shaping, which may serve to suppress noise and/or to reduce the low-frequency dynamics, and/or the strength of an edge rise may be varied. The strength of the deviation of the output data of the respective part algorithm from its input data may be quantified by deviation measures, for example, by a sum of the squares of the differences in the individual pixels or voxels.

The reference algorithm may bean algorithm for which it is assumed that the algorithm is passing on the data uncorrupted. An observation of the deviation of reference data may be expedient when data processing is necessarily required in the respective part algorithm, for example, in order to carry out an image reconstruction, as may be required for magnetic resonance data and for the reconstruction of a computed tomography from an individual projection image. A pure reconstruction algorithm without further image preparation properties may be regarded as a reference algorithm, for example.

In particular, a choice may be made by various settings of the control parameter continuously or at least in a number of stages between a strong and a weaker change in the data by the respective part algorithm, so that by an adjustment of the control parameter toward the limit value, the result data approaches the origin data or reference data, which may be generated from the origin data by quite simple processing, for example, by a direct reconstruction with minimal filtering and without any other image optimization acts.

The subgroup of the possible origin data may include all origin data that occurs with an adequate image quality during imaging of a patient. For example, only origin data that does not image any real object and/or that is primarily hallmarked by noise elements or the like may lie outside the subgroup. Corresponding incorrect measurements may lead to origin data for which processing leads to unexpected results, so that the condition stated above for the subgroup is not necessarily met in these cases.

In one example, or when a direct algorithmic parameterization of the strength of the effect of processing an image by the respective part algorithm is not readily possible, which may be the case with neural networks or other trained processing functions, the effect of any given image processing P of input data u by a parameter value λ∈ [0,1] of the parameter setting and the scaled addition of the difference image may be parameterized:

If the parameter value λ is set to 1, the maximum processing strength results. Lowering this parameter value results in weaker processing, wherein for a parameter value of λ=0 the part algorithm is effectively skipped.

The respective part algorithm may be assigned a respective assignment specification, which allocates a parameter setting of the assigned part algorithm to the first and second limit value and to at least one intermediate value lying between the first and second limit value of the control parameter, wherein assignment specifications that are assigned to at least two of the part algorithms describe relationships differing from one another between the respective parameter setting and the control parameter.

For example, the parameter settings of the various part algorithms may be predetermined by a respective function of the control parameter or may be selected so that they lie at such a function, e.g., when they are predetermined by a Look-U p Table or the like. The various relationships between the respective parameter setting and the control parameter may result from different rises and/or curvatures of the respective function for at least one of the control parameter values.

The respective parameter setting may predetermine a respective parameter value of at least one parameter of the respective part algorithm, wherein the respective parameter value is a monotonously rising or falling function of the control parameter in each case.

The part algorithms may be parameterized in such a way that, at least for the subgroup of the possible origin data, an increase in the respective parameter value leads to a strengthening or to a reduction in the deviation of the output data from the input data of the respective part algorithm or from the reference data. As a result of the monotonous relationship between parameter value and control parameter explained above, it may be provided that, when the control parameter is changed toward the second limit value for each of the part algorithms, the deviation of the respective output data from the respective input data or reference data falls or at least remains the same, so that it may be assumed that here the result data overall is close to the origin data or at least to an image generated or reconstructed from the origin data with the minimum image manipulations.

It may be advantageous if, at least for a few of the part algorithms or parameter values, the monotonous function does not rise or fall strictly monotonously but remains constant for specific ranges of values of the control parameter constant. In particular, functions for establishing parameter values of different part algorithms in ranges of values of the control parameter differing from one another may be constant, so that the processings by the individual part algorithms for a variation of the control parameter may be faded out or deactivated separately one after another or also with overlapping cross-fade areas.

The maximum of the at least one parameter value of the parameter setting of a first of the part algorithms may be reached for a value of the control parameter other than the maximum of the at least one parameter value of the parameter setting of a second of the part algorithms. In addition, or as an alternative, the minimum of the at least one parameter value of the parameter setting of a first of the part algorithms may be reached for a value of the control parameter other than the minimum of the at least one parameter value of the parameter setting of a second of the part algorithm.

In particular, for more than two, in particular for all part algorithms or at least for all part algorithms switched after one another in series, maxima or minima of their parameter values may be reached for different values of the control parameter in each case, so that, in different sections of the range of values of the control parameter, a processing by different the part algorithms may be faded out or attenuated or be replaced by processing that potentially corrupts the image less greatly.

In particular, the respective function, which predetermines the respective parameter value as a function of the control parameter, may be constant over a respective range of values and equal to the maximum or the minimum of the parameter value. By reaching the maximum or the minimum of the parameter value of the different part algorithms with different values of the control parameter, enables the ranges, in which the respective parameter value is constant, to not overlap or only partly to overlap, so that there may be a change of the parameter values of the different part algorithms in different ranges of values of the control parameter.

At least two of the functions, which each predetermine a different parameter value of the part algorithms or predetermine the different parameter values of the same part algorithm as a function of the control parameter, may have a different curvature and a curvature that differs from zero over the entire range of values of the control parameter or at least over a part of the range of values of the control parameter.

In particular, the curvatures of two such functions may have different leading signs. By different curvatures, in particular by curvatures with different leading signs, the two functions may even be reached with a coincidence of the maxima and minima of the two functions, so that in a first range of values of the control parameter a first of the two parameter values is greatly changed, while a second of the parameter values remains almost constant, while in a second range of values of the control parameter the first of the parameter values remains almost constant and the second of the parameter values is greatly changed. Thus, in particular in the first of the ranges of values, one of the part algorithms is faded out or at least largely deactivated, while another part algorithm initially continues to be applied unchanged. Only when the second range of values is reached is the other part algorithm also appreciably attenuated or faded out.

The overall processing algorithm may include a processing algorithm based on a segmentation and/or classification of the respective input data as the first of the part algorithms or a processing algorithm based on machine learning and a filter algorithm as the second of the part algorithms. In this case, the functions of the control parameter predetermining the parameter values of the first and second part algorithm may be selected in such a way that, at least for control parameters that lie within a predetermined control parameter interval, the amount of the quotient of the difference between the parameter value assigned to the control parameter and the corresponding parameter value assigned to the second value of the parameter setting and the difference between the parameter values that are assigned to the respective first and second value of the parameter setting is greater for the second part algorithm than it is for the first part algorithm.

For easier understanding of the requirement discussed above, it is assumed below, for example, that each of the part algorithms is only parameterized by one parameter value and that the respective parameter value is 1 for the first value of the parameter settingand is 0 for the second value of the parameter setting. In this case, the result of the requirement is that the parameter value of the first part algorithm in the predetermined control parameter interval is less than the parameter value of the second part algorithm. To put it differently, regardless of the restriction of the parameter range to 0 to 1 used in the example, at least in the predetermined control parameter interval, the processing by the first part algorithm is more greatly attenuated than the processing by the second part algorithm.

This is advantageous since processing algorithms based on a segmentation and/or classification of the respective input data or on machine learning may change the impression of the image greatly and under some circumstances in a way not readily appreciated by an observer. The embodiment of the method explained thus makes it possible, within the framework of the adjustment of the control parameter toward the second limit value, initially to fade out these changes, but, for example, to retain unchanged filtering for noise suppression, for reduction of a low-frequency dynamic and/or for edge enhancement.

Outside the predetermined control parameter interval, the parameter values, or quotients may be the same for both part algorithms. In particular, control parameters may lie exclusively outside the predetermined control parameter interval, for which the increase of the respective function is zero and thus both parameter values are constant.

The overall processing algorithm, in addition or as an alternative, may include as part algorithms a first filter algorithm for adapting the spectral portions in a first frequency band and a second filter algorithm for adapting the spectral portions in a second frequency band of the origin data or of an intermediate result established using at least one of the part algorithms, wherein the first frequency band extends to higher frequencies than the second frequency band. In this case, the functions of the control parameter predetermining the parameter values of the first and second filter algorithm may be chosen in such a way that, at least for control parameters that lie within one or a further predetermined control parameter interval, the amount of the quotient of the difference between the parameter value assigned to the control parameter and the parameter value corresponding to the second value of the parameter setting and the difference between the parameter values that are assigned to the respective first and second value of the parameter setting is greater for the second filter algorithm than it is for the first filter algorithm.

Thus, as has already been explained above for the first and second part algorithm, at least in the or the further predetermined control parameter interval, the processing by the first filter algorithm may be more greatly attenuated than the processing by the second filter algorithm. This may be advantageous since a change in the mid to high frequency ranges may lead to marked artifacts, for example, to “halos,” while with a change in the spectral composition in the low-frequency range, e.g., with a reduction of the low-frequency dynamic, fewer artifacts or fewer obvious artifacts may occur. The parameterization explained as a function of the control parameter may thus make it possible for a user to first attenuate or fade out any kind of processing that frequently leads to a relatively strong artifact formation, wherein the less problematic adaptation of the lower frequency components may be retained unchanged.

If, for example, a reduction of the low-frequency dynamic, a change in the range of the high to mid image frequencies, for example, for edge enhancement, and a processing algorithm based on a classification of image contents are used as part algorithms, then with an adjustment of the control parameter from the limit value toward the second limit value, at the beginning of the adjustment path, only the processing based on the classification is deactivated or faded out. Only with a value of the control parameter, for which this is already largely deactivated, may a start be made on the fading out of the filtering in the mid and/or high frequency range and only after a marked reduction or fading out of this filtering may the filtering for reducing the low-frequency dynamics be attenuated or faded out.

As explained in greater detail below, a spectral decomposition algorithm may also be used to supply various part algorithms with various spectral components of the origin data or of the intermediate result. In this case, the respective filter algorithm may be implemented by there being a scaling of one or more of the spectral components by a respective part algorithm.

The overall processing algorithm may include a number or all of the following part algorithms: an algorithm for edge enhancement, a filter algorithm such as for reduction of a low-frequency dynamic, a processing algorithm based on a segmentation and/or classification of the respective input data, and/or a processing algorithm based on machine learning and/or a scaling of the input data. Thus, in the computer-implemented method, for example, the entire processing chain or also just parts of the processing chain may be parameterized by the control parameter.

A segmentation or classification of the respective input data of the part algorithm may be used to carry out a color coding of the image as a function of a classification of image contents, to highlight or suppress image regions and thus the features shown there as a function of the classification or segmentation, for example, by increasing or reducing the brightness or the contrast in segments with a specific classification, for highlighting specific segments by a surround or similar.

The processing algorithm based on machine learning may carry out a classification and/or segmentation of input data in order to use this in the way already explained. It is also possible for the desired overall functionality to be trained directly, for example, the color coding of specific segments, so that a classification or segmentation does not necessarily take place explicitly within this algorithm.

A processing algorithm based on machine learning may emulate the cognitive functions, which make the connection between humans and human thinking. By training on the basis of training data, such a processing algorithm may be capable of adapting itself to new circumstances and recognizing and extrapolating patterns. Such a processing algorithm may also be referred to as a “trained machine learning model” or “trained function.”

The parameters of a machine learning model may be adapted by training. In particular, a supervised training, a semi-supervised training, an unsupervised training, a reinforcement learning, and/or an active learning may be used. Further, representation learning or feature learning may also be employed. In particular, the parameters of the machine learning models may be adapted iteratively by a number of training acts. In particular, a specific cost function may be minimized within the framework of the training. In particular, the backpropagation algorithm may be used during training of a neural network.

A machine learning model may in particular include a neural network, a support vector machine, a decision tree, and/or a Bayesian network, and/or the machine learning model may be based on k-means clustering, Q learning, genetic algorithms, and/or association rules. A neural network may be a Deep Neural Network, a Convolutional Neural Network, or a Convolutional Deep Neural Network sein. Alternatively, the neural network may bean adversarial network, a deep adversarial network, and/or a generative adversarial network.

A highpass filter, a lowpass filter, and/or a bandpass filter may be used as filter algorithm or as filter algorithms, for example. The filter algorithm or at least one of the filter algorithms used as part algorithm may also carry out any given spectral shaping of its input data.

For implementing a complex spectral shaping or filtering, it may also be expedient to use a number of separate part algorithms that each differently scale various spectral components of the origin data or of an intermediate result provided by at least one of the further part algorithms.

In one case, an edge enhancement may be implemented by the enhancement of higher frequencies in an image. One realization is the scaled addition of a highpass image:

wherein u is the input image and H (u) a highpass image of u.