A computational resource management device uses a measured value of an execution time of data processing, a measured value of a resource amount, and a feature of input data as training data to learn a model indicating a relationship between the execution time and the resource. The device inputs, into the model, a feature of data scheduled to be input to calculate an estimated value of the execution time of the scheduled data processing, and uses the estimated value of the execution time, a variation index indicating variation in the estimated value of the execution time, and distribution of estimated residuals to calculate a resource amount required in the scheduled data processing. The device creates an execution plan of the scheduled data processing, based on the estimated value of the execution time, the variation index, the distribution of estimated residuals, and the calculated resource amount.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A computational resource management device for managing a system that performs data processing, comprising: a processor; and a memory storing instructions executable by the processor to: use a measured value of an execution time of data processing, a measured value of a resource amount allocated to the data processing, and a feature of input data to the data processing as training data to learn a model indicating a relationship between the execution time and the resource; input a feature of data scheduled to be input to data processing scheduled to be executed into the model to calculate an estimated value of the execution time of the scheduled data processing; generate, as a variation index, a distribution of statistics relating to changes in the feature of data, the variation index indicating variation in the estimated value of the execution time that depends on change in the feature of data input in past data processing; applies a specific function to a learning error of an estimation function to calculate a distribution of estimated residuals, during learning of the model; use the estimated value of the execution time, the variation index, and the distribution of estimated residuals to calculate a resource amount required in the scheduled data processing; and create an execution plan of the scheduled data processing, based on the estimated value of the execution time, the variation index, the distribution of estimated residuals, and the calculated resource amount.
2. The computational resource management device according to claim 1 , wherein the instructions are executable by the processor to further: use environment information specifying an environment in which data processing is executed to learn the model, and input environment information specifying an environment in which the scheduled data processing is to be executed into the model to calculate the estimated value of the execution time of the scheduled data processing.
3. The computational resource management device according to claim 1 , wherein the instructions are executable by the processor to further: in a case where a threshold is set in advance for a probability of not complying with a completion time limit in the scheduled data processing, creates the execution plan of the scheduled data processing so as to not exceed the threshold.
4. A computational resource management method for managing a system that performs data processing, comprising: using a measured value of an execution time of data processing, a measured value of a resource amount allocated to the data processing, and a feature of input data to the data processing as training data to learn a model indicating a relationship between the execution time and the resource; inputting a feature of data scheduled to be input to data processing scheduled to be executed into the model to calculate an estimated value of the execution time of the scheduled data processing; generating, as a variation index, a distribution of statistics relating to changes in the feature of data, the variation index indicating variation in the estimated value of the execution time that depends on change in the feature of data input in past data processing; applying a specific function to a learning error of an estimation function to calculate a distribution of estimated residuals, during learning of the model; using the estimated value of the execution time, the variation index, and the distribution of estimated residuals to calculate a resource amount required in the scheduled data processing; and creating an execution plan of the scheduled data processing, based on the estimated value of the execution time, the variation index, the distribution of estimated residuals, and the calculated resource amount.
5. The computational resource management method according to claim 4 , wherein environment information specifying an environment in which data processing is executed is further used to learn the model, and environment information specifying an environment in which the scheduled data processing is to be executed is further input into the model to calculate the estimated value of the execution time of the scheduled data processing.
6. The computational resource management method according to claim 4 , wherein, in a case where a threshold is set in advance for a probability of not complying with a completion time limit in the scheduled data processing, the execution plan of the scheduled data processing is created so as to not exceed the threshold.
7. A non-transitory computer-readable recording medium on which is recorded a computer program for managing, by computer, a system that performs data processing, the computer program including a command for causing the computer to execute processing comprising: using a measured value of an execution time of data processing, a measured value of a resource amount allocated to the data processing, and a feature of input data to the data processing as training data to learn a model indicating a relationship between the execution time and the resource; inputting a feature of data scheduled to be input to data processing scheduled to be executed into the model to calculate an estimated value of the execution time of the scheduled data processing; generating, as a variation index, a distribution of statistics relating to changes in the feature of data, the variation index indicating variation in the estimated value of the execution time that depends on change in the feature of data input in past data processing; applying a specific function to a learning error of an estimation function to calculate a distribution of estimated residuals, during learning of the model; using the estimated value of the execution time, the variation index, and the distribution of estimated residuals to calculate a resource amount required in the scheduled data processing; and creating an execution plan of the scheduled data processing, based on the estimated value of the execution time, the variation index, the distribution of estimated residuals, and the calculated resource amount.
8. The non-transitory computer-readable recording medium according to claim 7 , wherein environment information specifying an environment in which data processing is executed is further used to learn the model, and environment information specifying an environment in which the scheduled data processing is to be executed is further input into the model to calculate the estimated value of the execution time of the scheduled data processing.
9. The non-transitory computer-readable recording medium according to claim 7 , wherein, in a case where a threshold is set in advance for a probability of not complying with a completion time limit in the scheduled data processing, the execution plan of the scheduled data processing is created so as to not exceed the threshold.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
April 27, 2017
March 30, 2021
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.