Input Evaluations Through Model Inversion

Many stakeholders across multiple industries rely on models of processes to achieve certain objectives that can include meeting profitability goals, producing products that meet performance standards, and the like. When using models, stakeholders often provide known inputs to a model, wherein the model provides outputs of desired information to the stakeholder. For example, in some industrial processes, a stakeholder may provide process parameters as inputs to a model, and the model will provide outputs based on the construction of the model. The stakeholders may then use the provided outputs to change process parameters to more capably achieve their interests.

Systems and methods for input evaluations through model inversion are described herein. In certain embodiments, a system includes a memory configured to store a model of a physical process, wherein the model is configured to receive one or more input values and provide one or more output values, wherein the one or more input values represent potential input parameters for the physical process and the one or more output values represent potential measures of process outputs. The system also includes an interface capable of receiving one or more desired output values for the physical process. Further, the system includes one or more processors executing computer executable instructions associated with an application that cause the one or more processors to receive the one or more desired output values; test a plurality of values for at least one of the one or more input values; identify a combination of the one or more input values that is associated with the one or more desired output values; and provide the combination for output through the interface.

Per common practice, the drawings do not show the various described features according to scale, but the drawings show the features to emphasize the relevance of the features to the example embodiments.

The following detailed description refers to the accompanying drawings that form a part of the present specification. The drawings, through illustration, show specific illustrative embodiments. However, it is to be understood that other embodiments may be used and that logical, mechanical, and electrical changes may be made.

Systems and methods for a model explainability tool that evaluates inputs using model inversion are described herein. In particular, a computational device may store a model of a physical process, wherein the model receives input information and generates output information. For example, the input information may represent potential values for one or more parameters for a physical process. Also, the generated output information may represent performance indicators for the process or other process-related information. Further, the computational device includes an interface that can receive one or more desired outputs for the modeled process. The computational device may then provide various combinations of inputs to the model, wherein the computational device then evaluates the output with respect to the desired output. When the computational device identifies a combination of inputs associated with the desired output, the computational device may provide the combination of inputs as an output through the interface.

There are often multiple stakeholders in an industrial process. As used herein, a stakeholder refers to an individual or group of individuals that have an interest in or interact with the industrial process. For example, a stakeholder may be a business owner, a site engineer, a laboratory technician, a consultant, or any other individual or group. Often, stakeholders desire the process to meet one or more performance indicators. However, as industrial processes are often complex, many stakeholders lack the knowledge or ability to control or understand how to control the process to achieve the desired performance indicators. To help stakeholders control industrial processes, models are often created to model a process to help stakeholders more capably control the industrial processes. However, especially with the advent of machine learning models, models have become increasingly difficult for stakeholders to understand the workings of the models. For example, to many stakeholders, using a model involves providing inputs to a black box that provides outputs.

Thus, it is difficult for stakeholders to understand the relationship between the inputs and the outputs, leading to difficulty in gaining an understanding of the model that can help the stakeholders change process parameters that would cause the process to achieve the desired performance indicators.

In certain embodiments, to help stakeholders achieve an understanding of the model, a user interface application may be provided to one or more stakeholders in a particular process. The stakeholders may provide information through the user interface application that the user interface application uses to identify process inputs that are more likely to cause the process to achieve a desired performance indicator. For example, the user interface application may receive input from the stakeholder representing a desired performance indicator for the modeled process. In response, the user interface application may provide initial inputs and check the output provided by the model. If the output matches the desired performance indicator or is within a desired tolerance, the user interface application then provides the input parameters to the stakeholder as an output through the user interface application. However, if the output does not match the desired performance, the user interface application may iteratively adjust the input parameters provided to the model until the model provides outputs that match the desired performance indicators. The user interface application then provides users with the input parameters that result in the desired output.

1 FIG. 100 100 122 122 122 101 101 103 105 105 107 107 103 111 122 100 109 101 109 113 111 107 103 is a block diagram of a systemfor evaluating inputs through model inversion. As illustrated, the systemincludes a sitewhere an industrial process is performed. Further, some of the components located at the siteoptionally communicate with a computing system. The computing systemincludes one or more processorsand a memory, where the memorystores a model. Further, the modelis designed to be executed by the processorto simulate at least a portion of the industrial processperformed on the site. Further, the systemincludes a user interfacethat is connected to the computing system, where the user interfaceallows a stakeholderin the industrial processto interact with the simulations of the modelexecuted by the processoron the computing system.

111 111 111 111 111 111 111 111 111 111 111 111 111 111 111 In embodiments described herein, an industrial processrefers to a series of steps performed using various components and materials, where the steps facilitate the production of a valuable product or the performance of a service that involves the transformation or processing of raw materials. Examples of industries that employ industrial processesinclude mining, oil refining, pharmaceuticals, and many other industries. For instance, in industries generally, tasks performed as part of the industrial processmay include material processing, surface treatments, process controls and automation, quality management, logistics, and other industrial tasks. In particular, when the industrial processincludes material processing tasks, the industrial processmay include tasks that shape materials. Shaping tasks may include casting and molding tasks (where materials are shaped by pouring liquid materials into molds, forming tasks (where materials are shaped through deformation), Machining tasks (where materials are shaped through removing material), joining tasks (where materials are shaped by joining different objects together), and the like. Additionally, when the industrial processincludes surface treatment tasks, the industrial processmay include tasks that affect the surface of materials. Surface treatment tasks may include coating tasks (where layers are applied to the material to protect and decorate an external surface), heat treating tasks (where surface properties are altered through controlled heating and cooling), and the like. Further, when the industrial processincludes process controls and automation tasks, the industrial processmay include tasks that monitor and configure other process steps. Process controls and automation tasks include sensing tasks (where process variables are monitored), controlling tasks (where systems that control other process tasks are configured), and the like. Moreover, when the industrial processincludes quality management tasks, the industrial processmay include tasks that monitor and control the quality of tasks performed as part of the industrial process. Quality control tasks include assurance tasks (where activities are performed to ensure quality requirements are met), control tasks (where activities are performed to fulfill quality requirements), compliance tasks (where activities are performed to ensure the industrial processadheres to specified regulations), and the like. Also, when the industrial processincludes logistic tasks, the industrial processmay include tasks that control the movement and distribution of materials for the performance of other process steps. Logistic tasks include procuring tasks (where materials are acquired from other sources for the performance of other tasks), inventory managing tasks (where activities are performed for controlling the amount of materials in the process), distributing tasks (where activities are performed to control the movement of materials involved in the process and the distribution of finished products to customers), and the like. Examples of industries that perform industrial processes, such as those described above, may include oil-refining industries, pharmaceutical industries, mining industries, power providers, and many other industries that produce products or refine materials to provide a service.

111 111 111 111 111 111 111 111 In additional implementations, certain industries may employ industrial processthat include industry-specific tasks. For example, when the industrial processis performed as part of a mining industry, the industrial processmay involve the extraction of minerals and other geologic materials from the Earth and the refinement of the extracted materials for use as raw materials for other industries. In particular, when the industrial processis performed within the mining industry, tasks may include exploring tasks, extracting tasks, processing tasks, refining tasks, environmental managing tasks, and the like. When the industrial processperformed within the mining industry includes exploring tasks, the industrial processincludes tasks that identify the locations of likely mineral deposits. Exploring tasks include geological tasks (where surface geology is studied to predict subsurface conditions), geophysical tasks (where tasks are performed to detect subsurface anomalies), geochemical tasks (where tasks are performed to detect evidence of mineral deposits in samples), sensing tasks (where images are analyzed to predict the location of mineral deposits), and the like. Further, when the industrial processperformed within the mining industry includes extracting tasks, the industrial processincludes tasks that remove minerals from the Earth. Extraction tasks include surface mining tasks (where tasks are performed to remove minerals near the surface of the Earth), underground mining (where tasks are performed to remove minerals from deep beneath the surface), and the like.

111 111 111 111 111 111 Moreover, when the industrial processperformed within the mining industry includes processing tasks, the industrial processincludes tasks for processing ores for refinement. Processing tasks include crushing tasks (where tasks are performed to reduce the size of ore), separation tasks (where tasks are performed to separate different minerals), and the like. Also, when the industrial processperformed within the mining industry includes refining tasks, the industrial processincludes tasks for producing extracted minerals. Refining tasks include smelting tasks (where ore is heated for the extraction of metal), chemical tasks (where chemical processes are performed to extract metals from ores), purifying tasks (where tasks are performed to remove impurities from extracted metals), and the like. Further, when the industrial processperformed within the mining industry includes environmental management tasks, the industrial processincludes tasks for controlling the impact of mining tasks on the environment. Environmental management tasks include tailings management tasks (where tasks are performed for controlling the impact of waste materials produced during the production of metals), treatment tasks (where tasks are performed for treating waste to remove impurities), reclamation tasks (where tasks are performed for restoring or repurposing mined land), and the like.

111 111 111 111 111 111 111 111 111 111 111 111 111 In further implementations, the industrial processmay be performed within oil-refinement industries, where the industrial processinvolves the processing of crude oil into useable products that may include gasoline, diesel fuel, asphalt base, heating oil, kerosene, and other oil-derived products. In particular, when the industrial processis performed within oil-refinement industries, tasks may include distilling tasks, converting tasks, treating tasks, blending tasks, environmental managing tasks, and the like. When the industrial processperformed within the oil-refinement industries includes distilling tasks, the industrial processincludes tasks that separate oil into different components based on boiling points. Distilling tasks include heating tasks (where crude oil is heated), distillation tasks (where different compounds are heated to different boiling points), collection tasks (where different compounds are collected), and the like. Further, when the industrial processperformed within the oil-refinement industries includes converting tasks, the industrial processincludes tasks that break down and combine hydrocarbons into desired compounds. Converting tasks may include cracking tasks (where hydrocarbons are broken down into smaller molecules), reforming tasks (where molecules are converted into different molecules), combining tasks (where smaller molecules are combined into larger molecules), and the like. Moreover, when the industrial processperformed within the oil-refinement industries includes treating tasks, the industrial processincludes tasks that remove impurities from petroleum products. Also, when the industrial processperformed within the oil-refinement industries includes blending tasks, the industrial processincludes tasks that mix different compounds to produce final products. Further, when the industrial processperformed within the oil-refinement industries includes environmental managing tasks, the industrial processincludes tasks to reduce the impact of the refinement process on the environment. Environmental Managing tasks include emissions control tasks (where tasks are performed to reduce air pollutants), treatment tasks (where contaminants are removed from waste products), safety tasks (where tasks are performed to prevent dangerous accidents), and the like.

111 111 111 111 111 In other implementations, the industrial processmay be performed within pharmaceutical industries, where the industrial processinvolves the production of medications. In particular, when the industrial processis performed within pharmaceutical industries, tasks may include discovery tasks, testing tasks, manufacturing tasks, quality control tasks, compliance tasks, and the like. When the industrial processperformed within the pharmaceutical industries includes discovery tasks, the industrial processincludes tasks to identify molecules that could be medicinal. Discover tasks include targeting tasks (where tasks are performed to identify a biological target), identification tasks (where tasks are performed to find molecules that affect targets), screening tasks (where tasks are performed to screen identified molecules), and the like.

111 111 111 111 111 111 111 111 Further, when the industrial processperformed within the pharmaceutical industries includes testing tasks, the industrial processincludes tasks to test the impact of molecules on biological targets. Testing tasks include preclinical tasks (where testing is performed through chemical and biological methods), clinical tasks (where testing is performed on potential end users, approval tasks (where testing data is provided to regulatory bodies for approval), and the like. Moreover, when the industrial processperformed within the pharmaceutical industries includes manufacturing tasks, the industrial processincludes tasks to produce medicine for consumers. Manufacturing tasks include synthesizing tasks (where tasks are performed to produce the desired molecules), formulation tasks (where tasks are performed to create a final drug product from the molecules), and the like. Also, when the industrial processperformed within the pharmaceutical industries includes quality control tasks, the industrial processincludes tasks to assure the quality of produced medications. Quality control tasks include adherence tasks (where tasks are performed to ensure products are consistent), validation tasks (where tasks are performed to ensure produced medicine continues to satisfy desired performance), and the like. Further, when the industrial processperformed within the pharmaceutical industries includes compliance tasks, the industrial processincludes tasks to ensure that medications comply with regulations. Compliance tasks include documentation tasks (where tasks are performed to gather documentation related to the medication), inspection tasks (where tasks are performed to support regular audits), and safety tasks (where tasks are performed to ensure the continued safety of the medicine).

111 122 100 117 121 119 117 121 111 117 111 117 111 In further embodiments, to perform the industrial processat the site, the systemmay include one or more sensors, one or more actuators, and one or more controllers. As used herein, the sensorsand actuatorsrepresent components or equipment in an industrial system that perform one or more of a wide variety of functions associated with a process. In particular, as used herein, the sensorsinclude equipment that can measure or evaluate a wide variety of characteristics of the industrial process. For example, the sensorsmay measure characteristics that include flow, pressure, temperature, inventory levels, material compositions, and other measurable characteristics of the industrial process.

111 121 111 121 111 121 111 121 In additional embodiments, where the sensors measure information about the industrial process, the one or more actuatorsinclude equipment that is controllable to perform one or more steps of the industrial process. For example, the actuatorsmay include distillation columns, cracking equipment, smelters, or other equipment that can perform steps in industrial processesin various industrial domains. Additionally, the one or more actuatorsmay include equipment that controls one or more characteristics of the industrial process. For example, the one or more actuatorsmay include valves that are controllable to control the amount of material provided for a process. the process system, such as valve openings.

100 119 122 122 122 119 100 111 119 117 121 119 119 111 119 In further embodiments, the systemmay include one or more controllerslocated at the siteor located remotely to the sitebut communicatively connected to equipment located at the site. The one or more controllerscan be used within the systemto control the performance of various functions that are part of the one or more industrial processes. For example, a first set of controllersmay use measurements from one or more sensorsto control the operation of one or more actuators. A second set of controllerscould then be used to optimize the control logic or other operations performed by the first set of controllers. A third set of controllerscould perform additional functions related to the industrial process. The controllerscould, therefore, support a combination of approaches, such as regulatory control, advanced regulatory control, supervisory control, and advanced process control.

119 119 119 In some embodiments, each controllermay include any suitable structure for controlling one or more aspects of an industrial process. At least some of the controllerscould, for example, represent proportional-integral-derivative (PID) controllers or multivariable controllers, such as controllers implementing model predictive control (MPC) or other advanced predictive control (APC). In a particular example, each controllercould represent a computing device running a real-time operating system, a WINDOWS operating system, a LINUX operating system, or other operating system.

119 119 117 121 111 119 119 117 121 In additional embodiments, at least one of the controllerscould denote a controller that operates according to a predetermined process model. For example, a control model may control the controllersto operate using one or more process models to determine, based on measurements from one or more sensors, how to adjust one or more actuatorsto control the performance of the processaccording to the process model. In some embodiments, each model that directs the operation of the controllersmay direct the controllersto be responsive to various process variables that could include information acquired from the sensorsor information about the operation of the one or more actuators.

115 111 115 111 111 115 117 115 111 115 113 In some embodiments, a process manager or other usermay interact with the process. The usermay control the industrial processby interacting with the equipment that performs one or more of the steps of the industrial process. Additionally, the usermay perform some of the sensing functions similar to the functions provided by the sensors. For example, the usermay acquire samples of materials used within or produced by the industrial process. In some embodiments, the usermay also be a stakeholder.

100 122 111 117 121 117 121 119 115 111 115 111 In some embodiments, the systemmay include at least one network that enables electronic communications between various components located at the siteassociated with the performance of the one or more industrial processes. For example, the sensorsand the actuatorsmay be connected to a network that connects the sensorsand the actuatorsto the one or more controllers. Additionally, the network may be connected to a human-machine interface through which the usercan interact with the industrial process. Accordingly, the usermay communicate through the human-machine interface to control or send communications associated with the industrial processthrough the network. The network may be implemented as any suitable network or combination of networks. For example, the network may be one or more of an Ethernet network, an electrical signal network, a pneumatic control signal network, a wireless network, or any other type of network.

113 113 111 113 111 111 122 111 113 111 In certain embodiments, the stakeholdermay refer to an individual, group, or organization that has a vested interest or is affected by an industrial process. For example, a stakeholdermay be an individual or group affected by the outcome of the industrial process. Also, a stakeholdermay be an individual, group, or organization that makes decisions related to the execution of the industrial process. In some implementations, where the industrial processis performed at a sitethat is associated with a company or other organization in control of the industrial process, the stakeholdermay be an internal stakeholder or an individual who is employed by or otherwise part of the organization. For example, internal stakeholders may be owners, managers, employees, unions, and other groups or individuals who are part of an organization that controls the industrial process.

113 111 113 111 111 111 111 111 111 111 In additional situations, a stakeholdermay be an individual or organization that is not part of the organization that controls the industrial process. For example, a stakeholdermay be an external stakeholder. As used herein, an external stakeholder may be an individual or group who is not part of the organization in control of the industrial processbut benefits or participates in the products produced by the industrial process. For example, an external stakeholder may be a customer that buys a product produced by the industrial process, a supplier of materials to the organization for the performance of the industrial process, a distributor that helps distribute the products of the industrial process, competitors with the organization in control of the industrial process, investors and financial institutions that have a financial interest in the organization in control of the industrial process, and advisors that provide legal and consulting services to the organization.

113 111 111 113 111 113 122 111 111 In further situations, a stakeholdermay be an individual or organization that performs a regulatory role in relation to the performance of the industrial process. For example, agencies may create and enforce laws or rules that affect the execution of the industrial process. These agencies may be government bodies, industry standards organizations, taxing authorities, and the like. Also, a stakeholdermay include individuals or organizations that are tangentially affected by the performance of the industrial process. For example, the stakeholdersmay include individuals and communities that live close to the sitewhere the industrial processis performed, news services that may disseminate information about the industrial process, among other individuals affected by the process.

113 111 113 111 113 111 111 111 111 113 111 113 111 113 111 111 As shown, many different types of individuals may act as a stakeholderin the performance of an industrial process. Yet, the various stakeholdersmay have different interests and motivations with regard to the industrial process. Thus, to effectively balance these different interests, it is important for the various stakeholderto accurately understand the industrial processand their relationship to the industrial process. However, industrial processesmay be extremely complex, making it difficult for the stakeholders to understand the industrial process. Further, many of the stakeholdersdo not have access to the equipment that performs the industrial processor have the knowledge to operate and understand the associated equipment. Thus, even though many individuals are stakeholderin an industrial process, many barriers exist that prevent the stakeholderfrom understanding an industrial processand their relationship to the industrial process.

113 111 107 111 111 107 113 111 107 113 107 111 113 107 111 111 In certain embodiments, as many of the stakeholderslack access to the equipment involved in the industrial process, it is often helpful to create modelsof the industrial processthat can simulate one or more aspects of the industrial process. In particular, a modelmay enable a stakeholderto simulate aspects of the industrial process. However, the modelsare often mathematically complex and often present themselves as black boxes to the stakeholders. Thus, while modelsincrease access to the industrial processfor the various stakeholders, they often fail to provide sufficient understanding of the modelin a way that the stakeholder can evaluate the industrial processand their relationship to the industrial process.

107 111 107 113 111 111 107 111 107 111 As stated above, the modelmay refer to a mathematical, computational, or conceptual representation of the industrial process. The modelmay allow the stakeholdersto simulate, analyze, control, and gain further understanding of the associated industrial process. Also, as the industrial processmay be complex, involving numerous variables such as raw materials, energy, equipment, labor, and environmental conditions, the modelmay provide a simplified representation of the industrial process. Additionally, the modelmay be created using one or more of many different modeling techniques that can be used to represent different aspects of the industrial process.

107 107 111 107 107 In certain implementations, the modelmay be a deterministic model, where the modelprovides a precise, predictable output for a given set of inputs. In particular, a deterministic model assumes that all the relationships between inputs and outputs are known and fixed. They are particularly useful when the industrial processis well understood and variability is minimal or controllable. Examples of deterministic models may include mathematical models such as linear or non-linear models. Linear models can be used when changes in inputs have a proportional effect on outputs. Non-linear models can be used when the relationships between inputs and outputs are more complex. Further, a deterministic models may be an empirical model. When the modelis a deterministic model, the modelis based on experimental data or other observations. Often, empirical models are built using statistical techniques that involve fitting data to an equation or curve.

107 107 111 In additional implementations, the modelmay be a stochastic model, where the modelincorporates randomness or uncertainty in the model input variables. Stochastic models are useful because they incorporate variability that is often present in an industrial process. In contrast to the deterministic outputs of the deterministic models, a stochastic model provides probabilistic outputs. Examples of stochastic models may include Monte Carlo simulations, Markov models, queuing models, and other probabilistic methods. In particular, a Monte Carlo simulation may be useful for modeling the uncertainty in a process through the use of random sampling techniques. Markov models are useful for processes where the future state of the system depends only on a current state and are commonly used in systems that have discrete states. Queuing models may be useful for modeling inventory management and other processes where entities queue for a service.

107 In further implementations, the modelmay be a first principles model. A first principles model can be a mechanistic model that is based on fundamental physical, chemical, or biological laws that govern a particular process. A first principles model may be represented through equations that describe the associated fundamental laws. For example, a first principles model may be based on conservation laws, kinetics and thermodynamics, and other known fundamental laws. When the first principles model is based on conservation laws, the first principles model may use mass balance equations to track the flow of materials through a process, ensuring that the mass is conserved. Similarly, the first principles model may employ energy balancing to ensure that the energy inputs and outputs of the process follow the conservation of energy. When the first principles model is based on kinetics and thermodynamics, the first principles model may predict the rate of reactions based on physical characteristics and how systems and materials change in response to variations in the environment and the inputs.

107 In some implementations, the modelmay be a dynamic model. Dynamic models may be used to simulate processes that evolve over time. Dynamic models may be able to handle both continuous and discrete changes to account for time-dependent phenomena. Examples of dynamic models may include ordinary and differential equations that describe how a system changes with time in one or more dimensions. Another example may be a control system model where process variables are maintained using feedback loops. Control system models may employ dynamic equations that predict system behavior and dictate appropriate control actions.

107 107 107 107 In additional implementations, the modelmay be a data-driven model. A data-driven model may use historical data, learning algorithms, and statistical methods to predict and optimize outcomes without needing detailed physical knowledge of a system. For example, a data-driven model may be a machine learning model, a statistical process control model, and the like. When the modelis a machine learning model, the modelmay be built using training algorithms on large datasets of information to recognize patterns and relationships in the dataset of information. When the modelis a statistical process control model, statistical techniques may be used to identify trends, variations, deviations, and other information in historical data.

107 107 107 107 107 In certain embodiments, when the modelis produced using machine learning. The training algorithms are used to train the modelusing very large data sets of information, where the model“learns” from the data how to identify patterns, make decisions, or generate additional information with minimal human intervention. Further, a model, produced using machine learning methods, may be capable of improving performance over time because machine learning models are adaptable to additional data. Generally, machine learning model training is performed using one or more various learning paradigms. These learning paradigms include supervised learning, unsupervised learning, and reinforcement learning. When training the model, a training computer system may use a combination of learning paradigms.

107 107 107 107 When the modelis trained using supervised learning, the model is trained using labeled datasets. In particular, training data may include a dataset where the inputs to a model are known, and the output that the model should produce in response to the known inputs is also known. Thus, during training, the modellearns the relationship between the input data and the desired output. As the modellearns the relationship between the input data and associated outputs, the modelmay improve its ability to make generalized predictions upon receiving new unseen or non-labeled data. Various machine learning algorithms may be used to train a model using supervised learning. These algorithms may include combinations of decision trees, support vector machines (SVM), neural networks, and the like.

107 107 107 When the modelis trained using unsupervised learning, the learning is focused on identifying patterns in input data that lack labeled outputs. For example, in contrast to learning relationships between inputs and outputs, the modelmay learn to organize data into groups or clusters based on similarities or hidden structures. Various machine learning algorithms may be used to train the modelusing unsupervised learning. These algorithms include clustering, principal component analysis, and dimensionality reduction, among other unsupervised learning techniques.

107 107 107 107 107 When the modelis trained using reinforcement learning, the modellearns through interaction with data received from outside the modeland then receives feedback in the form of rewards or penalties through the interactions. Through these interactions, the modellearns to perform actions associated with rewards and to avoid actions associated with penalties. Reinforcement learning is an effective tool for training a modelthat performs decision-making and also for optimizing actions over time.

107 107 107 107 107 107 The modelmay be trained within a variety of processing environments. For example, the modelmay be trained on a local computing system, a computing center, or a cloud-based platform. Often, the selection of the processing environment for the training of the modeldepends on the computational complexity of the modeland the size of the data gathered for training the model. Where the training environment is a local computing environment, the computing systems used to train themay include one or more locally operating computers, such as workstations or servers. Often, workstations and servers used to train machine learning models include one or more high-performance CPUs or GPUs. However, local environments are often constrained in their processing capabilities and are generally used for training smaller-scale models or initial testing of machine learning algorithms. In contrast, where the training environment is a distributed system, the computing system includes multiple computing devices (like workstations and servers) distributed across one or more locations. Further, the multiple computing devices often train the modelusing parallel computation. These distributed systems are often suitable for training models with larger datasets and complexity. Further, a cloud-based platform may provide scalable resources for training and deploying machine learning models.

107 107 107 In certain embodiments, when training the model, the computing systems may execute instructions that implement algorithms developed using a variety of programming languages and specialized libraries. For example, a model developer may use programming languages such as Python, R, Java, C++, and Matlab, among others, which offer different benefits. For example, the model developer may use Python because it supports many libraries that facilitate the implementation of machine learning models. A model developer may use R to perform statistical analysis through libraries that are optimized for data exploration and modeling. The model developer may use Java for its scalability and production-ready solutions. Further, the model developer may use C++when the modelrequires low-level memory management. Matlab may also be used to perform research into machine learning algorithms, prototyping, and data visualization. Other programming languages can be used as well, depending on the characteristics of the training of the model.

107 107 107 113 113 107 113 113 107 As described above, the modelmay be constructed using one of multiple methods. Further, the modelmay be a hybrid model that uses different model types to represent one or more aspects of the model. As the models often use advanced mathematical techniques to represent a system and, in some implementations, have black-box type qualities (like in machine learning models), for many stakeholders, it is often as challenging to understand the model as it is to understand the actual process. In particular, it is difficult for the stakeholderto understand the relationships that the modelto arrive at the outputs from the inputs. Thus, when a stakeholderwould like to know how to change the process inputs to achieve desired process outputs, stakeholdersare often unable to understand the relationship between the inputs and outputs from interacting with typically complex and opaque models.

101 108 107 108 107 101 103 105 105 103 107 111 105 103 108 105 107 108 In certain embodiments, the system may include a computing systemconfigured to execute a model inversion applicationand to also execute the model. To execute the inversion applicationand the model, the computing systemincludes a processorand memory. In particular, the memorystores executable code that the processorexecutes to perform the modelassociated with aspects of the industrial process. Also, the memorystores executable code that the processorexecutes as the model inversion application. Additionally, the memorymay store other data that may be both executable or non-executable to support the execution of the modeland the model inversion application.

101 101 103 103 103 103 In certain embodiments, the various systems and methods described above, in particular, the computing systemand the applications executed on the computing system, may be performed by hardware or through the execution of instructions performed by the one or more processors. For example, the processorsand/or other computational devices may be implemented using software, firmware, hardware, or an appropriate combination thereof. The processorsor other computational devices may be supplemented by, or incorporated in, specially-designed application-specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). The processorsand other computational devices can also include or function with software programs, firmware, or other computer-readable instructions for carrying out various process tasks, calculations, and control functions used in the methods and systems described herein.

107 108 The methods described herein may be implemented or controlled by computer-executable instructions, such as program modules or components, executed by the one or more processors or other computing devices. For example, the program modules may include the modeland the model inversion application. Generally, program modules include routines, programs, objects, data components, data structures, algorithms, and the like, which perform particular tasks or implement particular abstract data types.

Instructions for carrying out the various process tasks, calculations, and generation of other data used in the operation of the methods described herein may be implemented in software, firmware, or other computer-readable instructions. These instructions are typically stored on appropriate computer program products that include computer-readable media used to store computer-readable instructions or data structures. The computer-readable media may store computer-readable instructions or data structures. Such a computer-readable medium may be available media that can be accessed by a general-purpose or special-purpose computer or processor, or any programmable logic device.

Suitable computer-readable storage media may include, for example, non-volatile memory devices that include semiconductor memory devices such as Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory devices; magnetic disks such as internal hard disks or removable disks; optical storage devices such as compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs; or any other media that can carry or store desired program code as computer-executable instructions or data structures.

100 109 101 109 113 101 109 113 101 109 101 107 108 In further embodiments, the systemincludes a user interfacethat is coupled to the computing system. The user interfaceis an electronic device that allows a stakeholderto interact with the computing system. The user interfacemay be a monitor, a mouse, a touchscreen, a keyboard, a microphone, or other device that provides a way for a stakeholderto interact with the computing system. In some implementations, though not shown, the user interfacemay be another computing device that communicates with the computing systemthrough a network and provides access to the modeland the model inversion application.

108 107 113 107 108 108 111 107 111 In some embodiments, the model inversion applicationis an application that inverts the modelto aid in helping improve the understanding of the stakeholderof the model. As the model functions by receiving inputs and generating outputs, inversion, as defined herein, refers to providing a desired output to the model inversion application, where the model inversion applicationthen identifies input parameters for the modeled processthat will cause the modelto produce the desired output. Often, the desired output for the processmay be a key performance indicator (KPI) such as product quality, product quantity, and the like.

108 107 107 108 113 109 108 113 115 111 111 113 107 111 108 113 In certain embodiments, the model inversion applicationmay invert the modelby providing initial input parameters to the model, then iteratively adjusting the input parameters for the model until the model provides the desired output. The model inversion applicationthen provides the stakeholderthe input parameters that resulted in the desired output through the user interface. After receiving the input parameters from the model inversion application, the stakeholdersor the usermay then adjust the operational parameters of the industrial processto cause the industrial processto produce the desired outputs. Thus, the stakeholdersmay learn information about the modeland the process, as the model inversion applicationprovides the stakeholderadditional understanding of the relationships between inputs and outputs.

113 108 113 108 107 107 108 107 In some implementations, a stakeholdermay control one or more of the inputs, and the model inversion applicationmay identify values for the other inputs that may potentially result in outputs with the desired performance indicators. For example, stakeholdermay set some of the inputs as constants, and then the model inversion applicationiteratively changes the other inputs until modelprovides the desired outputs. Further, as described above, the modelmay be one or more of various types of models. However, the model inversion applicationmay operate agnostically with regards to the type of modelsave having access to the inputs and outputs of the model.

108 107 111 107 108 In some embodiments, the model inversion applicationmay receive a desired performance indicator for a model output that is outside of what the modelcan produce. For example, in some embodiments, the inputs may be constrained within defined ranges that reflect physical limitations on the inputs. For example, first principles may be applied to identify desired performance indicators and inputs that are outside a range of physical possibilities. Also, information about the limitations of the physical capabilities of the equipment that implements the industrial processmay be used to limit the range of possible inputs for the model. Accordingly, the model inversion applicationmay iteratively change the inputs within a defined range of possible inputs.

107 108 108 107 107 113 109 113 108 108 113 109 107 108 113 109 107 108 109 In certain embodiments, after iteratively submitting the possible inputs to the modeland checking the outputs against the desired performance indicators, the model inversion applicationmay identify the outputs that are closest to the desired performance indicators. Alternatively, the model inversion applicationmay iteratively change the inputs until the output fails to improve with reference to the desired performance indicators. Alternatively, the different combinations of input parameters provided to the modelmay be iteratively changed according to discrete increments, and the combination of input parameters that causes the modelto generate output values closest to the desired performance indicators may be provided to the stakeholdersthrough the user interface. Also, the stakeholdersmay establish output thresholds or an acceptable range of acceptable values for the desired performance indicators, and then the model inversion applicationmay identify similar ranges of input parameters that yield the acceptable range of desired performance indicators. The model inversion applicationmay then provide the acceptable range of input parameters to the stakeholderthrough the user interface. In some implementations, the user interface application may fail to identify inputs that result in an acceptable output. For example, based on the first principles, the desired performance indicator may be physically impossible, or the modelis unable to produce the desired performance indicator as an output. In such a situation, the model inversion applicationmay provide inputs to the stakeholderthrough the user interface, which will cause the modelto produce the outputs that are closest to the desired performance indicators. Alternatively, the model inversion applicationmay provide a notification that indicates that the desired performance indicators are not achievable through the user interface.

108 113 115 111 111 In some embodiments, when the model inversion applicationidentifies input parameters associated with a desired performance indicator, the stakeholdermay provide the input parameters to the userto configure the actual input parameters of the industrial processto match the identified input parameters. Thus, the actual performance of the industrial processmay be improved and/or optimized to produce goods or perform a service according to one or more desired performance indicators.

115 111 108 117 111 111 105 108 107 107 107 107 107 In some embodiments, after the useradjusts the input of the industrial processto match the input parameters identified by the model inversion application, the sensorsmay monitor the performance indicators for the industrial process. In some embodiments, the resultant performance indicators from the actual industrial processmay be saved in a memory (such as the memoryor other memory device), where the resultant performance indicators are associated with the desired performance indicators and the input parameters identified by the model inversion application. The desired performance indicators, resultant performance indicators, and input parameters may then be used to improve the performance of the modelbased on the method used to create the model. For example, the additional data may be used to validate the modelwhen the modelis a machine learning model or provide additional statistical information that can be used to improve the performance of the model.

108 113 107 111 107 108 111 111 113 108 107 111 108 113 As described above, the model inversion applicationmay help a stakeholderimprove their understanding of the modeland the processrepresented by the model. In particular, the model inversion applicationmay improve the understanding of the relationships between the input parameters of the industrial processand the outputs of the industrial process. Further, as the stakeholderuses the model inversion application, additional data is generated that can improve the performance of the model, which improves the accuracy of the modeled relationships between the input parameters and the outputs of the industrial process. As the accuracy of the modeled relationship improves, the model inversion applicationwill also help improve the understanding of the stakeholder.

2 FIG. 200 200 108 113 109 108 111 107 200 203 108 107 108 113 107 111 is a flowchart diagram of a methodfor evaluating inputs through model inversion. In certain embodiments, the methodproceeds at 201, where a desired output value is received. For example, the model inversion applicationmay receive a desired output value from a stakeholderthrough the user interface. The model inversion applicationmay then identify that the desired output value is the one or more desired performance indicators for an industrial processthat is represented by a model. After receiving the desired output value, the methodproceeds at, where initial input parameter values are identified. For example, the model inversion applicationmay be configured to have one or more default initial input parameters for the various input parameters of the model. Alternatively, the model inversion applicationmay receive initial input parameter values from the stakeholder. In another implementation, the modelmay use the current or most recently measured input parameters being used for the industrial process.

200 205 107 108 107 107 200 207 107 108 107 108 In additional embodiments, when the initial input parameter values are identified, the methodproceeds at, where the identified input parameter values are provided to the model. For example, the model inversion applicationmay provide the identified input parameter values to the model. When the input parameter values are provided to the model, the methodproceeds at, where output values are received from the model. For example, the model inversion applicationmay receive the outputs generated by the modelin response to receiving the input parameters from the model inversion application.

108 107 209 108 107 113 108 107 200 211 113 109 113 111 107 111 In further embodiments, when the model inversion applicationhas received the input parameters from the model, the method proceeds at, where it is determined whether the output values equal the desired output values. For example, the model inversion applicationmay compare the output values received from the modelagainst the desired output values received from the stakeholder. In some implementations, the model inversion applicationmay determine whether the output values are within a defined acceptable range of the desired output value or other measures indicating that the output from the modelis acceptable in reference to the desired output value. If the output values are acceptable, the methodproceeds at, where the input parameters and the desired output values are provided as outputs to the stakeholderthrough the user interface. The stakeholdermay then use the output information to adjust the industrial processor improve their understanding of the modeland industrial process.

200 213 108 107 107 108 108 200 217 200 205 107 In some embodiments, when the output values are not acceptable with respect to the desired output values, the methodproceeds at, where it is determined whether additional changes may be made to the input parameters. For example, the model inversion applicationmay iteratively change the input parameters provided to the modeland provide the changed input values to the model. However, the input parameters may have a limited number of possible changes that the model inversion applicationmay iteratively perform. The number of possible changes may be limited either by an allowed execution time or by a finite number of possible changes to the input values. When the model inversion applicationdetermines that there are still possible changes that can be made to the input values, the methodproceeds at, where the input parameter values are adjusted and the methodreturns to, where the adjusted input parameter values are provided to the model.

108 108 108 200 215 108 113 109 108 When the model inversion applicationhas exhausted the number of possible changes or timed out without achieving an acceptable output value, the model inversion applicationmay determine that there are no additional changes. When the model inversion applicationdetermines that there are no additional changes, the methodmay proceed at, where the current input parameters and the output values are provided as outputs. For example, the model inversion applicationmay provide the most recently used input parameter values and associated model outputs to the stakeholderthrough the user interface. Additionally, the model inversion applicationmay provide an indication that the output values are not within an acceptable range with respect to the desired output value.

3 FIG. 300 300 113 113 108 300 113 108 108 113 300 is an exemplary illustration of a user interface displayfor a system for evaluating inputs through model inversion. In particular, the user interface displaymay provide a graphical representation to a stakeholderthat enables the stakeholderto exchange information with a model inversion application. For example, the user interface displaymay be a graphical user interface that provides various fields for receiving information from the stakeholderfor use by the model inversion applicationand for displaying information generated from the model inversion applicationfor use by the stakeholder. The user interface displaymay be constructed using one or more of various technologies used by those with skill in the art for creating graphical user interfaces.

300 301 301 113 108 300 301 300 113 300 301 113 As illustrated, the user interface displayincludes four graphical components: a labeled spinner box, two buttons, and an output text field. As shown, the spinner box may be configured to function as a target value input. The target value inputallows a stakeholderto provide a desired performance indicator to the model inversion applicationthrough the user interface display. While only one target value inputis shown, the user interface displaymay include other components that allow the stakeholderto provide one or more input values through the user interface display. Also, while a spinner box is illustrated, the target value inputmay include other types of graphical components (like a table, a text field, a combo box, and the like) that can receive the desired performance indicator from a stakeholder.

113 301 113 305 108 107 111 111 301 108 107 113 305 108 107 108 113 307 108 108 113 300 303 113 108 After a stakeholderhas provided a value for the target value input, the stakeholdermay select a run buttonthat is configured to direct the model inversion applicationto interact with the modelto identify the input parameters for the industrial processthat will cause the industrial processto achieve the desired performance indicator provided through the target value input. While a button is illustrated, any other type of visual component that directs the model inversion applicationto begin interacting with the modelmay be used. After the stakeholderselects the run buttonand the model inversion applicationcompletes interacting with the model, the model inversion applicationmay display information instructing the stakeholderhow to achieve the desired performance indicator in the application output, which is represented as a text field, though the model inversion applicationmay use other means of conveying this information. For example, the model inversion applicationmay provide this information as a downloadable file, an email, a combination of visual components, or other techniques used to convey information between stakeholdersand computational equipment. In some implementations, the user interface displaymay also include a resetthat allows the stakeholderto clear information related to previous requests. Alternatively, the model inversion applicationmay be configured to clear information related to previous requests when new information is requested or input information is changed.

300 301 113 108 107 113 301 113 305 108 107 108 107 108 102 307 113 111 113 303 In the specific example illustrated, the user interface displayreceives the target value inputfrom the stakeholderthat will cause the model inversion applicationto interact with the modelto determine input parameters for an oil refining process that will yield an output that meets a reformate octane level specified as the desired performance indicator. After the stakeholderenters the reformate octane level into the target value input, the stakeholdermay then select the run buttonto instruct the model inversion applicationto interact with the model. After the model inversion applicationcompletes the interaction with the model, the model inversion applicationthen provides input parameters for the oil refining process that may lead to a refined process output having a reformate octane level of. For example, the application outputdisplays input parameters for multiple process parameters for an oil refining process. The stakeholdermay then use the provided input parameters to improve the actual industrial process. Further, the stakeholdermay reset the interaction using the resetto find different sets of input parameters that can be used to achieve different desired performance indicators.

4 FIG. 400 107 108 400 108 107 400 401 113 108 109 401 400 403 108 107 107 109 is a flowchart diagram illustrating a methodfor improving the performance of a modelusing model inversion, such as the model inversion provided by the model inversion application. As illustrated, the methodis an iterative loop that can be implemented throughout the lifecycle of the model inversion applicationand the model. As shown, the methodmay proceed at, where a prediction is made. The prediction may be the desired performance indicator provided by a stakeholderto the model inversion applicationthrough the user interface. After performing the prediction, the methodproceeds at, where one or more simulations are performed. For example, the model inversion applicationmay provide input parameters for the model, from which the modelgenerates output values, where the output values are associated with the desired performance indicator received through the user interface.

108 400 405 108 107 107 107 400 407 108 113 109 108 400 409 113 111 111 111 107 107 108 After the model inversion applicationbegins simulations, the methodmay proceed at, where the input parameters are optimized. For example, the model inversion applicationmay iteratively adjust the input parameters provided to the modelto identify the input parameters that cause the modelto produce the desired performance indicator. When the modelreceives input parameters that produce the desired performance indicator, the methodproceeds at, where the optimized input parameters are recommended. For example, the model inversion applicationwill provide the identified input parameters to the stakeholderthrough the user interface. After the model inversion applicationprovides the recommended input parameters, the methodproceeds at, where the recommended input parameters are evaluated. For example, the stakeholderor other user may configure an industrial processwith the recommended input parameters and then may measure the performance of the industrial process. Then, the input parameters can be evaluated to see if the industrial processachieves the desired performance indicator. The evaluation information may then be used to improve the model, where the improved modelmay then be used to interact with the model inversion applicationto identify input parameters that help achieve the desired performance indicators.

5 FIG. 500 500 501 500 503 500 505 500 507 is a flowchart diagram of a methodfor evaluating inputs through model inversion. In particular, the methodproceeds at, where one or more desired output values are received from a stakeholder through a user interface. Further, the methodproceeds at, where one or more combinations of one or more input values are iterated through a model of an industrial process, wherein the model outputs outputs one or more output values and the one or more input values are associated with potential input parameters for the industrial process. Additionally, the methodproceeds at, where a combination of the one or more input values that causes the model to generate output values associated with the one or more desired output values are identified. Moreover, the methodproceeds at, where the combination of the one or more input values are provided to the stakeholder through the user interface.

Example 1 includes a system comprising: a memory configured to store a model of a physical process, wherein the model is configured to receive one or more input values and provide one or more output values, wherein the one or more input values represent potential input parameters for the physical process and the one or more output values represent potential measures of process outputs; an interface capable of receiving one or more desired output values for the physical process; and one or more processors executing computer executable instructions associated with an application that cause the one or more processors to: receive the one or more desired output values; test a plurality of values for at least one of the one or more input values; identify a combination of the one or more input values that is associated with the one or more desired output values; and provide the combination for output through the interface.

Example 2 includes the system of Example 1, wherein the physical process is controlled based on the combination of the one or more input values.

Example 3 includes the system of any of Examples 1-2, wherein the one or more processors are further configured to: test multiple output values; and identify the one or more input values associated with each of the multiple output values.

Example 4 includes the system of any of Examples 1-3, wherein the one or more processors is configured to determine that the combination of the one or more input values is associated with the one or more input values through at least one of: the combination of the one or more input values cause the one or more output values to equal the one or more desired output values; the combination of the one or more input values cause the one or more output values to be within an acceptable range of the one or more desired output values; and the combination of the one or more input values is a closest combination of tested combinations of the plurality of values that caused the one or more output values to be closest to the one or more desired output values.

Example 5 includes the system of any of Examples 1-4, wherein the one or more processors are further configured to provide a notification of no identified combination through the interface, when the one or more processors is unable to identify the combination associated with the one or more desired output values.

Example 6 includes the system of any of Examples 1-5, wherein the one or more processors are further configured to: save the combination and receive measures of physical outputs from the physical process when implementing input parameters associated with the combination; and improve performance of the model using the combination and the received measures of the physical outputs.

Example 7 includes the system of any of Examples 1-6, wherein the computer executable instructions cause the one or more processors to constrain the plurality of values within physical limits of the potential input parameters associated with the plurality of values.

Example 8 includes the system of any of Examples 1-7, wherein the model is at least one of: a machine learning model; a statistical model; and a mathematical model.

Example 9 includes the system of any of Examples 1-8, wherein the application displays a user interface for receiving the one or more desired output values from a user and providing the combination of the one or more input values through the user interface to the user.

Example 10 includes a method comprising: receiving one or more desired output values from a stakeholder through a user interface; iterating through one or more combinations of one or more input values for a model of an industrial process, wherein the model outputs one or more output values and the one or more input values are associated with potential input parameters for the industrial process; identifying a combination of the one or more input values that causes the model to generate output values associated with the one or more desired output values; and providing the combination of the one or more input values to the stakeholder through the user interface.

Example 11 includes the method of Example 10, further comprising: configuring the industrial process with the combination; and evaluating the model based on outputs of the model as configured with the combination.

Example 12 includes the method of any of Examples 10-11, wherein identifying the combination of the one or more input values comprises identifying multiple combinations of the one or more input values, wherein each combination of the one or more input values is associated with a different output value.

Example 13 includes the method of any of Examples 10-12, wherein identifying the combination of the one or more input values further comprises at least one of: determining that the combination causes the model to produce the one or more output values that equal the one or more desired output values; determining that the combination causes the model to produce the one or more output values to be within an acceptable range of the one or more desired output values; and determining that the combination of the one or more input values causes the model to produce the one or more output values that are closest to the one or more desired output values than other tested combination of input values provided to the model.

Example 14 includes the method of any of Examples 10-13, further comprising providing a notification of no identified combination to the stakeholder through the user interface when the combination is not able to be identified.

Example 15 includes the method of any of Examples 10-14, further comprising constraining the one or more input values to be within physical limits of the potential input parameters.

Example 16 includes the method of any of Examples 10-15, wherein the model is at least one of: a machine learning model; a statistical model; and a mathematical model.

Example 17 includes the method of any of Examples 10-16, further comprising providing a graphical user interface through the user interface for receiving the one or more desired output values from the stakeholder and providing the combination to the stakeholder.

Example 18 includes a system comprising: a memory configured to store a model of an industrial process and a model inversion application, wherein the model is configured to receive one or more input values and generate one or more output values, wherein the one or more input values represent potential input parameters for the industrial process and the one or more output values represent potential measures of process outputs; a user interface capable of receiving one or more desired output values for the industrial process from one or more stakeholders and providing the one or more output values to the one or more stakeholders; and one or more processors executing the model inversion application that cause the one or more processors to: receive the one or more desired output values through the user interface; iterate through a plurality of input values for at least one of the one or more input values; identify a combination of the one or more input values that causes the model to generate output values associated with the one or more desired output values; and provide the combination to the one or more stakeholders through the user interface.

Example 19 includes the system of Example 18, wherein the one or more processors are configured to determine that the combination of the one or more input values is associated with the one or more input values through at least one of: the combination of the one or more input values cause the one or more output values to equal the one or more desired output values; the combination of the one or more input values cause the one or more output values to be within an acceptable range of the one or more desired output values; and the combination of the one or more input values is a closest combination of tested combinations of the plurality of input values that caused the one or more output values to be closest to the one or more desired output values.

Example 20 includes the system of any of Examples 18-19, wherein the one or more processors are further configured: to save the combination and receive measures of physical outputs from the industrial process when implementing input parameters associated with the combination; and improve performance of the model using the combination and the received measures of the physical outputs.

Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that any arrangement, which is calculated to achieve the same purpose, may be substituted for the specific embodiments shown. Therefore, it is manifestly intended that this invention be limited only by the claims and the equivalents thereof.