Predicting Resource-Related Values Using Multi-Dimensional Machine Learning-Based Techniques

A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.

In many contexts, enterprises and/or other organizations face multiple variables which can affect outcomes for resource-related processes, particularly with respect to one or more resource-related values during such processes. However, conventional resource management techniques typically fail to determine accurate resource-related values at various portions of such processes, which can often lead to negative outcomes resulting in losses and inefficiencies with respect to time and other resources.

Illustrative embodiments of the disclosure provide techniques for predicting resource-related values using multi-dimensional machine learning-based techniques.

An exemplary computer-implemented method includes obtaining data pertaining to at least one resource-related activity involving at least one resource, and predicting one or more values associated with the at least one resource by processing at least a portion of the obtained data using one or more machine learning techniques. The method also includes predicting one or more values attributed to the at least one resource-related activity by processing the at least a portion of the obtained data using the one or more machine learning techniques. Further, the method additionally includes performing one or more automated actions based at least in part on at least a portion of the one or more predicted values associated with the at least one resource and at least a portion of the one or more predicted values attributed to the at least one resource-related activity.

Illustrative embodiments can provide significant advantages relative to conventional resource management techniques. For example, problems associated with losses and inefficiencies with respect to time and other resources are overcome in one or more embodiments through automatically predicting values for individual resources as well as values for activities involving the one or more individual resources using machine learning techniques.

These and other illustrative embodiments described herein include, without limitation, methods, apparatus, systems, and computer program products comprising processor-readable storage media.

Illustrative embodiments will be described herein with reference to exemplary computer networks and associated computers, servers, network devices or other types of processing devices. It is to be appreciated, however, that these and other embodiments are not restricted to use with the particular illustrative network and device configurations shown. Accordingly, the term “computer network” as used herein is intended to be broadly construed, so as to encompass, for example, any system comprising multiple networked processing devices.

shows a computer network (also referred to herein as an information processing system)configured in accordance with an illustrative embodiment. The computer networkcomprises a plurality of user devices-,-, . . .-M, collectively referred to herein as user devices. The user devicesare coupled to a network, where the networkin this embodiment is assumed to represent a sub-network or other related portion of the larger computer network. Accordingly, elementsandare both referred to herein as examples of “networks” but the latter is assumed to be a component of the former in the context of theembodiment. Also coupled to networkis automated resource-related value prediction systemand one or more additional systems(e.g., one or more services systems, one or more fulfillment systems, one or more sales and/or marketing systems, one or more quote and order management system, etc.).

The user devicesmay comprise, for example, mobile telephones, laptop computers, tablet computers, desktop computers or other types of computing devices. Such devices are examples of what are more generally referred to herein as “processing devices.” Some of these processing devices are also generally referred to herein as “computers.”

The user devicesin some embodiments comprise respective computers associated with a particular company, organization or other enterprise. In addition, at least portions of the computer networkmay also be referred to herein as collectively comprising an “enterprise network.” Numerous other operating scenarios involving a wide variety of different types and arrangements of processing devices and networks are possible, as will be appreciated by those skilled in the art.

Also, it is to be appreciated that the term “user” in this context and elsewhere herein is intended to be broadly construed so as to encompass, for example, human, hardware, software or firmware entities, as well as various combinations of such entities.

The networkis assumed to comprise a portion of a global computer network such as the Internet, although other types of networks can be part of the computer network, including a wide area network (WAN), a local area network (LAN), a satellite network, a telephone or cable network, a cellular network, a wireless network such as a Wi-Fi or WiMAX network, or various portions or combinations of these and other types of networks. The computer networkin some embodiments therefore comprises combinations of multiple different types of networks, each comprising processing devices configured to communicate using internet protocol (IP) or other related communication protocols.

Additionally, the automated resource-related value prediction systemcan have an associated resource-related databaseconfigured to store data pertaining to historical resource-related value data associated with various resource-related activities, historical total value data attributed to various resource-related activities, user data associated with various resource-related activities and/or resources, etc.

The resource-related databasein the present embodiment is implemented using one or more storage systems associated with the automated resource-related value prediction system. Such storage systems can comprise any of a variety of different types of storage including network-attached storage (NAS), storage area networks (SANS), direct-attached storage (DAS) and distributed DAS, as well as combinations of these and other storage types, including software-defined storage.

Also associated with the automated resource-related value prediction systemare one or more input-output devices, which illustratively comprise keyboards, displays or other types of input-output devices in any combination. Such input-output devices can be used, for example, to support one or more user interfaces to the automated resource-related value prediction system, as well as to support communication between the automated resource-related value prediction systemand other related systems and devices not explicitly shown.

Additionally, the automated resource-related value prediction systemin theembodiment is assumed to be implemented using at least one processing device. Each such processing device generally comprises at least one processor and an associated memory, and implements one or more functional modules for controlling certain features of the automated resource-related value prediction system.

More particularly, the automated resource-related value prediction systemin this embodiment can comprise a processor coupled to a memory and a network interface.

The processor illustratively comprises a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other type of processing circuitry, as well as portions or combinations of such circuitry elements.

The memory illustratively comprises random access memory (RAM), read-only memory (ROM) or other types of memory, in any combination. The memory and other memories disclosed herein may be viewed as examples of what are more generally referred to as “processor-readable storage media” storing executable computer program code or other types of software programs.

One or more embodiments include articles of manufacture, such as computer-readable storage media. Examples of an article of manufacture include, without limitation, a storage device such as a storage disk, a storage array or an integrated circuit containing memory, as well as a wide variety of other types of computer program products. The term “article of manufacture” as used herein should be understood to exclude transitory, propagating signals. These and other references to “disks” herein are intended to refer generally to storage devices, including solid-state drives (SSDs), and should therefore not be viewed as limited in any way to spinning magnetic media.

The network interface allows the automated resource-related value prediction systemto communicate over the networkwith the user devices, and illustratively comprises one or more conventional transceivers.

The automated resource-related value prediction systemfurther comprises resource-related data processor, multi-dimensional machine learning-based prediction engine, and automated action generator.

It is to be appreciated that this particular arrangement of elements,andillustrated in the automated resource-related value prediction systemof theembodiment is presented by way of example only, and alternative arrangements can be used in other embodiments. For example, the functionality associated with elements,andin other embodiments can be combined into a single module, or separated across a larger number of modules. As another example, multiple distinct processors can be used to implement different ones of elements,andor portions thereof.

At least portions of elements,andmay be implemented at least in part in the form of software that is stored in memory and executed by a processor.

It is to be understood that the particular set of elements shown infor predicting resource-related values using multi-dimensional machine learning techniques involving user devicesof computer networkis presented by way of illustrative example only, and in other embodiments additional or alternative elements may be used. Thus, another embodiment includes additional or alternative systems, devices and other network entities, as well as different arrangements of modules and other components. For example, in at least one embodiment, two or more of automated resource-related value prediction system, resource-related database, and additional system(s)can be on and/or part of the same processing platform.

An exemplary process utilizing elements,andof an example automated resource-related value prediction systemin computer networkwill be described in more detail with reference to the flow diagram of.

Accordingly, at least one embodiment includes predicting resource-related values using multi-dimensional machine learning techniques. As detailed herein, such an embodiment includes predicting one or more resource-related values at one or more moments during at least one resource-related activity (e.g., the unit price of a product at the beginning of a transactional opportunity lifecycle to reduce and/or minimize sales pipeline discrepancies). Such an embodiment includes leveraging one or more machine learning models trained on multi-dimensional resource-related information such as, for example, data pertaining to historical resource-related processes (e.g., historical sales data with respect to resource information, quantity data, user details, resource value (e.g., price), etc.). Predicting resource-related values using such a machine learning model can facilitate and/or initiate one or more corrective actions in connection with a corresponding resource-related activity.

By way merely of example, one or more embodiments can include predicting the unit resource (e.g., a particular product) price of a given resource associated with a corresponding resource-related activity (e.g., a transactional opportunity involving the resource) by leveraging at least one machine learning algorithm as a regressor which is trained using multi-dimensional, historical data (e.g., data pertaining to historical transactional opportunities that were successfully converted into a quote and an order). In accordance with such an example use case, as successful transactional opportunities get converted into quotes, the unit product price is commonly updated with pricing information from pricing guidance. While this may occur towards the end of a lifecycle for the given transactional opportunity, such actions can be a useful predictor for future transactional opportunities. Additionally, such a capability, enabled via use of one or more machine learning techniques, to develop pricing insights from previous transactional opportunities and applying such insights to new transactional opportunities at the beginning stages of the opportunity lifecycle can, for example, reduce and/or eliminate pipeline discrepancies, as well as improve resource capture and/or utilization.

As further detailed herein, at least one embodiment includes implementing at least one machine learning-based prediction engine that provides resource-value estimation and/or insights (e.g., resource price estimations and/or insights) for one or more users and/or related automated systems for use in connection with one or more resource-related activities (e.g., a transactional opportunity, involving a corresponding resource, during an early lifecycle stage such as a pre-proposal stage). In such an embodiment, input features for training a regression-based machine learning model can include historical resource-related activity information including, e.g., the account and/or activity name, the type of activity, the owner and/or division of an enterprise associated with the activity, resources included in the activity, resource quantities involved in the activity, etc. By way merely of example, at least one target label for such a machine learning model can include the unit resource price and/or the total price of the resource-related activity.

shows example system architecture in an illustrative embodiment. By way of illustration,depicts a sales and marketing system-, which provides quote and/or order data to multi-dimensional machine learning-based prediction engine, which is trained using data from resource-related database, which includes historical quote-related and/or order-related data provided by quote and order management system-. Based at least in part on the input provided by sales and marketing system-, multi-dimensional machine learning-based prediction enginegenerates a prediction regarding at least one resource value associated with the given quote and/or order and a prediction regarding at least one value attributed to the entire quote and/or order. Such predictions are then provided back to the sales and marketing system-, which provides at least a portion of the predictions, along with original quote and/or order data, to quote and order management system-. Based at least in part on those provided inputs, quote and order management system-generates outputs and/or instructions for fulfillment system-and services system-to facilitate execution of the quote and/or order, in accordance with the noted predictions.

Accordingly, and as further detailed herein, one or more embodiments include implementing at least one resource-related database, and at least one multi-dimensional machine learning-based prediction engine.

With respect to the at least one resource-related database, one or more embodiments include building and/or maintaining such a data repository to contain historical resource-related value data (e.g., product price data) across at least one enterprise. Data engineering and/or data analysis techniques can be carried out in connection with at least a portion of relevant historical data to, for example, learn and/or understand one or more features and/or one or more data elements that can influence one or more target values (e.g., resource unit price, total price of the corresponding resource-related activity) such that only those data features and/or data elements are filtered for storage in the repository. Accordingly, in such an embodiment, the at least one resource-related database can store and/or contain data pertaining to relevant data features and/or data elements including, e.g., date information, user information, resource (e.g., product) information, resource quantity information, activity type, resource value type (e.g., currency), etc.shows example system architecture for a prediction engine in an illustrative embodiment. By way of illustration,depicts architecture of an example multi-dimensional machine learning-based prediction engine, which includes at least one multi-output neural network(e.g., one or more artificial neural network-based (ANN-based) multi-output regression models), which is trained using historical opportunity and order data. As also depicted in, new opportunity datacan be provided to and/or proceed by the multi-output neural network, which generates a prediction as to at least one product unit price associated with the new opportunity, and a prediction as to the total price of the new opportunity.

Accordingly, with respect to the multi-dimensional machine learning-based prediction engine, one or more embodiments include utilizing such a machine learning-based engine to predict and/or estimate one or more resource-related values associated with a given resource-related activity, and/or to predict and/or estimate the total value of the given resource-related activity.

shows an example architecture of a neural network in an illustrative embodiment. By way of illustration,depicts a multi-output neural network, a type of deep neural network model that has two parallel branches of a network for two types of outputs. Accordingly, such an embodiment includes taking the same set of input variables(e.g., opportunity type, brand description, product group, quantity, currency, etc.) as a single input layer, and building and/or training a dense, multi-layer neural network model which acts as two sophisticated classifiers for multi-output predictions. The example multi-output neural networkdepicted inincludes one input layer, hidden layers-and-, and output layers-and-.

As a multi-output neural network model, the multi-output neural networkcreates two separate branches of the network (e.g., two hidden layers,-and-, as well as two output layers,-and-) that connect to the same input layer. In at least one embodiment, such an input layercan include a number of neurons that matches the number of input and/or independent variables. Also, in such an embodiment, the two hidden layers,-and-, can include neurons on each layer in amounts/numbers that depend upon the number of neurons in the input layer. Further, in such an embodiment, the two output layers,-and-(e.g., one output layer for each branch of the model), can each contain a single neuron related to the type of output generated.

Additionally, in one or more embodiments, the at least one resource-related value prediction engine leverages at least one supervised learning mechanism and trains the corresponding model with historical data containing actual resource values for each of one or more related users (i.e., related to one or more users involved in a given resource-related activity), as well as data pertaining to the type of resource, the type of activity, the resource quantities involved in the activity, resource value denomination type (e.g., currency), location data, etc. As noted above and further detailed herein, during the training of such a machine learning model, such data features are fed into the model as the independent variable(s), and the actual resource value unit and/or the total value of the resource-related activity are utilized as the dependent and/or target variable(s).

Also, in at least one embodiment, the at least one resource-related value prediction engine utilizes at least one deep neural network by building and/or configuring a dense, multi-layer neural network model which acts as a sophisticated regressor. Such a neural network model can include at least one input layer, one or more hidden layers (e.g., two hidden layers), and at least one output layer. In one or more embodiments, the input layer includes a number of neurons that matches the number of input and/or independent variables, and the hidden layer(s) include two hidden layers, and the number of neurons on each of the two hidden layers depends on the number of neurons in the input layer. Additionally, in such an embodiment, the output layer contains a single neuron, as this neural network is a regression model, meaning that the output is a continuous, numerical value representing a given resource-related value price and/or the total price of a given resource-related activity.

By way merely of illustration, consider an example neural network model which includes five neurons in a first hidden layer and three neurons in a second hidden layer. That said, it should be appreciated, as noted above, that the actual number(s) of neurons in the hidden layer(s) can depend on the total number of neurons in the input layer. By way of further example, in one or more embodiments, the number of neurons in the first hidden layer can be calculated based on an algorithm of matching the power of two to the number of input layer neurons. For example, if the number of input variables is 19, it falls in the range of 25. That means that the first hidden layer will have 25=32 neurons, and the second hidden layer will contain 24=16 neurons. If there were to be a third hidden layer, that hidden layer would include 23=8 neurons. Also, in at least one embodiment, the neurons in the hidden layer(s) and the output layer contain at least one activation function which helps determining whether a given neuron will fire or not. In such an example embodiment (and as depicted in example multi-output neural networkillustrated in), the rectified linear unit (ReLU) activation function can be used in both hidden layers, and, as the neural network model described in this example is configured as a multi-class regression model (also referred to herein as a regressor), the output layer neurons will contain a linear activation function or no activation functions.

As one or more embodiments include using a dense neural network model (e.g., such as depicted in example multi-output neural networkillustrated in), in such an embodiment, each neuron will connect with each other neuron. Further, each connection will have a weight factor, and the neurons will have a bias factor, wherein these weight and bias values can be set randomly by the neural network model (e.g., such values can be set as 1 or 0 for all values). Also, in such an embodiment, each neuron performs a linear calculation by combining the multiplication of each input variable with their corresponding weight factors, and then adding the bias value of the neuron. The formula for this calculation is illustrated as follows:

wherein ws1 represents the weighted sum of neuron1, x1, x2, etc. represent the input variables to the neural network model, w1, w2, etc. represent the weight values applied to the connections to neuron1, and b1 represents the bias value of neuron1. This weighted sum is then input to an activation function (e.g., ReLU) to compute the value of the activation function. Similarly, weighted sum and activation function values of all other neurons in the layer are calculated, and these values are fed to the neuron(s) of the next layer. In at least one embodiment, the same process is then repeated in the next layer of neurons until the values are fed to the neuron of the output layer, wherein the weighted sum is also calculated and compared to the actual target value. Depending upon the difference, a loss value and/or error value can be calculated.

Such a pass through of the neural network model is referred to as a forward propagation, which calculates the loss value and/or error value and drives a backpropagation through the neural network model to reduce and/or minimize the loss value and/or error value at each neuron of the network. Considering that the loss value and/or error value is generated by all neurons in the network, backpropagation goes through each layer, from back to front, and attempts to reduce and/or minimize the loss value and/or error value by using at least one gradient descent-based optimization mechanism. Considering that, in one or more embodiments, a multi-output neural network model is used as a regressor and classifier, such an embodiment can include using the loss function of “mean_squared_error” for the both regressor branches, and using the adaptive moment estimation (Adam) function as the optimization algorithm for both branches.

In at least one embodiment, the result of a backpropagation such as detailed above is to adjust the weight and bias values at each connection and neuron to reduce the loss value and/or error value. Additionally, in accordance with one or more embodiments, once all observations of the training data are passed through the neural network model, an epoch is completed. Another forward propagation is then initiated with the adjusted weight and bias values, which is considered as epoch2, and the same process of forward and backpropagation is repeated in one or more subsequent epochs. This process of iterating multiple epochs results in the reduction of loss values and/or error values to a small number (e.g., close to zero), at which point, the neural network model is considered to be sufficiently trained for prediction.

The implementation of at least portions of one or more embodiments can be achieved, for example, as depicted inthrough, by using Keras with a Tensorflow backend, Python language, as well as Pandas, Numpy and ScikitLearn libraries.

shows example pseudocode for implementing data preprocessing techniques in an illustrative embodiment. In this embodiment, example pseudocodeis executed by or under the control of at least one processing system and/or device. For example, the example pseudocodemay be viewed as comprising a portion of a software implementation of at least part of automated resource-related value prediction systemof theembodiment.

The example pseudocodeillustrates importing necessary libraries, reading a dataset of historical opportunity outcomes, and generating a Pandas data frame. The data frame contains columns including independent variables, as well as the dependent and/or target variable columns (e.g., resource value unit and/or total value of the resource-related activity). Additionally, in connection with example pseudocode, preprocessing of such data includes handling any null or missing values in the columns. In one or more embodiments, null or missing values in numerical columns can be replaced by the median value of that column. After handling null or missing values in the columns, such an embodiment can include performing initial data analysis by creating one or more univariate and/or bivariate plots of the columns, whereby the importance and/or influence of each column can be understood. Columns that have limited importance and/or influence (e.g., no importance and/or influence) on the actual prediction (i.e., the dependent and/or target variable) can be removed.

It is to be appreciated that this particular example pseudocode shows just one example implementation of data preprocessing techniques, and alternative implementations can be used in other embodiments.

shows example pseudocode for data filtering in an illustrative embodiment. In this embodiment, example pseudocodeis executed by or under the control of at least one processing system and/or device. For example, the example pseudocodemay be viewed as comprising a portion of a software implementation of at least part of automated resource-related value prediction systemof theembodiment.

The example pseudocodeillustrates filtering out certain data, from larger opportunity datasets, which will be used for data visualization and data engineering and/or feature engineering. Opportunity data often contains a lot of information that is not related and/or relevant to the predictions detailed herein in connection with one or more embodiments. Accordingly, such an embodiment includes filtering out only the features that are pertinent and the target labels (e.g., resource value unit and/or total value of the resource-related activity) from the opportunity data, and using such filtered data for data visualization and data engineering and/or feature engineering.

It is to be appreciated that this particular example pseudocode shows just one example implementation of data filtering techniques, and alternative implementations can be used in other embodiments.