Iterative Training of a Nodal Network with Data Influence Weights

1. A method of training a nodal network, the method comprising: evaluating performance of the nodal network with a plurality of training data sets, wherein the plurality of training data sets comprises a first group of training data sets and a second group of training data sets, wherein the training data sets in the first group comprise a first data item and wherein the training data sets in the second group of training data sets do not comprise the first data item, wherein evaluating the performance comprises determining, by the computer system, using a multiple variable regression, an effect on the performance of the nodal network, on validation data, from training with the first data item and without the first data item; based on the determination, assigning, by the computer system, a data-item-specific weight to the first data item, wherein the data-item-specific weight is a weight specifically assigned to the first data item based on the effect on the performance of the nodal network from training with the first data item; and training, by the computer system, the nodal network, wherein the training comprises: computing, by the computer system, for data items, including the first data item, in a set of training data items, a gradient estimate for a connection weight for a connection between two nodes of the nodal network; and after computing the gradient estimates, computing, by the computer system, an updated connection weight for the connection, wherein computing the updated connection weight comprises, for the first data item, computing, by the computer system, a product that is a multiplication of (i) gradient estimate for the first data item and (ii) the data-item-specific weight for the first data item.

2. The method of claim 1 , wherein the data-item-specific weight for the first data item is a value between 0 and 1.

3. The method of claim 1 , wherein: the plurality of training data sets comprises a third group of training data sets and a fourth group of training data sets, wherein the training data sets in the third group comprise a second data item and the training data sets in the fourth group do not comprise the second data item; the method further comprises the steps of: evaluating the performance of the nodal network by determining, by the computing system, using a multiple variable regression, an effect on the performance of the nodal network, on the validation data, from training with the second data item and without the second data item; based on the determination, assigning, by the computer system, a second data-item-specific weight to the second data item, wherein the second data-item-specific weight is a weight specifically assigned to the second data item based on the effect on the performance of the nodal network from training with the second data item; and computing the updated connection weight comprises, for the second data item, computing, by the computer system, a product that is a multiplication of (i) gradient estimate for the second data item and (ii) the second data-item-specific weight for the second data item.

4. The method of claim 1 , wherein assigning the data-item-specific weight comprises changing the data-item-specific weight based on the effect on the performance of the nodal network from training with the first data item.

5. The method of claim 4 , wherein changing the data-item specific weight comprises decreasing the data-item-specific weight upon a determination that the performance of the nodal network deteriorates from training with the first data item.

6. The method of claim 5 , wherein decreasing the data-item specific weight comprises decreasing the data-item-specific weight upon a determination that the performance of the nodal network deteriorates from overfitting the set of training data items.

7. The method of claim 1 , further comprising labeling, by a trained recognizer, unlabeled data to generate the validation data.

8. The method of claim 1 , wherein: the nodal network comprises an ensemble of ensemble network members; determining the effect on the performance of the nodal network from training with the first data item comprises determining, by the computer system, the effect on the performance on each ensemble network members from training with the first data item; and assigning the data-item-specific weight comprises assigning a data-item-specific weight for the first data item for each of the ensemble network members based on the effect on the performance of each of the ensemble network members from training with the first data item.

9. A method of training a nodal network, wherein the nodal network comprises a plurality of connections, wherein each of the plurality of connections is between two nodes of the nodal network, the method comprising: evaluating performance of the nodal network with a plurality of training data sets, wherein the plurality of training data sets comprises a first group of training data sets and a second group of training data sets, wherein the training data sets in the first group comprise a first data item and wherein the training data sets in the second group of training data sets do not comprise the first data item, wherein evaluating the performance comprises determining, by the computer system, using a multiple variable regression, an effect on the performance of the nodal network, on validation data, from training with the first data item and without the first data item; based on the determination, assigning, by the computer system, a data-item-specific weight to the first data item, wherein the data-item-specific weight is a weight specifically assigned to the first data item based on the effect on the performance of the nodal network from training with the first data item; and training, by the computer system, the nodal network, wherein the training comprises: computing, by the computer system, for data items, including the first data item, in a set of training data items and for each connection of the nodal network, a gradient estimate for a connection weight for the connection; and after computing the gradient estimates, computing, by the computer system, an updated connection weight for each connection, wherein computing the updated connection weight comprises, for the first data item, and for each connection of the nodal network, computing, by the computer system, a product that is a multiplication of (i) gradient estimate for the first data item and (ii) the data-item-specific weight for the first data item.

10. The method of claim 9 , wherein the data-item-specific weight for the first data item is a value between 0 and 1.

11. The method of claim 9 , wherein: the plurality of training data sets comprises a third group of training data sets and a fourth group of training data sets, wherein the training data sets in the third group comprise a second data item and the training data sets in the fourth group do not comprise the second data item; the method further comprises the steps of: evaluating the performance of the nodal network by determining, by the computing system, using a multiple variable regression, an effect on the performance of the nodal network, on the validation data, from training with the second data item and without the second data item; based on the determination, assigning, by the computer system, a second data-item-specific weight to the second data item, wherein the second data-item-specific weight is a weight specifically assigned to the second data item based on the effect on the performance of the nodal network from training with the second data item; and computing the updated connection weights for each connection comprises, for the second data, for each connection of the nodal network, computing, by the computer system, a product that is a multiplication of (i) gradient estimate for the second data item and (ii) the second data-item-specific weight for the second data item.

12. A computer system comprising: one or more processor cores; and a memory in communication with the one or more processor cores, wherein the memory stores software that, when executed by the one or more processor cores, cause the one or more processor cores to: evaluate performance of the nodal network with a plurality of training data sets, wherein the plurality of training data sets comprises a first group of training data sets and a second group of training data sets, wherein the training data sets in the first group comprise a first data item and wherein the training data sets in the second group of training data sets do not comprise the first data item, wherein evaluating the performance comprises determining, using a multiple variable regression, an effect on the performance of the nodal network from training with the first data item and without the first data item; based on the determination, assign a data-item-specific weight to the first data item, wherein the data-item-specific weight is a weight specifically assigned to the first data item based on the effect on the performance of the nodal network from training with the first data item; and train the nodal network, wherein the training comprises: computing, for data items, including the first data item, in a set of training data items, a gradient estimate for a connection weight for a connection between two nodes of the nodal network; and after computing the gradient estimates, computing an updated connection weight for the connection by, for the first data item, computing a product that is a multiplication of (i) gradient estimate for the first data item and (ii) the data-item-specific weight for the first data item.

13. The computer system of claim 12 , wherein the data-item-specific weight for the first data item is a value between 0 and 1.

14. The computer system of claim 12 , wherein: the plurality of training data sets comprises a third group of training data sets and a fourth group of training data sets, wherein the training data sets in the third group comprise a second data item and the training data sets in the fourth group do not comprise the second data item; and the memory stores further software that, when executed by the one or more processors, cause the one or more processors to: evaluate the performance of the nodal network by determining, using a multiple variable regression, an effect on the performance of the nodal network, on the validation data, from training with the second data item and without the second data item; based on the determination, assign a data-item-specific weight to the second data item, wherein the data-item-specific weight is a weight specifically assigned to the second data item based on the effect on the performance of the nodal network from training with the second data item; and after computing the gradient estimates, compute the updated connection weight for the connection by, for the second data item, computing a product that is a multiplication of (i) gradient estimate for the second data item and (ii) the data-item-specific weight for the second data item.

15. A computer system comprising: one or more processor cores; and a memory in communication with the one or more processor cores, wherein the memory stores software that, when executed by the one or more processor cores, cause the one or more processor cores to: evaluate performance of the nodal network with a plurality of training data sets, wherein the plurality of training data sets comprises a first group of training data sets and a second group of training data sets, wherein the training data sets in the first group comprise a first data item and wherein the training data sets in the second group of training data sets do not comprise the first data item, wherein evaluating the performance comprises determining, using a multiple variable regression, an effect on performance of the nodal network, on validation data, from training with the first data item and without the first data item; based on the determination, assign a data-item-specific weight to the first data item, wherein the data-item-specific weight is a weight specifically assigned to the first data item based on the effect on the performance of the nodal network from training with the first data item; train the nodal network, wherein the training comprises: computing, for data items, including the first data item, in a set of training data items and for each connection of the nodal network, a gradient estimate for a connection weight for the connection; and after computing the gradient estimates, computing an updated connection weight for each connection, wherein computing the updated connection weight comprises, for the first data item, and for each connection of the nodal network, computing a product that is a multiplication of (i) gradient estimate for the first data item and (ii) the data-item-specific weight for the first data item.

16. The computer system of claim 15 , wherein the data-item specific weight for the first data item is a value between 0 and 1.

17. The computer system of claim 15 , wherein: the plurality of training data sets comprises a third group of training data sets and a fourth group of training data sets, wherein the training data sets in the third group comprise a second data item and the training data sets in the fourth group do not comprise the second data item; the memory stores further software that, when executed by the one or more processors, cause the one or more processors to: evaluate the performance of the nodal network by determining, using a multiple variable regression, an effect on the performance of the nodal network, on the validation data, from training with the second data item and without the second data item; based on the determination, assign a data-item-specific weight to the second data item, wherein the data-item-specific weight is a weight specifically assigned to the second data item based on the effect on the performance of the nodal network from training with the second data item; and after computing the gradient estimates, compute the updated connection weight for each connection by, for the second data item, and for each connection of the nodal network, computing a product that is a multiplication of (i) gradient estimate for the second data item and (ii) the data-item-specific weight for the second data item.

18. The computer system of claim 15 , wherein the memory stores software that when executed by the one or more processor cores, causes the one or more processor cores to assign the data-item-specific weight by changing the data-item-specific weight based on the effect on the performance of the nodal network from training with the first data item.

19. The computer system of claim 18 , wherein the memory stores software that when executed by the one or more processor cores, causes the one or more processor cores to change the data-item specific weight by decreasing the data-item-specific weight upon a determination that the performance of the nodal network deteriorates from training with the first data item.

20. The computer system of claim 19 , wherein the memory stores software that when executed by the one or more processor cores, causes the one or more processor cores to decrease the data-item specific weight by decreasing the data-item-specific weight upon a determination that the performance of the nodal network deteriorates from overfitting the set of training data items.

21. The computer system of claim 15 , wherein the memory stores software that when executed by the one or more processor cores, causes the one or more processor cores to label, by a trained recognizer, unlabeled data to generate the validation data.

22. The computer system of claim 15 , wherein: the nodal network comprises an ensemble of ensemble network members; the memory stores software that when executed by the one or more processor cores, causes the one or more processor cores to: determine the effect on the performance on each ensemble network members from training with the first data item; and assign a data-item-specific weight for the first data item for each of the ensemble network members based on the effect on the performance of each of the ensemble network members from training with the first data item.