Method and System for Predicting Relevant Network Relationships

1. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, the computer-program product including system instructions operable to cause a computing device to: obtain a first data set associated with a plurality of nodes to generate one or more sets of networks; train a first model on the first data set using a first graph to predict relevant links between the plurality of nodes by executing operations comprising: determine one or more features for one or more links between the plurality of nodes; determine a target variable indicator for the one or more links between the plurality of nodes using the first graph by executing operations comprising: determine a set of subgraphs from the first graph; determine whether each of the one or more links between each node of the plurality of nodes connect within a single subgraph of the set of subgraphs from the first graph; based on the determination of whether each of the one or more links between each node of the plurality of nodes connect within the single subgraph of the set of subgraphs from the first graph, label the one or more links as intra-community links in the single subgraph of the set of subgraphs from the first graph; determine whether each of the one or more links between each node of the plurality of nodes connect between at least two subgraphs of the set of subgraphs from the first graph; based on the determination of whether each of the one or more links between each node of the plurality of nodes connect between the at least two subgraphs of the set of subgraphs from the first graph, label the one or more links as inter-community links in the at least two subgraphs of the set of subgraphs from the first graph; output the labeled one or more links as the intra-community links in the single subgraph of the set of subgraphs from the first graph; and output the labeled one or more links as the inter-community links in the at least two subgraphs of the set of subgraphs from the first graph; and based on the determination of the one or more features and the determination of the target variable indicator for the one or more links between the plurality of nodes using the first graph, train the first model to predict the relevant links of the one or more links between the plurality of nodes, wherein the relevant links comprise the intra-community links; obtain the first data set or a second data set associated with the plurality of nodes; for each node of the plurality of nodes from the first data set or the second data set, execute operations comprising: determine the one or more features for the one or more links between the plurality of nodes; based on the determination of the one or more features for the one or more links between the plurality of nodes, apply the trained first model to the one or more links between the plurality of nodes; based on the application of the trained first model to the one or more links between the plurality of nodes, output the relevant links and non-relevant links of the one or more links between the plurality of nodes and output a trained model variable, wherein the non-relevant links comprise the inter-community links; based on the output of the relevant links and the non-relevant links of the one or more links between the plurality of nodes, remove the non-relevant links of the one or more links between the plurality of nodes; based on the output of the trained model variable, optimize the application of the trained first model to the one or more links between the plurality of nodes by automatically computing a first threshold for the trained model variable for one or more factors; and based on the removal of the non-relevant links of the one or more links between the plurality of nodes, connect each node of the plurality of nodes with the relevant links to generate one or more first sets of networks; and output the one or more first sets of generated networks in a first graphical user interface, as a first input to an automated analytical process, or as a first input to an investigative system.

2. The computer-program product of claim 1, wherein instructions operable to cause the computing device to: obtain the first data set associated with the plurality of nodes to generate the one or more sets of networks; train an updated model on the first data set using the first graph to predict the relevant links between the plurality of nodes by executing operations comprising: determine the one or more features for the one or more links between the plurality of nodes; determine the target variable indicator for the one or more links between the plurality of nodes using the first graph by executing operations comprising: determine the set of subgraphs from the first graph; determine whether each of the one or more links between each node of the plurality of nodes connect within the single subgraph of the set of subgraphs from the first graph; based on the determination of whether each of the one or more links between each node of the plurality of nodes connect within the single subgraph of the set of subgraphs from the first graph, label the one or more links as intra-community links in the single subgraph of the set of subgraphs from the first graph; determine whether each of the one or more links between each node of the plurality of nodes connect between at least two subgraphs of the set of subgraphs from the first graph; based on the determination of whether each of the one or more links between each node of the plurality of nodes connect between the at least two subgraphs of the set of subgraphs from the first graph, label the one or more links as inter-community links in the at least two subgraphs of the set of subgraphs from the first graph; output the labeled one or more links as the intra-community links in the single subgraph of the set of subgraphs from the first graph; and output the labeled one or more links as the inter-community links in the at least two subgraphs of the set of subgraphs from the first graph; and based on the determination of the one or more features and the determination of the target variable indicator for the one or more links between the plurality of nodes using the first graph, train the updated model to predict the relevant links of the one or more links between the plurality of nodes, wherein the relevant links comprise the intra-community links; obtain the first data set or the second data set associated with the plurality of nodes; based on the first data set or the second data set associated with the plurality of nodes, determine the one or more features for the one or more links between the plurality of nodes; based on the determination of the one or more features for the one or more links between the plurality of nodes, apply the trained updated model to the one or more links between the plurality of nodes; based on the application of the trained updated model to the one or more links between the plurality of nodes, output the relevant links and the non-relevant links of the one or more links between the plurality of nodes and output the trained model variable, wherein the non-relevant links comprise the inter-community links; based on the output of the relevant links and the non-relevant links of the one or more links between the plurality of nodes, remove the non-relevant links of the one or more links between the plurality of nodes; based on the output of the trained model variable, optimize the application of the trained first model to the one or more links between the plurality of nodes by automatically computing the first threshold for the trained model variable for one or more factors; based on the removal of the non-relevant links of the one or more links between the plurality of nodes, connect each node of the plurality of nodes with the relevant links to generate one or more second sets of networks; and output the one or more second sets of generated networks in a second graphical user interface, as a second input to the automated analytical process, or as a second input to the investigative system.

3. The computer-program product of claim 1, wherein the one or more features for the one or more links between the plurality of nodes comprise: for each of the one or more links between two nodes of the plurality of nodes: one or more link attributes for the one or more links between each pair of connected nodes of the plurality of nodes, one or more node attributes of each pair of the connected nodes at a first end or a second end of the one or more links, one or more node attribute computations or one or more node attribute thresholds applied to each pair of the connected nodes at the first end or the second end of the one or more links, one or more link attribute computations or one or more link attribute thresholds applied to one or more attributes of the one or more links for each pair of the connected nodes, and a first network for the one or more links between each pair of the connected nodes that connects a first node from each pair of the connected nodes to a set of nodes from each pair of the connected nodes.

4. The computer-program product of claim 3, wherein the one or more link attributes comprise one or more roles.

5. The computer-program product of claim 3, wherein the one or more node attributes comprise one or more node types, one or more date of births, or one or more values.

6. The computer-program product of claim 3, wherein the one or more node attribute computations comprise one or more sums of one or more related applications, one or more counts of one or more nodes that are connected to a selected node in a relationship, or the one or more counts of the one or more nodes for the one or more node attributes that are connected to the selected node in the relationship.

7. The computer-program product of claim 1, wherein the plurality of nodes comprises data representing at least one person, at least one location, at least one telephone number, at least one email address, at least one business, at least one application, at least one account, at least one vehicle, at least one IP address, at least one organization, at least one agent, at least one supplier, or at least one event.

8. The computer-program product of claim 1, wherein the first model comprises a decision tree, a random forest, a neural network, or another type of predictive model.

9. The computer-program product of claim 1, wherein the application of the trained first model to the one or more links between the plurality of nodes is further optimized based on the output of the trained model variable by: applying a second threshold for the trained model variable to each of the one or more links or to the one or more links for the one or more factors including mean, standard deviation, or network size, and applying a third threshold for the trained model variable from one or more user preferences.

10. The computer-program product of claim 1, wherein the application of the trained first model to the one or more links between the plurality of nodes comprises running the trained first model on each row of the first data set or the second data set.

11. A computer-implemented method comprising: obtaining a first data set associated with a plurality of nodes to generate one or more sets of networks; training a first model on the first data set using a first graph to predict relevant links between the plurality of nodes by: determining one or more features for one or more links between the plurality of nodes; determining a target variable indicator for the one or more links between the plurality of nodes using the first graph by: determining a set of subgraphs from the first graph; determining whether each of the one or more links between each node of the plurality of nodes connect within a single subgraph of the set of subgraphs from the first graph; based on the determination of whether each of the one or more links between each node of the plurality of nodes connect within the single subgraph of the set of subgraphs from the first graph, labeling the one or more links as intra-community links in the single subgraph of the set of subgraphs from the first graph; determining whether each of the one or more links between each node of the plurality of nodes connect between at least two subgraphs of the set of subgraphs from the first graph; based on the determination of whether each of the one or more links between each node of the plurality of nodes connect between the at least two subgraphs of the set of subgraphs from the first graph, labeling the one or more links as inter-community links in the at least two subgraphs of the set of subgraphs from the first graph; outputting the labeled one or more links as the intra-community links in the single subgraph of the set of subgraphs from the first graph; and outputting the labeled one or more links as the inter-community links in the at least two subgraphs of the set of subgraphs from the first graph; and based on the determination of the one or more features and the determination of the target variable indicator for the one or more links between the plurality of nodes using the first graph, training the first model to predict the relevant links of the one or more links between the plurality of nodes, wherein the relevant links comprise the intra-community links; obtaining the first data set or a second data set associated with the plurality of nodes; for each node of the plurality of nodes from the first data set or the second data set: determining the one or more features for the one or more links between the plurality of nodes; based on the determination of the one or more features for the one or more links between the plurality of nodes, applying the trained first model to the one or more links between the plurality of nodes; based on the application of the trained first model to the one or more links between the plurality of nodes, outputting the relevant links and non-relevant links of the one or more links between the plurality of nodes and outputting a trained model variable, wherein the non-relevant links comprise the inter-community links; based on the outputting of the relevant links and the non-relevant links of the one or more links between the plurality of nodes, removing the non-relevant links of the one or more links between the plurality of nodes; based on the outputting of the trained model variable, optimizing the application of the trained first model to the one or more links between the plurality of nodes by automatically computing a first threshold for the trained model variable for one or more factors; and based on the removal of the non-relevant links of the one or more links between the plurality of nodes, connecting each node of the plurality of nodes for the relevant links to generate one or more first sets of networks; and outputting the one or more first sets of generated networks in a first graphical user interface, as a first input to an automated analytical process, or as a first input to an investigative system.

12. The computer-implemented method of claim 11, wherein the computer-implemented method comprises: obtaining the first data set associated with the plurality of nodes to generate the one or more sets of networks; training an updated model on the first data set using the first graph to predict the relevant links between the plurality of nodes by: determining the one or more features for the one or more links between the plurality of nodes; determining the target variable indicator for the one or more links between the plurality of nodes using the first graph by: determining the set of subgraphs from the first graph; determining whether each of the one or more links between each node of the plurality of nodes connect within the single subgraph of the set of subgraphs from the first graph; based on the determination of whether each of the one or more links between each node of the plurality of nodes connect within the single subgraph of the set of subgraphs from the first graph, labeling the one or more links as intra-community links in the single subgraph of the set of subgraphs from the first graph; determining whether each of the one or more links between each node of the plurality of nodes connect between at least two subgraphs of the set of subgraphs from the first graph; based on the determination of whether each of the one or more links between each node of the plurality of nodes connect between the at least two subgraphs of the set of subgraphs from the first graph, labeling the one or more links as inter-community links in the at least two subgraphs of the set of subgraphs from the first graph; outputting the labeled one or more links as the intra-community links in the single subgraph of the set of subgraphs from the first graph; and outputting the labeled one or more links as the inter-community links in the at least two subgraphs of the set of subgraphs from the first graph; and based on the determination of the one or more features and the determination of the target variable indicator for the one or more links between the plurality of nodes using the first graph, training the updated model to predict the relevant links of the one or more links between the plurality of nodes, wherein the relevant links comprise the intra-community links; obtaining the first data set or the second data set associated with the plurality of nodes; based on the first data set or the second data set associated with the plurality of nodes, determining the one or more features for the one or more links between the plurality of nodes; based on the determination of the one or more features for the one or more links between the plurality of nodes, applying the trained updated model to the one or more links between the plurality of nodes; based on the application of the trained updated model to the one or more links between the plurality of nodes, outputting the relevant links and the non-relevant links of the one or more links between the plurality of nodes and outputting the trained model variable, wherein the non-relevant links comprise the inter-community links; based on the outputting of the relevant links and the non-relevant links of the one or more links between the plurality of nodes, removing the non-relevant links of the one or more links between the plurality of nodes; based on the outputting of the trained model variable, optimizing the application of the trained first model to the one or more links between the plurality of nodes by automatically computing the first threshold for the trained model variable for one or more factors; based on the removal of the non-relevant links of the one or more links between the plurality of nodes, connecting each node of the plurality of nodes for the relevant links to generate one or more second sets of networks; and outputting the one or more second sets of generated networks in a second graphical user interface, as a second input to the automated analytical process, or as a second input to the investigative system.

13. The computer-implemented method of claim 11, wherein the one or more features for the one or more links between the plurality of nodes comprise: for each of the one or more links between two nodes of the plurality of nodes: one or more link attributes for the one or more links between each pair of connected nodes of the plurality of nodes, one or more node attributes of each pair of the connected nodes at a first end or a second end of the one or more links, one or more node attribute computations or one or more node attribute thresholds applied to each pair of the connected nodes at the first end or the second end of the one or more links, one or more link attribute computations or one or more link attribute thresholds applied to one or more attributes of the one or more links for each pair of the connected nodes, and a first network for the one or more links between each pair of the connected nodes that connects a first node from each pair of the connected nodes to a set of nodes from each pair of the connected nodes.

14. The computer-implemented method of claim 13, wherein the one or more link attributes comprise one or more roles.

15. The computer-implemented method of claim 13, wherein the one or more node attributes comprise one or more node types, one or more date of births, or one or more values.

16. The computer-implemented method of claim 13, wherein the one or more node attribute computations comprise one or more sums of one or more related applications, one or more counts of one or more nodes that are connected to a selected node in a relationship, or the one or more counts of the one or more nodes for the one or more node attributes that are connected to the selected node in the relationship.

17. The computer-implemented method of claim 11, wherein the plurality of nodes comprises data representing at least one person, at least one location, at least one telephone number, at least one email address, at least one business, at least one application, at least one account, at least one vehicle, at least one IP address, at least one organization, at least one agent, at least one supplier, or at least one event.

18. The computer-implemented method of claim 11, wherein the first model comprises a decision tree, a random forest, a neural network, or another type of predictive model.

19. The computer-implemented method of claim 11, wherein the application of the trained first model to the one or more links between the plurality of nodes is further optimized based on the outputting of the trained model variable by: applying a second threshold for the trained model variable to each of the one or more links or to the one or more links for the one or more factors including mean, standard deviation, or network size, and applying a third threshold for the trained model variable from one or more user preferences.

20. A computing device comprising processor and memory, the memory containing instructions executable by the processor wherein the computing device is configured to: obtain a first data set associated with a plurality of nodes to generate one or more sets of networks; train a first model on the first data set using a first graph to predict relevant links between the plurality of nodes by executing operations comprising: determine one or more features for one or more links between the plurality of nodes; determine a target variable indicator for the one or more links between the plurality of nodes using the first graph by executing operations comprising: determine a set of subgraphs from the first graph; determine whether each of the one or more links between each node of the plurality of nodes connect within a single subgraph of the set of subgraphs from the first graph; based on the determination of whether each of the one or more links between each node of the plurality of nodes connect within the single subgraph of the set of subgraphs from the first graph, label the one or more links as intra-community links in the single subgraph of the set of subgraphs from the first graph; determine whether each of the one or more links between each node of the plurality of nodes connect between at least two subgraphs of the set of subgraphs from the first graph; based on the determination of whether each of the one or more links between each node of the plurality of nodes connect between the at least two subgraphs of the set of subgraphs from the first graph, label the one or more links as inter-community links in the at least two subgraphs of the set of subgraphs from the first graph; output the labeled one or more links as the intra-community links in the single subgraph of the set of subgraphs from the first graph; and output the labeled one or more links as the inter-community links in the at least two subgraphs of the set of subgraphs from the first graph; and based on the determination of the one or more features and the determination of the target variable indicator for the one or more links between the plurality of nodes using the first graph, train the first model to predict the relevant links of the one or more links between the plurality of nodes, wherein the relevant links comprise the intra-community links; obtain the first data set or a second data set associated with the plurality of nodes; for each node of the plurality of nodes from the first data set or the second data set, execute operations comprising: determine the one or more features for the one or more links between the plurality of nodes; based on the determination of the one or more features for the one or more links between the plurality of nodes, apply the trained first model to the one or more links between the plurality of nodes; based on the application of the trained first model to the one or more links between the plurality of nodes, output the relevant links and non-relevant links of the one or more links between the plurality of nodes and output a trained model variable, wherein the non-relevant links comprise the inter-community links; based on the output of the relevant links and the non-relevant links of the one or more links between the plurality of nodes, remove the non-relevant links of the one or more links between the plurality of nodes; based on the output of the trained model variable, optimize the application of the trained first model to the one or more links between the plurality of nodes by automatically computing a first threshold for the trained model variable for one or more factors; and based on the removal of the non-relevant links of the one or more links between the plurality of nodes, connect each node of the plurality of nodes with the relevant links to generate one or more first sets of networks; and output the one or more first sets of generated networks in a first graphical user interface, as a first input to an automated analytical process, or as a first input to an investigative system.

21. The computing device of claim 20, wherein the instructions operable to cause the computing device to: obtain the first data set associated with the plurality of nodes to generate the one or more sets of networks; train an updated model on the first data set using the first graph to predict the relevant links between the plurality of nodes by executing operations comprising: determine the one or more features for the one or more links between the plurality of nodes; determine the target variable indicator for the one or more links between the plurality of nodes using the first graph by executing operations comprising: determine the set of subgraphs from the first graph; determine whether each of the one or more links between each node of the plurality of nodes connect within the single subgraph of the set of subgraphs from the first graph; based on the determination of whether each of the one or more links between each node of the plurality of nodes connect within the single subgraph of the set of subgraphs from the first graph, label the one or more links as intra-community links in the single subgraph of the set of subgraphs from the first graph; determine whether each of the one or more links between each node of the plurality of nodes connect between at least two subgraphs of the set of subgraphs from the first graph; based on the determination of whether each of the one or more links between each node of the plurality of nodes connect between the at least two subgraphs of the set of subgraphs from the first graph, label the one or more links as inter-community links in the at least two subgraphs of the set of subgraphs from the first graph; output the labeled one or more links as the intra-community links in the single subgraph of the set of subgraphs from the first graph; and output the labeled one or more links as the inter-community links in the at least two subgraphs of the set of subgraphs from the first graph; and based on the determination of the one or more features and the determination of the target variable indicator for the one or more links between the plurality of nodes using the first graph, train the updated model to predict the relevant links of the one or more links between the plurality of nodes, wherein the relevant links comprise the intra-community links; obtain the first data set or the second data set associated with the plurality of nodes; based on the first data set or the second data set associated with the plurality of nodes, determine the one or more features for the one or more links between the plurality of nodes; based on the determination of the one or more features for the one or more links between the plurality of nodes, apply the trained updated model to the one or more links between the plurality of nodes; based on the application of the trained updated model to the one or more links between the plurality of nodes, output the relevant links and the non-relevant links of the one or more links between the plurality of nodes and output the trained model variable, wherein the non-relevant links comprise the inter-community links; based on the output of the relevant links and the non-relevant links of the one or more links between the plurality of nodes, remove the non-relevant links of the one or more links between the plurality of nodes; based on the output of the trained model variable, optimize the application of the trained first model to the one or more links between the plurality of nodes by automatically computing the first threshold for the trained model variable for one or more factors; based on the removal of the non-relevant links of the one or more links between the plurality of nodes, connect each node of the plurality of nodes with the relevant links to generate one or more second sets of networks; and output the one or more second sets of generated networks in a second graphical user interface, as a second input to the automated analytical process, or as a second input to the investigative system.

22. The computing device of claim 20, wherein the one or more features for the one or more links between the plurality of nodes comprise: for each of the one or more links between two nodes of the plurality of nodes: one or more link attributes for the one or more links between each pair of connected nodes of the plurality of nodes, one or more node attributes of each pair of the connected nodes at a first end or a second end of the one or more links, one or more node attribute computations or one or more node attribute thresholds applied to each pair of the connected nodes at the first end or the second end of the one or more links, one or more link attribute computations or one or more link attribute thresholds applied to one or more attributes of the one or more links for each pair of the connected nodes, and a first network for the one or more links between each pair of the connected nodes that connects a first node from each pair of the connected nodes to a set of nodes from each pair of the connected nodes.

23. The computing device of claim 22, wherein the one or more link attributes comprise one or more roles.

24. The computing device of claim 22, wherein the one or more node attributes comprise one or more node types, one or more date of births, or one or more values.

25. The computing device of claim 22, wherein the one or more node attribute computations comprise one or more sums of one or more related applications, one or more counts of one or more nodes that are connected to a selected node in a relationship, or the one or more counts of the one or more nodes for the one or more node attributes that are connected to the selected node in the relationship.

26. The computing device of claim 20, wherein the plurality of nodes comprises data representing at least one person, at least one location, at least one telephone number, at least one email address, at least one business, at least one application, at least one account, at least one vehicle, at least one IP address, at least one organization, at least one agent, at least one supplier, or at least one event.

27. The computing device of claim 20, wherein the first model comprises a decision tree, a random forest, a neural network, or another type of predictive model.

28. The computing device of claim 20, wherein the application of the trained first model to the one or more links between the plurality of nodes is further optimized based on the output of the trained model variable by: applying a second threshold for the trained model variable to each of the one or more links or to the one or more links for the one or more factors including mean, standard deviation, or network size, and applying a third threshold for the trained model variable from one or more user preferences.