System and Method for Main Page Identification in Web Decoding

1. A method for communication analysis via a decoding processor, the method comprising the steps of: by the decoding processor, processing packets so as to identify data elements viewed by a target user during a web browsing session; by the decoding processor, for each identified data element, calculating a confidence level that the identified data element is on a main web page; by the decoding processor, determining identified data elements whose calculated confidence levels are below a threshold confidence level; by the decoding processor, determining if a percentage of identified data elements whose calculated confidence levels are below the threshold confidence level exceeds a level; and when it is determined that the percentage exceeds the level: receiving feedback as to whether the identified data elements are on the main web page or are displayed on a different web page; and determining which of the identified data elements to display based on the received feedback.

2. The method of claim 1 , further comprising, by the decoding processor, intercepting the packets exchanged between a user computer and one or more servers over a network during the web browsing session associated with the target user.

3. The method of claim 1 , further comprising, by the decoding processor, displaying the determined identified data elements to the operator on a display of an operator terminal in communication with the decoding processor, each of the determined identified data elements displayed as a separate web page with a request for feedback as to whether the displayed web page is the main web page.

4. The method of claim 3 , wherein displaying the determined identified data elements comprises displaying at least some of the data elements without a request for feedback and displaying at least some of the data elements with a request for feedback.

5. The method of claim 4 , wherein displaying determined identified data elements comprises displaying data elements identified as on the main page without a request for feedback and displaying data elements identified as possibly being on the main page with a request for feedback.

6. The method of claim 4 , wherein displaying determined identified data elements comprises displaying some of the identified data elements and not displaying at least one of the data elements identified as not being on the main web page.

7. The method of claim 6 , wherein data elements identified as not being on the main web page comprise data elements identified as being downloaded by the target user for display on its own, not embedded in a different web page.

8. The method of claim 1 , wherein determining which ones of the identified data elements to display comprises determining using an artificial intelligence engine.

9. The method of claim 1 , wherein the received feedback used in the determination comprises indications of embedded data elements.

10. The method of claim 9 , wherein determining which ones of the identified data elements to display comprises determining for a plurality of data elements having equivalent values of at least one parameter, for which at least some of the elements were indicated as being on the main web page and at least some were indicated as being an embedded data element, one or more parameters to be used in differentiating between the main web page and embedded data elements.

11. The method of claim 10 , wherein the plurality of data elements having equivalent values comprise data elements from a same web address.

12. The method of claim 1 , wherein determining which ones of the identified data elements display comprises determining for identified data elements belonging to the same session as at least one of the elements for which the received feedback was used in the determination.

13. The method of claim 1 , further comprising: when it is determined that the percentage does not exceed the level: by the decoding processor, displaying the determined identified data elements to the operator on the display of an operator terminal in communication with the decoding processor, each of the determined identified data elements displayed as a separate web page without a request for feedback as to whether the displayed web page is a main web page.

14. A decoding processor with instructions stored thereon that when executed by the decoding processor cause the decoding processor to: process packets so as to identify data elements viewed by a target user during a web browsing session; for each identified data element, calculate a confidence level that the identified data element is on a main web page; determine identified data elements whose calculated confidence levels are below a threshold confidence level; determine if a percentage of identified data elements whose calculated confidence levels are below the threshold confidence level exceeds a level; and when it is determined that the percentage exceeds the level: receive feedback as to whether the identified data elements are on the main web page or are displayed on a different web page; and determine which of the identified data elements to display based on the received feedback.

15. The decoding processor of claim 14 , wherein the processor further intercepts the packets exchanged between a user computer and one or more servers over a network during the web browsing session associated with the target user.

16. The decoding processor of claim 14 , further comprising, by the decoding processor, displaying the determined identified data elements to the operator on a display of an operator terminal in communication with the decoding processor, each of the determined identified data elements displayed as a separate web page with a request for feedback as to whether the displayed web page is the main web page.

17. The decoding processor of claim 16 , wherein displaying the determined identified data elements comprises displaying at least some of the data elements without a request for feedback and displaying at least some of the data elements with a request for feedback.

18. The decoding processor of claim 17 , wherein displaying determined identified data elements comprises displaying data elements identified as on the main page without a request for feedback and displaying data elements identified as possibly being on the main page with a request for feedback.

19. The decoding processor of claim 14 , wherein the identified data elements comprise data elements with a parameter value that is indicative of an embedded element, wherein the parameter value is present in more than a threshold number of identified main pages.

20. The decoding processor of claim 1 , further comprising: when it is determined that the percentage does not exceed the level: by the decoding processor, displaying the determined identified data elements to the operator on the display of an operator terminal in communication with the decoding processor, each of the determined identified data elements displayed as a separate web page without a request for feedback as to whether the displayed web page is a main web page.