Testing Web Applications Using Clusters

1. A system, comprising: a memory; and a processor coupled to the memory configured with code to: receive a list of web page clusters, each of the web page clusters representing a grouping of a plurality of web pages of a web application to be tested, where each web page cluster includes an associated input maximal distance score generated by calculating the similarity of each of one or more web page requests that resulted in a web page associated with a given web page cluster of the web page clusters, the maximal distance score is lower than other similarity scores in a cluster of the list; receive an intercepted input to the web application, where the input is configured to cause the web application to provide a responsive web page as an output for the input; obtain a similarity score between the input and one or more web page requests; cause the input to be sent to the web application as a request input where the similarity score between the input and any one of the one or more web page requests is less than the maximal distance score of the given web cluster; receive the output for the input from the web application; detect testable elements in the input and output; generate a combined fingerprint for the input and output based on the detected testable elements from the input and output by counting the number of occurrences of the detected testable elements in the input and output, wherein fingerprints generated for input elements of the input and fingerprints generated for output elements of the output are concatenated; add the output to one cluster of the list of web page clusters based on similarity between the combined fingerprint and the one cluster; and test a single web page from each cluster of the list of web page clusters.

2. The system of claim 1 , wherein the input comprises a hypertext transfer protocol (HTTP) request and the output comprises a hypertext transfer protocol (HTTP) response.

3. The system of claim 1 , wherein the test of the single web page from each cluster includes the processor being further configured to detect a security vulnerability based on the intercepted input; and modify the web application to prevent the detected security vulnerability in each web page in each web page cluster.

4. The system of claim 1 , wherein the processor is configured to calculate a similarity score between a fingerprint for a respective web page from the plurality of web pages and a respective cluster from the list of clusters and add the respective web page to the respective cluster in response to detecting that the similarity score exceeds a similarity threshold.

5. The system of claim 1 , wherein the processor is configured to calculate a similarity score between the fingerprint and each cluster in the list of clusters based on a calculated hamming distance.

6. A computer-implemented method, carried out by one or more processors executing with code, comprising the steps of: receiving a list of web page clusters, each of the web page clusters representing a grouping of a plurality of web pages of a web application to be tested, where each web page cluster includes an associated input maximal distance score generated by calculating the similarity of each of one or more web page requests that resulted in a web page associated with a given web page cluster of the web page clusters, the maximal distance score is lower than other similarity scores in a cluster of the list; receiving, via a processor, an intercepted hypertext transfer protocol (HTTP) request to the web application where the request is configured to cause the web application to provide a hypertext transfer protocol (HTTP) response from the web application; obtaining a similarity score between the HTTP request and one or more web page requests; causing the intercepted HTTP request to be sent to the web application as a request input where the similarity score between the HTTP request and any one of the one or more web page requests is less than the maximal distance score of the given web cluster; receiving the HTTP response for the HTTP request from the web application; detecting, via a processor, testable elements in the received request and response; generating, via a processor, a combined fingerprint for each web page based on a first fingerprint generated from the detected testable elements from the intercepted request and a second fingerprint generated from the detected testable elements of the response by counting the number of occurrences of the detected testable elements in the HTTP request and response, wherein fingerprints generated for input elements of the HTTP request and fingerprints generated for output elements of the HTTP response are concatenated; adding, via a processor, the HTTP response to one cluster of the list of web page clusters based on similarity between the combined fingerprint and the one cluster; and testing, via a processor, a single web page from each cluster of the list of web page clusters.

7. The computer-implemented method of claim 6 , further comprising the steps of: calculating, via the processor, a similarity score between the combined fingerprint for a web page from the plurality of web pages and a cluster from the list of clusters; and adding the web page to the cluster in response to detecting that the similarity score exceeds a similarity threshold.

8. The computer-implemented method of claim 6 , further comprising the step of calculating, via a processor, a similarity score between the fingerprint and each cluster in the list of clusters based on a calculated hamming distance.

9. The computer-implemented method of claim 6 , wherein the step of generating the combined fingerprint for each web page comprises the steps of: generating a first fingerprint representing each element in a GET request and a second fingerprint for each element of a document object model returned as a response to the GET request; and combining the first and second fingerprints for the elements to generate the combined fingerprint for each web page.

10. The computer-implemented method of claim 6 , further comprising steps of: sending the HTTP request to the web application if the HTTP request would not result in a web page that belongs to any cluster based on maximal distances calculated for the list of clusters, and receiving an output from the web application in response to the HTTP request.

11. A computer program product for testing web applications, the computer program product comprising a computer-readable storage medium having program code embodied therewith, wherein the computer readable storage medium is not a transitory signal per se and the program code is executable by one or more processors to: receive a list of web page clusters, each of the web page clusters representing a grouping of a plurality of web pages of a web application to be tested, where each web page cluster includes an associated input maximal distance score generated by calculating the similarity of each of one or more web page requests that resulted in a web page associated with a given web page cluster of the web page clusters, the maximal distance score is lower than other similarity scores in a cluster of the list; receive an intercepted input to the web application, where the input is configured to cause the web application to provide a responsive web page as an output for the input; obtain a similarity score between the input and one or more web page requests; cause the input to be sent to the web application as a request input where the similarity score between the input and any one of the one or more web page requests is less than the maximal distance score of the given web cluster; receive the output for the input from the web application; detect testable elements in the input and output; generate a combined fingerprint for each web page based on a first fingerprint of the detected testable elements from the intercepted input and the second fingerprint of the detected testable element from the output by counting the number of occurrences of the detected testable elements in the input and output, wherein fingerprints generated for input elements of the input and fingerprints generated for output elements of the output are concatenated; add the output to one cluster of the list of web page clusters comprising one or more similar web pages based on similarity between the combined fingerprint and the one cluster; and test a single web page from each cluster of the list of web page clusters.

12. The computer program product of claim 11 , further comprising program code executable by a processor to calculate a similarity score between a fingerprint for a web page from the plurality of web pages and a cluster from the list of clusters and to add the web page to the cluster in response to detecting that the similarity score exceeds a similarity threshold.

13. The computer program product of claim 11 , further comprising program code executable by a processor to calculate a similarity score between the fingerprint for each web page and each cluster in the list of clusters based on a calculated hamming distance.

14. The computer program product of claim 11 , further comprising program code, executable by a processor to: generate a maximal distance between requests for each cluster in the list of clusters; receive a request to be sent to the web application; detect that the request would not result in a web page that belongs to any cluster based on the maximal distances for the clusters; and send the request in response to detection that the request would not result in the web page that belongs to any cluster.