A code coverage-based taint perimeter detection system and method for testing software code by determining code coverage and detecting new coverage of the code. Embodiments of the system and method perform tainted data flow analysis on execution traces of the code to determine tainted branch targets. The tainted branch targets may be filtered to remove any tainted branch targets that have already been covered. New coverage can be determined by monitoring the filtered tainted branch targets, which in some embodiments involves the use of software breakpoints that are automatically placed at the locations in the tainted branch targets at runtime. Embodiments of the system and method use an iterative process to ensure that only tainted branch targets that have not already been covered or tested are examined.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method implemented by at least one computing device, the method comprising: generating multiple execution traces of software code using a set of multiple different inputs to the software code; determining tainted branch targets by performing data flow analysis on the multiple execution traces of the software code, the tainted branch targets being associated with tainted conditional branches in the software code; filtering the tainted branch targets to identify a taint perimeter of the software code, the taint perimeter comprising a subset of the tainted branch targets that have not been covered by the multiple different inputs; automatically placing breakpoints into the taint perimeter during runtime of the software code while the software code is currently executing on the at least one computing device; upon triggering of an individual breakpoint in the taint perimeter when executing the software code using a particular input, detecting that new code from the software code has been covered by the particular input; generating a new execution trace for the new code; performing additional data flow analysis on the new execution trace to identify additional tainted branch targets in the new code; filtering the additional tainted branch targets to identify an updated taint perimeter of the software code; automatically placing a new breakpoint into the updated taint perimeter; and upon triggering of the new breakpoint when executing the software code using a further input, detecting that further new code from the software code has been covered by the further input.
2. The method of claim 1 , further comprising monitoring the breakpoints in the taint perimeter to detect when the new code is covered.
3. The method of claim 1 , wherein the set of multiple different inputs comprises a set of templates that are existing valid inputs for the software code.
4. The method of claim 3 , further comprising generating a new test case from the set of templates, the new test case comprising the further input.
5. The method of claim 4 , further comprising: responsive to detecting that the further new code is covered by the further input, adding the further input to the set of templates.
6. The method of claim 5 , further comprising adding the new code and the further new code to a code coverage and tainted branch database.
7. The method of claim 1 , wherein the filtering the tainted branch targets comprises removing individual tainted conditional branch targets that have already been covered from the taint perimeter.
8. The method of claim 1 , wherein the determining the tainted branch targets comprises determining that the tainted branch targets are controlled by external data and excluding, from the tainted branch targets, some other branch targets in the software code that are not controlled by the external data.
9. At least one computer-readable volatile memory, non-volatile memory, hard drive, or optical disk storing computer-executable instructions which, when executed by at least one processing unit, cause the at least one processing unit to perform acts comprising: generating one or more execution traces of software code using a set of multiple different inputs to the software code; determining tainted branch targets by performing data flow analysis on the one or more execution traces of the software code, the tainted branch targets being associated with one or more tainted conditional branches in the software code; filtering the tainted branch targets to identify a taint perimeter of the software code, the taint perimeter comprising a subset of the tainted branch targets that have not been covered by the multiple different inputs; automatically placing breakpoints into the taint perimeter during runtime of the software code; upon triggering of a first breakpoint in the taint perimeter using a first input, detecting that additional code from the software code has been covered by the first input; generating another execution trace for the additional code; determining additional tainted branch targets in the additional code by performing additional data flow analysis on the another execution trace; filtering the additional tainted branch targets to identify an updated taint perimeter of the software code; automatically placing a second breakpoint into the updated taint perimeter; and upon triggering of the second breakpoint when executing the software code using a second input, detecting that further additional code from the software code has been covered by the second input.
10. The at least one computer-readable volatile memory, non-volatile memory, hard drive, or optical disk of claim 9 , the acts further comprising: placing the subset of the tainted branch targets in a code coverage and tainted branch database; and determining the taint perimeter using the code coverage and tainted branch database.
11. The at least one computer-readable volatile memory, non-volatile memory, hard drive, or optical disk of claim 9 , wherein the set of multiple different inputs is a set of templates that are existing valid inputs for the software code.
12. The at least one computer-readable volatile memory, non-volatile memory, hard drive, or optical disk claim 11 , the acts further comprising: adding the second input to the set of templates responsive to detecting that the further additional code has been covered by the second input.
13. The at least one computer-readable volatile memory, non-volatile memory, hard drive, or optical disk of claim 9 , wherein the data flow analysis comprises excluding, from the tainted branch targets, some other branch targets in the software code that are not controlled by external data.
14. A system comprising: one or more processing units; and at least one computer-readable volatile memory, non-volatile memory, hard drive, or optical disk storing computer-executable instructions which, when executed by the one or more processing units, cause the one or more processing units to: obtain first execution traces of software code, the first execution traces reflecting execution of the software code using multiple inputs; using first data flow analysis on the first execution traces, determine first tainted branch targets in the software code; filter the first tainted branch targets to identify a first taint perimeter comprising a subset of the first tainted branch targets that have not been covered by the multiple inputs; automatically place a first breakpoint in the first taint perimeter; responsive to the first breakpoint in the first taint perimeter being triggered by an additional input, detect that the additional input causes additional code of the software code to be executed; obtain a second execution trace of the additional code of the software code; using second data flow analysis on the second execution trace, determine second tainted branch targets in the additional code of the software code; filter the second tainted branch targets to identify a second taint perimeter comprising a subset of the second tainted branch targets that have not been covered by the multiple inputs and the additional input; automatically place a second breakpoint in the second taint perimeter; and responsive to the second breakpoint in the second taint perimeter being triggered by a further additional input, detect that the further additional input causes further additional code of the software code to be executed.
15. The system of claim 14 , embodied as a single computer.
16. The system of claim 14 , wherein the computer-executable instructions cause the one or more processing units to: perform both the first data flow analysis and the second data flow analysis, the first data flow analysis comprising evaluating first data flow through the software code in the first execution traces and the second data flow analysis comprising evaluating second data flow through the software code in the second execution trace.
17. The system of claim 14 , wherein the computer-executable instructions cause the one or more processing units to: iteratively refine the second taint perimeter by inserting subsequent breakpoints into the software code, executing the software code using subsequent inputs, and identifying subsequent additional code coverage when the subsequent inputs trigger the subsequent breakpoints.
18. The system of claim 14 , wherein the computer-executable instructions cause the one or more processing units to: place multiple third breakpoints into a third taint perimeter, the third taint perimeter comprising the further additional code.
19. The system of claim 14 , wherein the additional code is not executed in any of the first execution traces.
20. The system of claim 19 , wherein the further additional code is not executed in any of the first execution traces and also is not executed in the second execution trace.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
May 25, 2011
September 16, 2014
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.