Methods, Systems, and Articles of Manufacture for Determining Optimal Parameter Settings for Business Initiative Testing Models

1. A method for determining optimal parameter settings for business initiative testing software used for testing initiatives for business locations included in a business network, comprising: identifying, by a computer, a business initiative testing model having a set of parameter settings; selecting a first parameter setting set for performing the virtual test, the first parameter setting set including a set of selected parameter setting options each respectively corresponding to one of the parameter settings for the business initiative testing model; performing, by a computer, a virtual test on a set of virtual test sites, each virtual test site reflecting a selected business location in the business network, wherein each virtual test is a simulated business initiative test performed on test sites where no actual initiative test has been implemented at those test sites, and wherein the virtual test is performed on the virtual test sites using a variation of each parameter setting; determining, by a computer, actual performance data associated with the set of virtual test sites; determining, by a computer, actual performance data associated with a set of control group sites reflecting second selected business locations in the business network using the tested parameter settings; determining a noise value for the first parameter setting set, the noise value reflecting an inconsistency between performance data associated, with the set of virtual test sites and performance data associated with the set of control group sites reflecting second selected business locations in the business network using the tested parameter settings; determining, by a computer, a set of optimal parameter settings for the business initiative testing model based on results from the virtual test whereby the optimal parameter settings best minimize noise from the results; and configuring, by a computer, the business initiative testing model using the optimal parameter settings to test a business initiative for application in the business network.

2. The method of claim 1, wherein performing the virtual test includes: performing a series of virtual tests for each possible combination of the parameter settings for the business initiative testing model.

3. The method of claim 2, wherein iteratively performing the series of virtual tests includes: determining a noise value for the respective parameters settings combination for each iteration of the series of virtual tests for a given parameter settings combination.

4. The method of claim 3, further including: for each iteration of the series of virtual tests for a given parameter settings combination, storing the determined noise value in a data structure including data reflecting noise values for each iteration of the virtual test performed on the virtual test sites for each of the parameter settings combinations.

5. The method of claim 3, further including: for each given parameter settings combination, determining an average noise value for the given parameter settings combination based on stored noise values for each iteration of the virtual test for the given parameter settings combination.

6. The method of claim 2, wherein performing the series of virtual tests for each possible combination of the parameter settings includes: for each iteration of the series of virtual tests, randomly selecting the set of virtual test sites, and randomly selecting a test start date for collecting historical data associated with each of the randomly selected virtual test sites.

7. The method of claim 1, further including: iteratively performing the virtual test on the set of virtual test sites using the first parameter setting set until an iteration threshold is met.

8. The method of claim 1, wherein iteratively performing the virtual test includes: storing a noise value for each iteration and first parameter setting in a data structure.

9. The method of claim 1, further including: selecting a second parameter setting set when the iteration threshold is met; and iteratively performing the virtual test on the set of virtual test sites using the second parameter setting set until the iteration threshold is met.

10. The method of claim 1, wherein iteratively performing the virtual test includes: storing a noise value for each iteration and second parameter setting in the data structure.

11. The method of claim 1, wherein determining optimal parameter settings for the first test type includes: analyzing the noise values associated with the first and second parameter settings sets; and selecting either the first or second parameter settings set as the optimal parameter settings based on whether the first or second parameter settings sets have a lower average noise value over the iterations of the virtual test.

12. The method according to claim 1, further comprising defining a test type associated with the business initiative testing model, wherein the test type is defined based on an attribute of the business initiative test.

13. The method of claim 12, wherein determining the set of optimal parameter settings includes: identifying the parameter settings combination having the lowest average noise value as the optimal parameter settings for the first test type of the business initiative testing model.

14. The method of claim 12, wherein configuring the business initiative testing model further includes: receiving a request identifying the first test type to test the business initiative to apply to the business network.

15. The method of claim 12, wherein defining the first test type includes: defining a test period and a test pre-period, each reflecting a period of time that historical data was collected for the business locations included in the business network.

16. The method of claim 12, further including: defining a second test type of the business initiative model; performing the virtual test based on the defined second test type to determine second optimal parameter settings for the second test type; and performing a business initiative test using the second test type of the business initiative testing model that is automatically configured using the second optimal parameter settings.

17. The method according to claim 1, wherein each parameter setting comprises a set of one or more parameter setting options, wherein a parameter setting option further defines a parameter setting.

18. A system for determining optimal parameter settings for business initiative testing software used for testing initiatives for business locations included in a business network, the system including: a computer configured to: identify a business initiative testing model having a set of parameter settings; selecting a first parameter setting set for performing the virtual test, the first parameter setting set including a set of selected parameter setting options each respectively corresponding to one of the parameter settings for the business initiative testing model; execute a virtual test on a set of virtual test sites, each virtual test site reflecting a selected business location in the business network, wherein each virtual test is a simulated business initiative test for-test sites where no actual initiative test has been implemented at those test sites, and wherein the virtual test is performed on the virtual test sites using a variation of each parameter setting; determine actual performance data associated with the set of virtual test sites; determine actual performance data associated with a set of control group sites reflecting second selected business locations in the business network using the tested parameter settings; determining a noise value for the first parameter setting set, the noise value reflecting an inconsistency between performance data associated, with the set of virtual test sites and performance data associated with the set of control group sites reflecting second selected business locations in the business network using the tested parameter settings; determine a set of optimal parameter settings for the business initiative testing model based on results from the virtual test whereby the optimal parameter settings best minimize noise from the results; and configure the business initiative testing model using the optimal parameter settings to a test on a business initiative to apply to the business network.

19. The system of claim 18, wherein the computer is further configured to: perform a series of virtual tests for each possible combination of the parameter settings for the business initiative testing model.

20. The system of claim 19, wherein the computer is further configured to perform the series of virtual tests by: determining a noise value for the respective parameters settings combination for each iteration of the series of virtual tests for a given parameter settings combination.

21. The system of claim 20, wherein the computer is further configured to: for each iteration of the series of virtual tests for a given parameter settings combination, store the determined noise value in a data structure including data reflecting noise values for each iteration of the virtual test performed on the virtual test sites for each of the parameter settings combinations.

22. The system of claim 20, wherein the computer is further configured to: for each given parameter settings combination, determine an average noise value for the given parameter settings combination based on stored noise values for each iteration of the virtual test for the given parameter settings combination.

23. The system of claim 22, wherein the computer is further configured to determine the set of optimal parameter settings by identifying the parameter settings combination having the lowest average noise value as the optimal parameter settings of the business initiative testing model.

24. The system of claim 19, wherein the computer is further configured to perform the series of virtual tests for each possible combination of the parameter settings by: for each iteration of the series of virtual tests, randomly selecting the set of virtual test sites, and randomly selecting a test start date for collecting historical' data associated with each of the randomly selected virtual test sites.

25. The system of claim 24, wherein the computer is further configured to: defining a test period and a test pre-period, each reflecting a period of time that historical data was collected for the business locations included in the business network.

26. The system of claim 18, wherein the computer is further configured to: receive a request identifying the first test type to test the business initiative to apply to the business network, and define the first test type of the business initiative testing model based on the request.

27. The system of claim 18, wherein the computer is further configured to: iteratively perform the virtual test on the set of virtual test sites using the first parameter setting set until an iteration threshold is met.

28. The system of claim 18, wherein the computer is further configured to iteratively perform the virtual test by storing a noise value for each iteration and first parameter setting in a data structure.

29. The system of claim 18, wherein the computer is further configured to: select a second parameter setting set when the iteration threshold is met; and iteratively perform the virtual test on the set of virtual test sites using the second parameter setting set until the iteration threshold is met.

30. The system of claim 18, wherein the computer is further configured to iteratively perform the virtual test by storing a noise value for each iteration and second parameter setting in the data structure.

31. The system of claim 18, wherein the computer is further configured to determine optimal parameter settings by: analyzing the noise values associated with the first and second parameter settings sets; and selecting either the first or second parameter settings set as the optimal parameter settings based on whether the first or second parameter settings sets have a lower average noise value over the iterations of the virtual test.

32. The system according to claim 18, wherein a test type is associated with the business initiative testing model and is defined based on an attribute of the business initiative test.

33. The system of claim 32, wherein the computer is further configured to: define a second test type of the business initiative model; perform the virtual test based on the defined second test type to determine second optimal parameter settings for the second test type; and perform a business initiative test using the second test type of the business initiative testing model that is configured using the second optimal parameter settings.

34. A non-transitory computer-readable medium including instructions for, when executed by a processor, performing a method for determining optimal parameter settings for business initiative testing software used for testing initiatives for business locations included in a business network, the method comprising: identifying, by a computer, a business initiative testing model having a set of parameter settings; selecting a first parameter setting set for performing the virtual test, the first parameter setting set including a set of selected parameter setting options each respectively corresponding to one of the parameter settings for the business initiative testing model; performing, by a computer, a virtual test on a set of virtual test sites, each virtual test site reflecting a selected business location in the business network, wherein each virtual test is a simulated business initiative test using a variation of each parameter setting and is performed on test sites where no actual initiative test is performed on the virtual test sites; determining, by a computer, actual performance data associated with the set of virtual test sites; determining, by a computer, actual performance data associated with a set of control group sites reflecting second selected business locations in the business network using the tested parameter settings; determining a noise value for the first parameter setting set, the noise value reflecting an inconsistency between performance data associated, with the set of virtual test sites and performance data associated with the set of control group sites reflecting second selected business locations in the business network using the tested parameter settings; determining, by a computer, a set of optimal parameter settings for the business initiative testing model based on results from the virtual test whereby the optimal parameter settings best minimize noise from the results; and configuring, by a computer, the business initiative testing model using the optimal parameter settings to test a business initiative to apply to the business network.

35. A method for testing business initiatives comprising: receiving, by a computer, a request to test a business initiative to be applied to a set of business locations in a business network; determining, by a computer, actual performance data associated with a set of virtual test sites; determining, by a computer, actual performance data associated with a set of control group sites reflecting second selected business locations in the business network using the tested parameter settings; configuring, by a computer, a business initiative testing model based on the request and a default set of parameter settings for the model, each default parameter setting having a set of parameter setting options and the default set of parameter settings reflecting optimal parameter settings for the model that are determined by identifying a combination of parameter setting options that produce the least amount of noise during iterations of a set of virtual business initiative tests performed for selected virtual test sites, wherein each virtual test is a simulated business initiative test performed on test sites where no actual initiative test has been implemented at those test sites and wherein noise is a measurement of inconsistency between performance data associated with the set of virtual test sites and performance data associated with the set of control group sites reflecting second selected business locations in the business network using the tested parameter settings; and testing, by a computer, the business initiative using the configured business initiative testing model to produce results reflecting a predicted performance of the business initiative if applied to the set of business locations.

36. A method for determining optimal parameter settings for business initiative testing software used for testing initiatives for business locations included in a business network, comprising: identifying, by a computer, a business initiative testing model having a set of parameter settings; selecting, by the computer, a first parameter setting set for performing a virtual test, the first parameter setting set including a set of selected parameter setting options each respectively corresponding to one of the parameter settings for the business initiative testing model; performing, by the computer, the virtual test on a set of virtual test sites, each virtual test site reflecting a selected business location in the business network, wherein each virtual test is a simulated business initiative test performed on virtual test sites where no actual initiative test has been implemented at those virtual test sites, and wherein the virtual test is performed on the set of virtual test sites using a variation of each parameter setting; determining, by the computer, actual performance data associated with the set of virtual test sites; determining, by the computer, actual performance data associated with a set of control group sites reflecting second selected business locations in the business network using tested parameter settings; determining, by the computer, a noise value for the first parameter setting set, the noise value reflecting an inconsistency between the actual performance data associated with the set of virtual test sites and the actual performance data associated with the set of control group sites reflecting second selected business locations in the business network using the tested parameter settings; determining, by the computer, a set of optimal parameter settings for the business initiative testing model based on results from the virtual test whereby the optimal parameter settings best minimize the noise value from the results; configuring, by the computer, the business initiative testing model using the optimal parameter settings to test a business initiative for application in the business network; and executing, by the computer, the business initiative testing model trained via the optimal parameter settings.

37. The method of claim 36, wherein performing the virtual test includes: performing, by the computer, a series of virtual tests for each possible combination of the parameter settings for the business initiative testing model.

38. The method of claim 37, wherein iteratively performing the series of virtual tests includes: determining, by the computer, a noise value for the respective parameters settings combination for each iteration of the series of virtual tests for a given parameter settings combination.

39. The method of claim 38, further including: for each iteration of the series of virtual tests for the given parameter settings combination, storing, by the computer, the determined noise value in a data structure including data reflecting noise values for each iteration of the virtual test performed on at least one of the set of virtual test sites for each of the parameter settings combinations.

40. The method of claim 38, further including: for each given parameter settings combination, determining, by the computer, an average noise value for the given parameter settings combination based on stored noise values for each iteration of the virtual test for the given parameter settings combination.

41. The method of claim 37, wherein performing the series of virtual tests for each possible combination of the parameter settings includes: for each iteration of the series of virtual tests, randomly selecting by the computer, the set of virtual test sites, and randomly selecting, by the computer, a test start date for collecting historical data associated with each of the randomly selected virtual test sites.

42. The method of claim 36, further including: iteratively performing by the computer, the virtual test on the set of virtual test sites using the first parameter setting set until an iteration threshold is met.

43. The method of claim 42, further including: selecting, by the computer, a second parameter setting set when the iteration threshold is met; and iteratively performing, by the computer, the virtual test on the set of virtual test sites using the second parameter setting set until the iteration threshold is met.

44. The method of claim 43, wherein iteratively performing the virtual test includes: storing, by the computer, a noise value for each iteration and second parameter setting set in a data structure.

45. The method of claim 43, wherein determining optimal parameter settings for a first test type includes: analyzing, by the computer, the noise values associated with the first and second parameter setting sets; and selecting, by the computer, either the first or second parameter setting set as the optimal parameter settings based on whether the first and second parameter setting sets have a lower average noise value over iterations of the virtual test, wherein the first test type is defined based on an attribute of the simulated business initiative test.

46. The method of claim 45, wherein determining the set of optimal parameter settings includes: identifying, by the computer, the parameter settings combination having a lowest average noise value as the optimal parameter settings for the first test type of the business initiative testing model.

47. The method of claim 45, wherein configuring the business initiative testing model further includes: receiving, by the computer, a request identifying the first test type to test the business initiative to apply to the business network.

48. The method of claim 45, wherein defining the first test type includes: defining, by the computer, a test period and a test pre-period, each reflecting a period of time that historical data was collected for the business locations included in the business network.

49. The method of claim 45, further including: defining, by the computer, a second test type of the business initiative testing model; performing, by the computer, the virtual test based on the defined second test type to determine second optimal parameter settings for the second test type; and performing, by the computer, the simulated business initiative test using the second test type of the business initiative testing model that is automatically configured using the second optimal parameter settings.

50. The method of claim 36, wherein iteratively performing the virtual test includes: storing, by the computer, a noise value for each iteration and first parameter setting in a data structure.

51. The method of claim 36, further comprising: defining, by the computer, a test type associated with the business initiative testing model, wherein the test type is defined based on an attribute of the simulated business initiative test.

52. The method according to claim 36, wherein each parameter setting comprises a set of one or more parameter setting options, wherein a parameter setting option further defines a parameter setting.