Intelligently Managing Automatic Performance of Distributed Tests

1. A computerized method comprising: obtaining, by a processor of a testing platform, test definition data of a target test, wherein the target test is configured for distributed execution; analyzing device configuration data and predicted usage context data of user devices in a set of user devices using test requirements of the test definition data; identifying, for each user device in a set of user devices, a likelihood that the user device will enter a required device state during a test interval of the target test; determining that an average likelihood data value for the set of user devices satisfies the test requirements; determining that a quantity of user devices in the set of user devices satisfies a device quantity requirement of the test definition data; sending test execution control signals to the set of user devices, wherein the test execution control signals instruct the set of user devices to automatically execute the target test; and receiving test results associated with execution of the target test from the user devices.

2. The computerized method of claim 1, further comprising: identifying a first subset of user devices, to include in the set of user devices, based on the likelihood of the predicted usage context data of the respective user devices of the first subset exceeding a defined likelihood threshold; determining that the first subset of user devices fails to satisfy the device quantity requirement; based on the determining, reducing the defined likelihood threshold to a reduced likelihood threshold; and identifying a second subset of user devices, to include in the set of user devices, based on the likelihood of the respective user devices of the second subset exceeding the reduced likelihood threshold, wherein a combined set of the first subset of user devices and the second subset of user devices satisfies the device quantity requirement.

3. The computerized method of claim 1, further comprising: obtaining, by the processor, device data from individual user devices via a network connection, wherein the device data includes device configuration data and device usage data; and generating, by the processor, an individual usage prediction model for each individual user device based on the obtained device data.

4. The computerized method of claim 3, further comprising: updating, by the processor, the individual usage prediction model for each individual user device using machine learning techniques, wherein the received test results are used as training data.

5. The computerized method of claim 1, wherein the test requirements include a configuration requirement requiring qualifying user devices to have a defined hardware component.

6. The computerized method of claim 1, wherein the test requirements include a configuration requirement requiring qualifying user devices to have a defined software application or a defined version of a software application.

7. The computerized method of claim 1, wherein the test requirements include a usage requirement requiring qualifying user devices to enter a defined device state during the test interval of the target test.

8. The computerized method of claim 1, wherein the test requirements include a usage requirement requiring a defined device event to occur on qualifying user devices during the test interval of the target test.

9. The computerized method of claim 1, wherein the test requirements include a usage requirement requiring qualifying user devices to perform a defined operation during the test interval of the target test.

10. A computer system comprising: a processor of a testing platform; and a non-transitory computer readable medium storing programming instructions, that upon execution by the processor, cause the computer system to perform the following operations: obtaining test definition data of a target test, wherein the target test is configured for distributed execution; analyzing device configuration data and predicted usage context data of user devices in a set of user devices using test requirements of the test definition data; identifying, for each user device in a set of user devices, a likelihood that the user device will enter a required device state during a test interval of the target test; determining that an average likelihood data value for the set of user devices satisfies the test requirements; determining that a quantity of user devices in the set of user devices satisfies a device quantity requirement of the test definition data; sending test execution control signals to the set of user devices, wherein the test execution control signals instruct the set of user devices to automatically execute the target test; and receiving test results associated with execution of the target test from the user devices.

11. The computer system of claim 10, wherein the test requirements include a configuration requirement requiring qualifying user devices to have a defined hardware component.

12. The computer system of claim 10, wherein the test requirements include a configuration requirement requiring qualifying user devices to have a defined software application or a defined version of a software application.

13. The computer system of claim 10, wherein the test requirements include a usage requirement requiring qualifying user devices to enter a defined device state during the test interval of the target test.

14. The computer system of claim 10, wherein the test requirements include a usage requirement requiring a defined device event to occur on qualifying user devices during the test interval of the target test.

15. The computer system of claim 10, wherein the test requirements include a usage requirement requiring qualifying user devices to perform a defined operation during the test interval of the target test.

16. A non-transitory computer storage medium storing programming instructions, that upon execution by a processor of a computer system, causes the computer system to perform the following operations: obtaining test definition data of a target test, wherein the target test is configured for distributed execution; analyzing device configuration data and predicted usage context data of user devices in a set of user devices using test requirements of the test definition data; identifying, for each user device in a set of user devices, a likelihood that the user device will enter a required device state during a test interval of the target test; determining that an average likelihood data value for the set of user devices satisfies the test requirements; determining that a quantity of user devices in the set of user devices satisfies a device quantity requirement of the test definition data; sending test execution control signals to the set of user devices, wherein the test execution control signals instruct the set of user devices to automatically execute the target test; and receiving test results associated with execution of the target test from the user devices.

17. The non-transitory computer storage medium of claim 16, wherein the test requirements include a configuration requirement requiring qualifying user devices to have a defined hardware component.

18. The non-transitory computer storage medium of claim 16, wherein the test requirements include a configuration requirement requiring qualifying user devices to have a defined software application or a defined version of a software application.

19. The non-transitory computer storage medium of claim 16, wherein the test requirements include a usage requirement requiring qualifying user devices to enter a defined device state during the test interval of the target test.

20. The non-transitory computer storage medium of claim 16, wherein the test requirements include a usage requirement requiring a defined device event to occur on qualifying user devices during the test interval of the target test.