Techniques for Distributing Tests and Test Suites Across a Network

1. A method, including steps of at one or more test leaders in a distributed computing system, said test leaders provisioning resources for one or more test suites, wherein said test suites compete for said resources to be provisioned by said test leaders in said distributed computing system; at said test leaders, directing said resources to perform operations according to those test suites, said operations according to those test suites including one or more operations of communicating using said distributed computing system, and said operations at least potentially interfering when performed concurrently; at said test leaders, accessing one or more results from said operations; comparing results from said operations with one or more results indicated by said test suites; at said test leaders, accessing results from said operations; including steps of sending messages from said resources to said test leaders; wherein said steps of comparing are performed at said test leaders.

2. A method as in claim 1 , including steps, at resources of sending messages to said test leaders on their own initiative, providing results to said test leaders when said results are available.

3. A method, as in claim 1 , including steps, at said test leaders, of sending messages to said resources directing them to provide results to said test leaders.

4. A method, as in claim 1 , including steps of sending messages from said resources to said test leaders; and wherein said steps of comparing are performed at said test leaders.

5. A method as in claim 1 , wherein, said operations are performed in relative time proximity, said relative time proximity being substantially within a time to send messages amount said resources.

6. A method as in claim 1 , wherein, said steps of directing said resources to perform operations include steps of distributing information interpretable as indicating said operations before said operations are to be performed; at said resources, interpreting said distributed information; and at said resources, performing said operations at times said test suites indicate they are to be performed.

7. A method as in claim 1 , wherein said steps of directing said resources to perform operations include steps of sending messages to said resources substantially at times when said operations are to be performed.

8. A method as in claim 1 , wherein said steps of directing said resources to perform operations include steps of synchronizing said resources with said test leader; and sending one or more messages to said resources substantially before said operations are to be performed.

9. A method as in claim 8 , wherein said steps of synchronizing include steps of at said resources, sending ore or more messages to said test leaders indicating that said operations are about to be performed; at said test leaders, responding to said messages to said test leaders.

10. A method as in claim 1 , wherein said steps of provisioning and directing include steps of provisioning said resources for a portion of said test suites, and directing said resources to perform a perform a portion of said operations; and repeating said steps of provisioning and directing portions, until substantially all of said test suites are substantially completed.

11. A method as in claim 1 , wherein said steps of provisioning and directing include steps of provisioning said resources and directing said resources for a plurality of test suites to be performed substantially concurrently.

12. A method as in claim 1 , wherein said steps of provisioning resources include steps of ordering said steps of provisioning in response to a measure of performance of said test suites.

13. A method as in claim 1 , wherein said steps of provisioning resources include steps of provisioning said resources in response to a plurality of test suites, wherein said resources overlap within said plurality of test suites; wherein said steps of provisioning are performed substantially before directing any said resources to perform operations.

14. A method as in claim 1 , wherein said test suites include information indicating one or more resources to be used in conjunction with said test suites; one or more operations to be performed in conjunction with said test suites; one or more tags distinguishing types of said test suites.

15. A method as in claim 14 , wherein said resources include one or more of: data structures, processors, storage devices; said data structures including one or more of: databases, locks, shared directories, test results.

16. A physical medium including information accessible by a computing device said information being interpretable by said computing device as a set of instructions directing an element to at one or more test leaders in a distributed operating system, disposing said test leaders to provision resources for one or more test suites, wherein said test suites compete for said resources to be provisioned by said test leaders in said distributed computing system; at said test leaders, directing said resources to perform operations according to those test suites, said operations according to those test suites including one or more operations of communicating using said distributed computing system, and said operations at least potentially interfering when performed concurrently; at said test leaders, access one or more results from said operations; at said resources, send messages to said test leaders on their own initiative, providing results to said test leaders at times when said results are available; send messages from said resources to said test leaders, wherein said instructions to compare results are performed at said test leaders.

17. A physical medium as in claim 16 , wherein said information interpretable as a set of instructions includes information interpretable as instructions to compare results from said operations with one or more results indicated by said test suites; and at said test leaders, access results from said operations.

18. A physical medium as in claim 17 , including instructions to at said test leaders, send messages to said resources directing them to provide results to said test leaders.

19. A physical medium as in claim 16 , wherein said instructions to direct said resources to perform operations include instructions to send messages to said resources substantially at times when said operations are to be performed.

20. A physical medium as in claim 16 , wherein said instructions to direct said resources to perform operations include instructions to synchronize said resources with one or more test leaders; and send one or more messages to said resources substantially at times when said operations are to be performed.

21. A physical medium as in claim 16 , wherein said instructions to direct said resources to perform operations include instructions to synchronize said resources with one or more test leaders; and send one or more messages to said resources substantially before said operations are to be performed.

22. A physical medium as in claim 21 , wherein said instructions to synchronize include instructions to at said resources, send one or more messages to said test leaders indicating that said operations are about to be performed; at said test leaders, respond to said messages to said test leaders.

23. A physical medium as in claim 16 , wherein said instructions to provision and direct include instructions to provision said resources for a portion of said test suites, and direct said resources to perform a portion of said operations; and repeat said instruction to provision and direct portions, until substantially all of said test suites are substantially completed.

24. A physical medium as in claim 16 , wherein said instructions to provision and direct include instructions to provision said resources and direct said resources for a plurality of test suites to be performed substantially concurrently.

25. A physical medium as in claim 16 , wherein said instructions to provision resources include instructions to order said instructions to provision in response to a measure of performance of said test suites.

26. A physical medium as in claim 16 , wherein said instructions to provision resources include instructions to provision said resources in response to a plurality of test suites, wherein said resources overlap within said plurality of test suites; and wherein said instructions to provision are performed substantially before directing any said resources to perform operations.