Simulated Bucket Testing

1. A method, comprising: obtaining a first probability distribution for a first group of users, the first group of users comprising at least two users, and a second probability distribution for a second group of users, the second group of users comprising at least two users; determining an intersection of the first probability distribution and of the second probability distribution; selecting a cohort of users associated with a value for an engagement metric based at least in part on the intersection; and comparing behavior of the first and second groups based at least in part on the selected cohort of users; wherein at least the determining, selecting, and comparing are performed via one or more computing platforms.

2. The method of claim 1 , further comprising determining the first probability distribution for the first group of users and the second probability distribution for the second group of users based at least in part on network log data.

3. The method of claim 1 , wherein said comparing the behavior of the first and second groups comprises comparing an effect of a given feature on the behavior of the first and second groups.

4. The method of claim 1 , wherein said comparing the behavior of the first and second groups comprises comparing an effect of a given feature on the behavior of the first and second groups, and wherein the given feature comprises a feature associated with a network web page.

5. The method of claim 1 , wherein said comparing the behavior of the first and second groups comprises comparing an effect of a given feature on the behavior of the first and second groups, and wherein the first group of users comprises users that did not select the given feature and the second group of users comprises users that did select the given feature.

6. The method of claim 1 , wherein the cohort of users comprises a first cohort set of users comprising users that did not select a given feature and a second cohort set of users comprising users that did select the given feature.

7. The method of claim 1 , further comprising determining a mean of the cohort of users based at least in part on a meeting point of the intersection of the first probability distribution for the first group of users and of the second probability distribution for the second group of users.

8. The method of claim 1 , further comprising determining a subset of the cohort of users which comprises a substantially normally distributed set of users, and wherein the subset of the cohort of users comprises a first cohort set of users comprising users that did not select a given feature and a second cohort set of users comprising users that did select the given feature.

9. The method of claim 1 , further comprising determining a subset of the cohort of users which comprises a substantially normally distributed set of users, wherein the subset of the cohort of users comprises a first cohort set of users comprising users that did not select a given feature and a second cohort set of users comprising users that did select the given feature, and wherein said determining the subset of the cohort of users comprises: setting an initial estimate of a standard deviation; determining a probability that the subset of the cohort of users will include an individual user for each individual user in the cohort of users; including individual users with a given probability within the subset of the cohort of users; increasing the size of the standard deviation by a step size amount; and repeating said determining the probability that the subset of the cohort of users will include an individual user for each user in the cohort of users until the standard deviation reaches a maximum standard deviation.

10. The method of claim 1 , further comprising: determining the first probability distribution for the first group of users and the second probability distribution for the second group of users based at least in part on Internet log data; wherein the cohort of users comprises a first cohort set of users comprising users that did not select a given feature and a second cohort set of users comprising users that did select the given feature, and wherein the given feature comprises a feature associated with an Internet web page; determining a mean of the cohort of users based at least in part on a meeting point of the intersection of the first probability distribution for the first group of users and of the second probability distribution for the second group of users; determining a subset of the cohort of users which comprises a normally distributed set of users, wherein the subset of the cohort of users comprises a first cohort set of users comprising users that did not select a given feature and a second cohort set of users comprising users that did select the given feature, and wherein said determining the subset of the cohort of users comprises: setting an initial estimate of standard deviation; determining a probability that the subset of the cohort of users will include an individual user for each individual user in the cohort of users; including individual users with a given probability within the subset of the cohort of users; increasing the size of the standard deviation by a step size amount; and repeating said determining the probability that the subset of the cohort of users will include an individual user for each user in the cohort of users until the standard deviation reaches a maximum standard deviation; and wherein said comparing the behavior of the first and second groups comprises comparing an effect of the given feature on the behavior of the first and second cohort sets of users.

11. An article comprising: a non-transitory storage medium comprising machine-readable instructions stored thereon which are executable by one or more processors to: process a first probability distribution for a first group of users, the first group of users comprising at least two users, and a second probability distribution for a second group of users, the second group of users comprising at least two users; determine an intersection of the first probability distribution and of the second probability distribution; select a cohort of users associated with a value for an engagement metric based at least in part on the intersection; and compare behavior of the first and second groups based at least in part on the selected cohort of users.

12. The article of claim 11 , wherein the cohort of users comprises a first cohort set of users comprising users that did not select a given feature and a second cohort set of users comprising users that did select the given feature.

13. The article of claim 11 , wherein said machine-readable instructions are further executable by said one or more processors to: determine a mean of the cohort of users based at least in part on a meeting point of the intersection of the first probability distribution for the first group of users and of the second probability distribution for the second group of users.

14. The article of claim 11 , wherein said machine-readable instructions are further executable by said one or more processors to: determine a subset of the cohort of users which comprises a normally distributed set of users, and wherein the subset of the cohort of users comprises a first cohort set of users comprising users that did not select a given feature and a second cohort set of users comprising users that did select the given feature.

15. The article of claim 11 , wherein said machine-readable instructions are further executable by said one or more processors to: determine a subset of the cohort of users which comprises a normally distributed set of users, wherein the subset of the cohort of users comprises a first cohort set of users comprising users that did not select a given feature and a second cohort set of users comprising users that did select the given feature, by: setting an initial estimate of a standard deviation; determining a probability that the subset of the cohort of users will include an individual user for each individual user in the cohort of users; including individual users with a given probability within the subset of the cohort of users; increasing the size of the standard deviation by a step size amount; and repeating said determining the probability that the subset of the cohort of users will include an individual user for each user in the cohort of users until the standard deviation reaches a maximum standard deviation.

16. An apparatus comprising: a computing platform, said computing platform being adapted to: obtain a first probability distribution for a first group of users, the first group of users comprising at least two users, and a second probability distribution for a second group of users, the second group of users comprising at least two users; determine an intersection of the first probability distribution and of the second probability distribution; select a cohort of users associated with a value for an engagement metric based at least in part on the intersection; and compare behavior of the first and second groups based at least in part on the selected cohort of users.

17. The apparatus of claim 16 , wherein the cohort of users comprises a first cohort set of users comprising users that did not select a given feature and a second cohort set of users comprising users that did select the given feature.

18. The apparatus of claim 16 , wherein said computing platform is further adapted to: determine a mean of the cohort of users based at least in part on a meeting point of the intersection of the first probability distribution for the first group of users and of the second probability distribution for the second group of users.

19. The apparatus of claim 16 , wherein said computing platform is further adapted to: determine a subset of the cohort of users which comprises a normally distributed set of users, and wherein the subset of the cohort of users comprises a first cohort set of users comprising users that did not select a given feature and a second cohort set of users comprising users that did select the given feature.

20. The apparatus of claim 16 , wherein said computing platform is further adapted to: determine a subset of the cohort of users which comprises a normally distributed set of users, wherein the subset of the cohort of users comprises a first cohort set of users comprising users that did not select a given feature and a second cohort set of users comprising users that did select the given feature, by: setting an initial estimate of a standard deviation; determining a probability that the subset of the cohort of users will include an individual user for each individual user in the cohort of users; including individual users with a given probability within the subset of the cohort of users; increasing the size of the standard deviation by a step size amount; and repeating said determining the probability that the subset of the cohort of users will include an individual user for each user in the cohort of users until the standard deviation reaches a maximum standard deviation.

21. An apparatus comprising: means for obtaining a first probability distribution for a first group of users, the first group of users comprising at least two users, and a second probability distribution for a second group of users, the second group of users comprising at least two users; means for determining an intersection of the first probability distribution and of the second probability distribution; means for selecting a cohort of users associated with a value for an engagement metric based at least in part on the intersection; and means for comparing behavior of the first and second groups based at least in part on the selected cohort of users.

22. The apparatus of claim 21 , wherein the cohort of users comprises a first cohort set of users comprising users that did not select a given feature and a second cohort set of users comprising users that did select the given feature.

23. The apparatus of claim 21 , further comprising: means for determining a mean of the cohort of users based at least in part on a meeting point of the intersection of the first probability distribution for the first group of users and of the second probability distribution for the second group of users.

24. The apparatus of claim 21 , further comprising: means for determining a subset of the cohort of users which comprises a normally distributed set of users, and wherein the subset of the cohort of users comprises a first cohort set of users comprising users that did not select a given feature and a second cohort set of users comprising users that did select the given feature.

25. The apparatus of claim 21 , further comprising: means for determining a subset of the cohort of users which comprises a normally distributed set of users, wherein the subset of the cohort of users comprises a first cohort set of users comprising users that did not select a given feature and a second cohort set of users comprising users that did select the given feature, and wherein said means for determining the subset of the cohort of users comprises: means for setting an initial estimate of a standard deviation; means for determining a probability that the subset of the cohort of users will include an individual user for each individual user in the cohort of users; means for including individual users with a given probability within the subset of the cohort of users; means for increasing the size of the standard deviation by a step size amount; and means for repeating said determining the probability that the subset of the cohort of users will include an individual user for each user in the cohort of users until the standard deviation reaches a maximum standard deviation.