Efficient Use of Computational Resources for Interleaving

1. One or more computer-readable media storing computer-useable instructions that, when used by a computing device, cause the computing device to perform a method for facilitating an efficient interleaving process, the method comprising: receiving a search query; determining that the search query is to be used in an interleaving experiment, the interleaving experiment using a primary ranking system and a secondary ranking system; based on the search query, analyzing the primary ranking system and the secondary ranking system to determine, prior to routing the search query to either ranking system, whether search results produced by the primary ranking system and the secondary ranking system will be identical, wherein analyzing the primary ranking system and the secondary ranking system further comprises, identifying one or more parameters used to rank documents that are different between the primary ranking system and the secondary ranking system, and determining whether the search query is associated with any of the one or more parameters such that, (1) if the search query is associated with any of the one or more parameters, determining that the search results would not be identical, and (2) if the search query is not associated with any of the one or more parameters, determining that the search results would be identical; if it is determined that the search results will be identical, routing the search query only to the primary ranking system; and if it is determined that the search results will not be identical, routing the search query to both the primary ranking system and the secondary ranking system and interleaving the search results from the primary ranking system and the secondary ranking system to generate a single set of search results to return to a user based on the search query.

2. The computer-readable media of claim 1 , wherein the interleaving experiment compares the search results produced by the primary ranking system and the secondary ranking system and determines which produces more relevant search results.

3. The computer-readable media of claim 2 , wherein determining which produces more relevant search results is based on associating the search results that are selected more often to the primary ranking system or the secondary ranking system.

4. The computer-readable media of claim 1 , wherein if it is determined that the search results would be identical, further comprising returning the search results produced by the primary ranking system to a user based on the search query.

5. The computer-readable media of claim 1 , further comprising communicating to the secondary ranking system that it is being used in the interleaving experiment.

6. The computer-readable media of claim 5 , further comprising during processing of the search query by the secondary ranking system but prior to completion of the search results, determining at the secondary ranking system that the search results will be identical to the search results of the primary ranking system.

7. The computer-readable media of claim 6 , further comprising discontinuing the processing of the search query in the secondary ranking system and providing a user the search results of the primary ranking system.

8. The computer-readable media of claim 1 , further comprising: receiving, at an interleaving component, a primary set of search results and a secondary set of search results; determining whether the primary and the secondary sets of search results are identical; if the primary and the secondary sets of search results are identical, returning the primary set of search results to a user based on the search query; and if the primary and the secondary sets of search results are not identical, (1) interleaving the primary set of search results with the secondary set of search results, and (2) returning an interleaved set of search results to the user.

9. One or more computer-readable media storing computer-useable instructions that, when used by a computing device, cause the computing device to perform a method for facilitating an efficient interleaving process, the method comprising: receiving a search query; determining that the search query is involved in an interleaving experiment that uses a primary ranking system and a secondary ranking system; routing the search query to the primary ranking system and the secondary ranking system, wherein the secondary ranking system has additional logic incorporated therein so that it is aware that it is being used in the interleaving experiment; communicating to the secondary ranking system that it is being used in the interleaving experiment so that it knows to discontinue the processing of the search query if it determines that the search results will be identical to the search results of the primary ranking system; at the secondary ranking system during processing of the search query, determining that the search results of the secondary ranking system will be identical to the search results of the primary ranking system, wherein the determining further comprises, identifying one or more parameters used to rank documents that are different between the primary ranking system and the secondary ranking system, and determining that the search query is not associated with any of the one or more parameters; and discontinuing the processing of the search query in the secondary ranking system.

10. The computer-readable media of claim 9 , further comprising continuing the processing of the search query in the primary ranking system.

11. The computer-readable media of claim 9 , further comprising providing the search results from the primary ranking system to a user based on the search query.

12. The computer-readable media of claim 9 , wherein the secondary ranking system does not return search results to an interleaving component.

13. One or more computer-readable media storing computer-useable instructions that, when used by a computing device, cause the computing device to perform a method for facilitating an efficient interleaving process, the method comprising: determining that a search query is involved in an interleaving experiment; duplicating the search query such that the search query can be sent to a primary ranking system and a secondary ranking system; identifying one or more ranking components associated with the secondary ranking system that are used to produce a ranked list of search results; routing the search query through each standard component of the primary ranking system; routing the duplicate search query only through the one or more ranking components identified in the secondary ranking system; and prior to interleaving the search results from the primary ranking system and the secondary ranking system, determining whether primary search results associated with the primary ranking system and secondary search results associated with the secondary ranking system are identical such that, (1) if the primary search results are identical to the secondary search results, not running an interleaving algorithm that interleaves the search results, and (2) if the primary search results are not identical to the secondary search results, running the interleaving algorithm that interleaves the search results.

14. The one or more computer-readable media of claim 13 , wherein the duplicate search query is routed to the secondary ranking system from the point at which components differ from those in the primary ranking system.

15. The one or more computer-readable media of claim 13 , wherein the primary search results are identical to the secondary search results if a predetermined quantity of the search results are identical.