Techniques are described herein for processing queries comprising joins specifying a plurality of tables. The techniques involve partitioning the tables by assigning rows to partitions. One or more partition maps may be generated to indicate the partitions. Subsequent tables may be partitioned based on the generated partition maps. The partitions may be used to determine results for the joins.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method comprising: receiving, at a database system, a query comprising a multi-way join comprising a plurality of join operations, each join operation of said plurality of join operations involving a respective first join key of a respective first table and a respective second join key of a respective second table; wherein the plurality of join operations includes a first join operation, a second join operation, and a third operation; for the first join operation: partitioning the respective first table of the first join operation into a plurality of partitions based on said the respective first join key of said first join operation; generating a first partition map that maps said plurality of partitions to values contained in a respective first join key of the second join operation, said respective first join key of the second join operation being a column of the respective first table of said first join operation; partitioning the respective second table of the first join operation into said plurality of partitions based on said the respective second join key of said first join operation; generating a second partition map that maps said plurality of partitions to values contained in a respective first join key of the third join operation, said respective first join key of the third join operation being a column of the respective second table of said first join operation; partitioning the respective second table of said second join operation based on the first partition map and the respective second join key of said second join operation; partitioning the respective second table of the third join operation based on the second partition map and the respective second join key of the third join operation.
2. The method of claim 1 , wherein partitioning the respective first table and the respective second table of the first join operation comprises: applying a hash function to the respective first join key; and applying a hash function to the respective second join key.
3. The method of claim 1 further comprising: generating a third partition map that maps values contained in a respective second join key of the third join operation, said respective second join key of said third join operation being a column of the respective second table of said second join operation; partitioning the respective second table of the third join operation based on the second partition map and the third partition map.
4. The method of claim 3 further comprising: filtering one or more rows of the respective second table of the third join operation based on the second partition map and the third partition map.
5. The method of claim 1 further comprising storing the plurality of partitions in local memory of the database system.
6. The method of claim 5 further comprising storing a particular partition in a disk if the local memory of the database system cannot store the entire partition.
7. The method of claim 1 , wherein the database system is a multi-node database system, the method further comprising: assigning rows of the first respective table and the second respective table of the first join operation to one or more nodes of the multi-node database system; based on the assignment, generating a distribution map; based on the distribution map, assigning rows of the first respective table and the second respective table of the second join operation and the third join operation to the one or more nodes in the multi-node database system; and each of the one or more nodes performing the partitioning and generating for tables assigned to the node.
8. The method of claim 7 further comprising storing the plurality of partitions in respective local memory the one or more nodes.
9. The method of claim 8 further comprising storing a particular partition in a shared disk if the respective local memory cannot store the entire partition.
10. The method of claim 1 further comprising determining, based on the plurality of partitions, results for the multi-way join.
11. The method of claim 10 wherein determining results for the multi-way join further comprises: generating a first hash table based on the respective second table for the first join operation and the respective second join key of the first join operation; generating a second hash table based on the respective first table of the second join operation and the respective first join key of the second join operation; wherein determining results is based on the first hash table and the second hash table.
12. One or more non-transitory computer-readable media storing instructions, wherein the instructions include: instructions which, when executed by one or more hardware processors, cause receiving, at a database system, a query comprising a multi-way join comprising a plurality of join operations, each join operation of said plurality of join operations involving a respective first join key of a respective first table and a respective second join key of a respective second table; wherein the plurality of join operations includes a first join operation, a second join operation, and a third operation; instructions which, when executed by one or more hardware processors, cause for the first join operation: partitioning the respective first table of the first join operation into a plurality of partitions based on said the respective first join key of said first join operation; generating a first partition map that maps said plurality of partitions to values contained in a respective first join key of the second join operation, said respective first join key of the second join operation being a column of the respective first table of said first join operation; partitioning the respective second table of the first join operation into said plurality of partitions based on said the respective second join key of said first join operation; generating a second partition map that maps said plurality of partitions to values contained in a respective first join key of the third join operation, said respective first join key of the third join operation being a column of the respective second table of said first join operation; instructions which, when executed by one or more hardware processors, cause partitioning the respective second table of said second join operation based on the first partition map and the respective second join key of said second join operation; instructions which, when executed by one or more hardware processors, cause partitioning the respective second table of the third join operation based on the second partition map and the respective second join key of the third join operation.
13. The one or more non-transitory computer-readable media of claim 12 , wherein partitioning the respective first table and the respective second table of the first join operation further comprises: applying a hash function to the respective first join key; and applying a hash function to the respective second join key.
14. The one or more non-transitory computer-readable media of claim 12 , the instructions further including: instructions, which when executed by one or more hardware processors, cause generating a third partition map that maps values contained in a respective second join key of the third join operation, said respective second join key of said third join operation being a column of the respective second table of said second join operation; instructions, which when executed by one or more hardware processors, cause partitioning the respective second table of the third join operation based on the second partition map and the third partition map.
15. The one or more non-transitory computer-readable media of claim 14 , the instructions further including: instructions which, when executed by one or more hardware processors, cause filtering one or more rows of the respective second table of the third join operation based on the second partition map and the third partition map.
16. The one or more non-transitory computer-readable media of claim 12 further including instructions which, when executed by one or more hardware processors, cause storing the plurality of partitions in local memory of the database system.
17. The one or more non-transitory computer-readable media of claim 16 further including instructions which, when executed by one or more hardware processors, cause storing a particular partition in a disk if the local memory of the database system cannot store the entire partition.
18. The one or more non-transitory computer-readable media of claim 12 , wherein the database system is a multi-node database system, the instructions further including: instructions which, when executed by one or more hardware processors, cause assigning rows of the first respective table and the second respective table of the first join operation to one or more nodes the multi-node database system; instructions which, when executed by one or more hardware processors, cause based on the assignment, generating a distribution map; instructions which, when executed by one or more hardware processors, cause based on the distribution map, assigning rows of the first respective table and the second respective table of the second join operation and the third join operation to the one or more nodes in the multi-node database system; and instructions which, when executed by one or more hardware processors, cause each of the one or more nodes performing the partitioning and generating for tables assigned to the node.
19. The one or more non-transitory computer-readable media of claim 18 further including instructions which, when executed by one or more hardware processors, cause storing the plurality of partitions in respective local memory the one or more nodes.
20. The one or more non-transitory computer-readable media of claim 19 further including instructions which, when executed by one or more hardware processors, cause storing a particular partition in a shared disk if the respective local memory cannot store the entire partition.
21. The one or more non-transitory computer-readable media of claim 12 further comprising instructions which, when executed by one or more hardware processors, cause determining, based on the plurality of partitions, results for the multi-way join.
22. The one or more non-transitory computer-readable media of claim 21 wherein determining results for the multi-way join further comprises: instructions which, when executed by one or more hardware processors, cause generating a first hash table based on the respective second table for the first join operation and the respective second join key of the first join operation; instructions which, when executed by one or more hardware processors, cause generating a second hash table based on the respective first table of the second join operation and the respective first join key of the second join operation; wherein determining results is based on the first hash table and the second hash table.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
September 15, 2016
August 4, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.