Query Engine for Graph Databases and Heterogeneous Hardware

This application is a continuation of U.S. Patent Application No. 18/762,401, entitled QUERY ENGINE FOR GRAPH DATABASES AND HETEROGENEOUS HARDWARE filed July 02, 2024 which is incorporated herein by reference for all purposes, which claims priority to U.S. Provisional Application No. 63/524,827, entitled QUERY ENGINE FOR GRAPH DATABASES AND HETEROGENEOUS HARDWARE filed July 03, 2023 which is incorporated herein by reference for all purposes.

Typically, data stored in a database is accessed via a client or web-based application. A user enters a query expressed using a query language, such as Structure Query Language (SQL) or other languages used to access and/or manage relational databases, or Cypher™, Graph Query Language (GQL), or other languages used to access graph databases.

Queries expressed using such languages typically are sent to a database management/access server, which parses the query as expressed using the query language and develops and executes a plan to perform the query and return results.

The invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention. Unless stated otherwise, a component such as a processor or a memory described as being configured to perform a task may be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task. As used herein, the term ‘processor’ refers to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.

Techniques are disclosed to apply advances in programming language capabilities to provide an improved database management system. In various embodiments, a novel query language runtime based on modern programming languages techniques is provided. In various embodiments, approaches disclosed herein match or even outperform traditional query planner/optimizer systems while enabling ease of query language evolution and seamless execution on a variety of hardware, e.g., Central Processing Unit (CPU), General-Purpose Graphics Processing Unit (GPGPU), and others.

In various embodiments, techniques from the managed programming languages community are used to create a database runtime system whose components are structured to form a dynamic optimization framework for query execution. A low-latency Just-In-Time (JIT) compilation framework is provided that can: (i) adapt to changes in database schema or the topology of the underlying graph as they occur; and (ii) make efficient use of heterogeneous hardware like GPGPUs. In various embodiments, these advances provide a significant increase in performance with lower system complexity. They also, when coupled with schema-optional databases in various embodiments, lessen the need to perform up-front schema design on performance grounds since the system would not require schemas as an input into a query planner (since there is no query planner).

In various embodiments, a runtime system as disclosed herein is decoupled from the query language. This decoupling is done, in some embodiments, by compiling each query into an executable Intermediate Representation (IR) which is executed by the runtime system, much like how bytecode is executed by the Jave Virtual Machine (JVM). This design enables a runtime system as disclosed herein to execute queries from different query languages simultaneously if they have a valid compiler to generate the required IR. This architectural change helps to lower the cost of implementing and experimenting with query languages: a situation that is prevalent in the graph database community where implementers are trying to keep up with changes to graph query languages, such as openCypher and the ISO standard GQL query language.

1 FIG. 1 FIG. 100 102 104 102 104 106 108 102 104 110 112 112 114 102 104 106 108 is a block diagram illustrating an embodiment of a database management system comprising a compiled query-based query engine. In the example shown, systemincludes a plurality of client computers/devices, represented inby client computers,. Each of the client computers,has a respective query compiler,configured to generate a byte code or other intermediate representation of a received query, as disclosed hereon. Each of the client computers,is connected via one or more networks, e.g., the Internet, a corporate or other enterprise network, etc. to a database server. Database serverincludes one or more virtual machinesconfigured to execute queries received in byte code form from client computers,and/or query compilers,.

114 112 112 116 In various embodiments, each virtual machinecomprises a runtime configured to execute a query based on the byte code representation of the query, as disclosed herein. In various embodiments, the byte code may be interpreted or instead compiled into machine code and executed by hardware comprising the database server. Database servermay include one or more GPGPUs configured to perform operations comprising a database query, in parallel as/if appropriate, as disclosed herein. In the example shown, each query is performed with respect to one or more databases stored in database store, e.g., one or more graphs comprising a graph database.

2 FIG. 2 FIG. 1 FIG. 1 FIG. 200 108 112 114 is a flow diagram illustrating an embodiment of a process to perform a database query. In various embodiments, the processofmay be performed in part by a query preprocessor and/or compiler, e.g., query compilers 106,ofand in part by a database server, e.g., database serverand/or virtual machineof.

202 204 206 204 206 204 In the example shown, ata query expressed in a database query language, e.g., Cypher or GQL, is received. At, the query is decomposed into a discrete set of streaming operators define over data frames. At, the operators determined in stepare encoded into optimized byte code and/or some other intermediate (e.g., runtime interpreted and/or executable) representation. In various embodiments, the encoding generated atincludes and/or is supplemented by byte code representing the query and the operators into which the query was decomposed at stepas a data flow graph describing the relationship between the operators, in terms of the data flow between them, and the data inputs and output of each, e.g., the data frames each is configured to receive as input and provide as output.

202 204 206 102 104 1 FIG. In various embodiments, steps,, andmay be performed at a first computing node, e.g., at a client device such as client computers,of.

208 206 114 210 1 FIG. At, the optimized byte code generated atis compiled (or interpreted) and executed, e.g., at a virtual machine or other runtime such as virtual machineof. At, a query result is received and returned in response to the query.

3 FIG. is a block diagram illustrating an example of query processing in a prior art database management system. In the example shown, system 300 processes an example query 302, in this case a simple example to find and count all the nodes having the label “Foo”. Query 302 is provided to a database server 304 configured to perform the query with respect to a graph or other data set in database 306. As shown at right, in the typical prior approach, a database server such as server 304 would process a query such as query 302 in a multi-step and/or multi-module process, typically running on a single machine, such as server 304.

310 312 312 314 316 314 316 314 316 316 318 Specifically, query 302 typically would be processed first by a parserwhich generates an abstract syntax tree (AST) representationof the query. ASTwould then be processed by a query planner, in this example including logical plannerand physical planner. For example, logical plannermay generate a logical plan comprising a collection of logical operators that describe the work required to generate query results and defining which data sources and operators to apply. Physical plannermay generate a physical plan which describes the chosen physical execution plan for a query statement. Logical plannerand/or physical plannermay comprise an optimization module and/or algorithm configured to apply various types of rules to rearrange operators and functions into an optimal plan. The physical plannerthen converts the optimized logical plan into a physical plan that tells the system how to execute the query, using execution enginein this example.

4 FIG. 4 FIG. 400 402 402 404 402 412 414 is a block diagram illustrating an example of query processing in an embodiment of a database management system comprising a compiled query-based query engine. In the example shown, systemis shown processing an illustrative example query, in this case the same as query 302 of. The queryis processed first by query compiler, which generates a byte code (or other intermediate) representation of the query. The byte code is sent to virtual machine, which compiles (or interprets) and executes the query with which the byte code is associated with respect to a graph or other data set in database.

404 406 402 408 402 202 204 206 408 410 412 408 412 2 FIG. As shown at right, the query compilerincludes a parserconfigured to generate, based on the queryas expressed in the query language, as shown in the figure, a byte code or other intermediate representationof the query. In various embodiments, the byte code is generated as described in connection with one or more of steps,, andof. The byte code or other intermediate representationis serialized by byte code emitterand transmitted, e.g., via a network, to the virtual machine. (In other embodiments, byte code or other intermediate representationis not serialized and is provided to the virtual machinein another form or format.

4 FIG. 412 416 418 420 418 422 Referring further to, in the example shown virtual machineincludes the following components/modules: graph metadatacomprising metadata describing one or more graphs and/or their content; just-in-time (JIT) compiler, configured to optimize and compile byte code for execution; scheduler, configured to use code compiled by JIT compilerto create, initialize, schedule and/or queue up work for, and move data frames between instances of operators defined to perform a query; and buffer managerconfigured to create, allocate, and/or otherwise manage resources such as memory.

418 416 In various embodiments, JIT compilermay use data comprising graph metadataand/or the underlying graph to optimize byte code associated with a query.

5 FIG.A 3 FIG. 4 FIG. 500 502 402 504 502 504 502 is a diagram illustrating an example of query decomposition in an embodiment of a database management system. In the exampleshown, query(the same as query 302 ofand queryof) is decomposed, e.g., by a query compiler as disclosed herein, into decomposed form, in which the queryis represented as a discrete set of streaming operators defined over data frames. The decomposed formof queryrepresents the query as a data flow, which can be conceptualized and/or represented, in various embodiments, as a graph.

504 In this example, the query is represented in decomposed formas a first operator (“op0”) that scans labels for those that match the argument “Foo” and a second operator (“op1”) that receives the outputs generated by the first operator and generates a count. The second operator is designated as the “sink” in this example, which in various embodiments identifies the second operator as the final operator in the data flow graph, such that the output of that operator is the query result. Note the text “follows=op0” specifies the relationship between the operators in the data flow.

5 FIG.B 520 522 524 526 524 0 1 2 3 is a diagram illustrating an example of query execution in an embodiment of a database management system. In the example shown, runtimehas been configured, by scheduler, to run a plurality of operators, which implement a data flow represented by graph. In various embodiments, each of the operators, comprising operators,,, andin this example, is instantiate by executing corresponding code compiled from byte code included in and/or derived (e.g., via optimization processing) from the byte code associated with a query.

522 524 530 530 522 526 522 0 1 2 5 FIG.B In various embodiments, schedulerassigns work to the operatorsby adding associated data frames, e.g., data frame 528, to their respective input (“Rx”) queues. Each operator pulls data frames from their input queue and processes the data frame according to its operator definition/code, using workers. For example, each “task” shown in a queue of a workermay comprise the task of executing with respect to a specific data frame code comprising or otherwise associated with an operator that pulled the data frame from its input queue. The result of each such task is placed in the output (“Tx”) queue of the associated operator. The work schedulermanages a flow of data frames between operators, e.g., according to the data flow graph. For example, in the example shown in, schedulermay pull data frames from the output queue of Operatoror the output queue of Operatorand place them in the input queue of Operator.

6 FIG. 6 FIG. 5 FIG.B 5 FIG.B 600 522 602 526 is a flow diagram illustrating an embodiment of a process to perform dynamically adaptive query execution. In various embodiments, the processofmay be performed by a scheduler or other component comprising and/or executing in or on a runtime, such as schedulerof. In the example shown, at, a data flow graph associated with a query is received, e.g., data flow graphof. In some embodiments, the graph may be inferred or otherwise determined based on data comprising or otherwise associated with a received byte code representation of a query.

604 606 608 610 612 604 606 608 614 600 At, operators comprising or otherwise associated with the query and/or data flow graph are instantiated. At, work is assigned to the respective operators, e.g., by placing in each operator’s queue one or more data frames comprising data read or otherwise derived from the graph or other data set to which the query relates and/or a data frame pulled from the output queue of an operator that precedes the operator in the data flow graph. At, as processing continues, data frames are moved between the output and input queues of operators that follow one another in the data flow graph. If back pressure is detected, at, e.g., an operator’s input queue is full or an operator from which a data frame is to be received has an empty output queue, then atresources adjustments may be made. For example, additional instances of an operator may be spun up and run in parallel and/or additional processing, memory, and/or other resources may be allocated to an operator, after which processing resumes/continues,,. Once all data frames have been processed, at, the processends.

7 FIG. 700 702 704 706 708 710 706 710 712 706 714 714 710 710 704 is a functional flow block diagram illustrating an embodiment of a system configured to perform a query. In the example shown, systemincludes query parserconfigured to parse queries, such as query, to produce a high-level intermediate representation (HIR). A high-level intermediate representation (HIR) to mid-level intermediate representation (MIR) translation moduleproduces a mid-level intermediate representationbased on the high-level intermediate representation. The mid-level intermediate representationis checked against and/or, if not already presented, stored in a query cache, along with a copy of the HIRand associated metadata. In various embodiments, the metadataincludes a fingerprint, such as a hash or other data that uniquely identifies the MIR, to enable the cached MIRto be identified and reused, e.g., to process a subsequent query that is the same as or similar to (e.g., same operations but different target and/or arguments) query.

710 716 702 708 712 716 7 FIG. The MIRis processed next by an MIR interpretation module. In some embodiments, query parser, HIR to MIR translation module, and query cachemay be present on one machine, e.g., a client computer, while the MIR interpretation moduleand subsequent components shown inare present on one or more other machines, such as one or more database servers.

716 718 718 In the example shown, MIR interpretation moduleperforms expression self-optimization processing to generate optimized expression trees. In some embodiments, graph data and/or metadata may be used to generate the optimized expression trees.

716 718 In some embodiments, the MIR interpretation moduleprocesses graph data in a data type agnostic manner and/or discovers data types only as/if and/or when needed to generate the expression trees. A graph database typically does not have or require a schema that specifies each entity stored in the database and its data type. As a result, the ability of a system as disclosed herein to operate in a data type agnostic manner and/or to discover data types in connection with optimization processing makes the techniques described herein advantageous in processing queries with respect to a graph database.

718 720 722 718 724 The expression treesare provided to low-level intermediate representation assembler, which generates low-level virtual machine (LLVM), i.e., runtime executable representationof the expression trees, which are in turn compiled (or interpreted) to generate and execute associated machine code.

8 FIG. 0 802 2 3 812 2 806 816 2 802 802 2 812 is a diagram illustrating an example of distributed query execution in an embodiment of a database management system. In the example shown, the operators are instantiated on two physically separated machines with "Operator" on machineand with "Operator" and "Operator" on machine. Each machine has a copy (full or partial) of the dataflow execution graph in its "L" scheduler&. A proxy operator "Operatorproxy" is instantiated on machinethat signifies that dataframes arriving in its "Rx" queue on machineshould be forwarded to the "Rx" queue on "Operator" on machine. Dataframes can be forwarded either directly by the proxy operator or indirectly via a scheduling framework. The proxy operator allows the dataflow graph to be partitioned across multiple machines so that operators can act independently whilst also preserving the execution semantics of the query.

818 802 812 0 802 1 804 2 806 2 812 818 812 814 816 2 2 3 3 3 A communication channelconfigured to communicate data frames between the first machineand the second machine. In the example shown, data frames are transferred via gRPC, a remote procedure call framework, but other transport mechanisms may be used. Specifically, in the example shown, data frames on the output (“Tx”) queue of Operatoron the first machineare passed by the Lschedulerto “OperatorProxy”, based on the data flow knowledge held by the higher-level scheduler. The OperatorProxy passes the data frames directly or indirectly via a scheduling framework (i.e. the proxy can be hardcoded to transfer only to machineor a scheduling framework can select the best machine from a group of machines based on some desirable criteria such as load) onto communication channelto the second machine, where the schedulers,place them in the input queue (“Rx”) of the “Operator”. Data frames in the output queue (“Tx”) of Operatorare placed in the input queue of Operator, as described above in the non-distributed example, and in this example Operatoris the “sink”, so the data frames in the output queue (“Tx”) of Operatorare used to provide a result to the query, as described above.

In various embodiments, processing may be distributed across many cores and/or physical machines, including across networks, to scale the computing capacity as needed to maximize performance, including through parallel processing, such as may be determined to be permitted and/or required based on the graph representation of the data flow determined for the query.

In various embodiments, resources may be added or removed dynamically, e.g., to achieve performance goals or requirements and/or as early nodes (operators) in the data flow complete their work and/or later nodes (operators) are ready to begin their work.

In various embodiments, techniques disclosed herein enable memory use to be maintained at a relatively low and consistent level, e.g., by managing the number of operator instances and the size of their respective input and output queues. Data is read from the graph selectively and as needed to provide data frames to the input queues of operators that operate on data read directly from the graph, rather than reading an entire graph or large portion thereof into memory prior to processing the query and/or without the requirement or practice of keeping such portions of the graph in memory longer than may be needed to perform required processing.

9 FIG.A 9 FIG.A 1 FIG. 900 106 108 is a flow diagram illustrating an embodiment of a process to retrieve or generate an intermediate representation of a query. In various embodiments, the processofis implemented by a query compiler, such as query compilers,of.

902 904 906 910 In the example shown, at, a query is received. At, a check is performed to determine whether byte code (or another intermediate representation) has already been generated and cached for the query. If byte code for the query is found to be in the cache (), a cached executable representation is used. This could be a pre-optimized interpretable expression tree or pre-compiled machine code; otherwise, an executable representation of the query is generated i.e. interpretable expression trees (or MIR) is generated for the query (), e.g., as described above.

9 FIG.B 9 FIG.B 9 FIG.A 7 FIG. 910 900 922 924 712 926 is a flow diagram illustrating an embodiment of a process to generate an intermediate representation of a query. In various embodiments, the process ofis performed to generate an executable intermediate representation of the bytecode (that transitively represents a query) for a query for which bytecode has not previously been seen in the system, e.g., as in stepof the processof. In the example shown, atan executable intermediate representation of the query is generated from the bytecode. At, this executable representation alongside the bytecode is added to a cache, e.g., query cacheof. At, a signature is generated from the bytecode for the query that uniquely identifies the query and can be used to recover both the cached bytecode and intermediate representation to facilitate the future execution or optimization of the query.

112 102 104 1 FIG. 1 FIG. 1 FIG. In some embodiments, a global or central cache may be maintained by a server, e.g., serverof, to enable a query submitted by a first client/user, e.g., clientof, to be used to speed processing of a query submitted at/by a second client/user, e.g., clientof. For example, upon receipt of a query, an associated signature may be generated and used to check a local cache. If present locally, a corresponding locally cached intermediate representation associated with the query may be used. If not present locally, a central/global cache may be checked. If present, an intermediate representation stored in the central/global cache may be used to perform the query.

10 FIG. is a flow diagram illustrating an embodiment of a process to iteratively revert to, compile, and execute a bytecode representation of a query. In various embodiments, the generation of bytecode (or another intermediate representation) of a query, as disclosed herein, provides a reliable intermediate state to which processing may revert, as/if needed, e.g., to recover from a downstream error, such as receiving an unexpected data type or null value or if a previously applied optimization is no longer valid or worthwhile.

10 FIG. 1002 1004 1006 1008 1010 1008 1012 1006 In the example shown in, at, a query is received. At, bytecode (or other intermediate representation) is generated for the query and cached. At, the bytecode is simultaneously optimized and executed, e.g., it is compiled/interpreted and executed by a virtual machine or other runtime, as described above. If the bytecode was executed successfully (), then ata query result is received and returned. If the bytecode was not executed successfully or a performance slowdown is detected (), then atthe bytecode generated for the query is retrieved (if not already at hand) and a further attempt to execute and optimize the bytecode is performed at. In various embodiments, execution may be attempted up to a prescribed number of tries, after which an error is generated and/or other exception handling occurs.

In various embodiments, techniques disclosed herein may be used to quickly and efficiently generate search results in response to a query. In various embodiments, techniques disclosed herein may be applied in contexts other than creating better graph database management systems. For example, and without limitation, in various embodiments techniques disclosed herein may be applied to provide better relational database management systems and/or systems other than database systems.

Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and not restrictive.