Workload Compression Using Large Language Models

A fundamental prerequisite to understanding database system usage is an in-depth analysis of workloads whose database language text, execution plans, and resource usage metrics can be collected with a query logging mechanism. Although such analyses can be attempted by a human database administrator (“DBA”), many database vendors offer automated services or tools that assist with this complex task. One prominent category are tools that analyze workloads and make tuning recommendations, but unfortunately, they often do not scale to very large workloads typical of enterprise customers. In addition, automated tuning tasks operating within cloud multi-tenant services may be responsible for analyzing thousands of individual workloads. The designers of such workload analysis tools rightly concluded that an effective solution to this scalability challenge was to first compress the workload(s) into smaller query sets whose characteristics were still representative of the whole.

Another popular use case for workload compression is the generation of benchmarks that can reliably predict the performance of a much larger workload. Such benchmarks are useful when conducting proof of concepts (“POCs”) for large customer workloads and for development operations regression testing. Unlike industry standard benchmarks such as TPC-H that are manually designed and generated once, this use case often requires a benchmark to be generated for each customer or test suite.

Existing methods for performing workload compression include variants of K-Medoid clustering with specially designed distance functions for queries and machine learning models that vectorize and plot queries. These methods often require adjustments or retraining for individual workloads and suffer from scalability issues themselves, which in turn defeats their original purpose. In contrast, the clustering capabilities offered by modern large language models (“LLMs”) offer dramatic improvements in general applicability and efficiency as evidenced in their growing popularity with natural language processing (“NLP”) tasks such as topic modeling.

While LLMs can provide a powerful tool in database analytics, some limitations remain. LLMs must be trained in a manner that allows them to accurately process information give. This also requires that the information to be processed be structured in a manner understandable by an LLM. Moreover, LLMs have token limits that requires judicious selection of inputs that may reduce the LLMs capacity for precision.

Because traditional LLMs are limited by input limitations, it would be desirable to design inputs to allow an LLM to have a more expansive input set to make workload compression decisions.

According to one aspect of the disclosure, a system may include a storage device. The system may further include a plurality of processing nodes in communication with the storage device. At least one processing node of the plurality of processing nodes may receive query history. The query history may contain information on a plurality of executed queries. The at least one processing node may generate, with a large language model (“LLM”), a document for each query describing the information associated with a respective query. The at least one processing node may divide generated documents into a plurality of groups. The at least one processing node may compress the documents to reduce a number of documents in each group until a predetermined threshold is reached. The at least one processing node may store a final compressed set of workload documents. The at least one processing node may perform workload analysis using workloads described in the final compressed set of workload documents.

According to another aspect of the disclosure, a method may include receiving, with a processor, query history. The query history may contain information on a plurality of executed queries. The method may further include generating, with the processor, with an LLM, a document for each query describing the information associated with a respective query. The method may further include dividing, with the processor, generated documents into a plurality of groups. The method may further include compressing, with the processor, the documents to reduce a number of documents in each group until a predetermined threshold is reached. The method may further include storing, with the processor, a final compressed set of workload documents. The method may further include performing, with the processor, workload analysis using workloads described in the final compressed set of workload documents.

According to another aspect of the disclosure, a computer-readable medium may be encoded with a plurality of instructions executable by a processor. The plurality of instructions may include instructions to receive query history. The query history may contain information on a plurality of executed queries. The plurality of instructions may include instructions to generate, with an LLM, a document for each query describing the information associated with a respective query. The plurality of instructions may include instructions to divide generated documents into a plurality of groups. The plurality of instructions may include instructions to compress the documents to reduce a number of documents in each group until a predetermined threshold is reached. The plurality of instructions may include instructions to store a final compressed set of workload documents. The plurality of instructions may include instructions to perform workload analysis using workloads described in the final compressed set of workload documents.

1 FIG. 1 FIG. 100 100 102 102 102 104 104 104 104 106 104 104 106 108 108 106 is a block diagram of an example analytic environment. In one example, the analytic environmentmay include an analytic platform (“AP”), such as Teradata Vantage. The analytic platformmay include one or more systems that may be used independently or with one another in conducting advanced analytics. The analytic platformmay include a relational database management system (“RDBMS”). In one example, the RDBMSmay implement a parallel-processing environment to conduct database management. The RDBMSmay be a combination of software (e.g., computer program routines, subroutines, applications, etc.) and hardware (e.g., processors, memory, etc.). In the example of, the RDBMSmay be a massively parallel processing (MPP) system having a number of processing nodes. In alternative examples, the RDBMSmay implement a single processing node, such as in a symmetric multiprocessing (SMP) system configuration. The RDBMSmay include one or more processing nodesused to manage the storage, retrieval, and manipulation of data in data storage facilities (DSFs). The DSFsmay represent various types of storage, such as persistent and/or non-persistent, for example. The processing nodesmay manage the storage, retrieval, and manipulation of data included in a database.

100 110 102 112 110 110 114 116 112 112 110 The analytic environmentmay include a client devicethat communicates with the analytic platformvia a network. The client devicemay represent one or more devices, such as a graphical user interface (“GUI”), that allows user input to be received. The client devicemay include one or more processorsand memory(ies). The networkmay be wired, wireless, or some combination thereof. The networkmay be a cloud-based environment, virtual private network, web-based, directly-connected, and/or some other suitable network configuration. In one example, the client devicemay run a dynamic workload manager (DWM) client (not shown).

100 118 118 120 118 102 The analytic environmentmay also include additional resources. Additional resourcesmay include processing resources (“PR”). In a cloud-based network environment, the additional resourcesmay represent additional processing resources that allow the analytic platformto expand and contract processing capabilities as needed.

2 FIG. 106 200 202 202 200 is an example of a processing node, which may include one or more physical processorsand memory (ies). Memory (ies)may include one or more memories and may be computer-readable storage media or memories, such as a cache, buffer, random access memory (RAM), removable media, hard drive, flash drive or other computer-readable storage media. Computer-readable storage media may include various types of volatile and nonvolatile storage media. Various processing techniques may be implemented by the processorssuch as multiprocessing, multitasking, parallel processing, and the like, for example.

106 204 206 204 206 202 200 202 202 200 206 The processing nodesmay include one or more other processing unit types such as parsing engine (PE) modulesand access modules (AM). As described herein, each module, such as the parsing engine modulesand access modules, may be hardware or a combination of hardware and software. For example, each module may include an application specific integrated circuit (ASIC), a Field Programmable Gate Array (FPGA), a circuit, a digital logic circuit, an analog circuit, a combination of discrete circuits, gates, or any other type of hardware or combination thereof. Alternatively, or in addition, each module may include memory hardware, such as a portion of the memory, for example, which includes instructions executable with the processoror other processor to implement one or more of the features of the module. When any one of the modules includes the portion of the memorythat comprises instructions executable with the processor, the module may or may not include the processor. In some examples, each module may just be the portion of the memoryor other physical memory that comprises instructions executable with the processoror other processor to implement the features of the corresponding module without the module including any other hardware. Because each module includes at least some hardware even when the included hardware comprises software, each module may be interchangeably referred to as a hardware module, such as the parsing engine hardware module or the access hardware module. The access modulesmay be access modules processors (AMPs), such as those implemented in the Teradata Vantage analytic platform, for example.

204 206 204 206 106 204 206 106 200 106 1 2 FIGS.- The parsing engine modulesand the access modulesmay each be virtual processors (vprocs) and/or physical processors. In the case of virtual processors, the parsing engine modulesand access modulesmay be executed by one or more physical processors, such as those that may be included in the processing nodes. For example, in, each parsing engine moduleand access moduleis associated with a respective processing nodeand may each be executed as one or more virtual processors by physical processorsincluded in the respective processing node.

2 FIG. 106 204 206 204 206 106 200 106 204 206 In, each processing nodeis shown as including multiple parsing engine modulesand access modules, such that there are more parsing engine modulesand access modulesthan processing nodes. In one example, during operation, the one or more physical processorsincluded in the processing nodesmay execute the parsing engine modulesand access modulesby switching between the executions of the various modules at a rapid rate allowing the vprocs to substantially operate in “parallel.”

104 122 108 122 108 108 206 The RDBMSstores datain one or more tables (or other data object formats) in the DSFs. In one example, the datamay represent rows of stored tables that are distributed across the DSFsand in accordance with their primary index. The primary index defines the columns of the rows that are used for calculating a hash value. The function that produces the hash value from the values in the columns specified by the primary index is called the hash function. Some portion, possibly the entirety, of the hash value is designated a “hash bucket.” The hash buckets are assigned to DSFsand associated access modulesby a hash bucket map. The characteristics of the columns chosen for the primary index determine how evenly the rows are distributed.

108 204 204 108 108 104 112 Rows of each stored table may be stored across multiple DSFs. Each parsing engine modulemay organize the storage of data and the distribution of table rows. The parsing engine modulesmay also coordinate the retrieval of data from the DSFsin response to queries received, such as those received from a client systemconnected to the RDBMSthrough connection with a network.

204 208 208 204 204 106 204 300 208 210 204 2 FIG. 3 4 FIGS.and 3 FIG. 2 FIG. Each parsing engine module, upon receiving an incoming database query may apply an optimizer moduleto assess the best plan for execution of the query. An example of an optimizer moduleis shown inwith regard to a parsing engine module. Additional description of the parsing engine modulesis provided with regard to. Selecting the optimal query-execution plan may include, among other things, identifying which of the processing nodesare involved in executing the query and which database tables are involved in the query, as well as choosing which data-manipulation techniques will serve best in satisfying the conditions of the query. To this end, for each parsing engine module, a parser module(see), and/or optimizer modulemay access a data dictionary module, shown inspecifically for parsing engine modulefor purposes of illustration.

210 104 104 210 104 104 210 108 The data dictionary module, which may reside in the RDBMS, may specify the organization, contents, and conventions of one or more databases, such as the names and descriptions of various tables maintained by the RDBMSas well as fields/columns of each database, for example. Further, the data dictionary modulemay specify the type, length, and/or other various characteristics of the stored tables. The RDBMStypically receives queries in a standard format, such as the structured query language (SQL) put forth by the American National Standards Institute (ANSI). However, other languages and techniques, such as contextual query language (CQL), data mining extensions (DMX), and multidimensional expressions (MDX), graph queries, analytical queries, machine learning (ML), large language modes (LLM) and artificial intelligence (AI), for example, may be implemented in the RDBMSseparately or in conjunction with SQL. The data dictionarymay be stored in the DSFsor some other storage device and selectively accessed.

104 212 104 106 212 104 212 208 208 212 214 206 206 2 FIG. 2 FIG. The RDBMSmay include a workload management system workload management (WM) module, which may be executed within the RDBMSby one or more processing nodes. The WM modulemay be implemented as a “closed-loop” system management (CLSM) architecture capable of satisfying a set of workload-specific goals. In other words, the RDBMSis a goal-oriented workload management system capable of supporting complex workloads and capable of self-adjusting to various types of workloads. The WM modulemay communicate with each optimizer module, as shown in, and is adapted to convey a confidence threshold parameter and associated parameters to the optimizer modulein communication. Further, the WM modulemay communicate with a dispatcher moduleof each parsing engine module(as shown in detail infor parsing engine module) to receive query execution plan costs therefrom, and to facilitate query exception monitoring and automated modifications of confidence threshold parameters in accordance with disclosed embodiments.

212 212 208 The WM moduleoperation has four major phases: 1) assigning a set of incoming request characteristics to workload groups, assigning the workload groups to priority classes, and assigning goals (referred to as Service Level Goals or SLGs) to the workload groups; 2) monitoring the execution of the workload groups against their goals; 3) regulating (e.g. adjusting and managing) the workload flow and priorities to achieve the SLGs; and 4) correlating the results of the workload and taking action to improve performance. In accordance with disclosed embodiments, the WM moduleis adapted to facilitate control of the optimizer modulepursuit of robustness with regard to workloads or queries.

106 106 204 106 204 206 106 206 106 106 An interconnection (not shown) allows communication to occur within and between each processing node. For example, implementation of the interconnection provides media within and between each processing nodeallowing communication among the various processing units. Such communication among the processing units may include communication between parsing engine modulesassociated with the same or different processing nodes, as well as communication between the parsing engine modulesand the access modulesassociated with the same or different processing nodes. Through the interconnection, the access modulesmay also communicate with one another within the same associated processing nodeor other processing nodes.

106 106 202 200 106 106 106 The interconnection may be hardware, software, or some combination thereof. In instances of at least a partial-hardware implementation the interconnection, the hardware may exist separately from any hardware (e.g., processors, memory, physical wires, etc.) included in the processing nodesor may use hardware common to the processing nodes. In instances of at least a partial-software implementation of the interconnection, the software may be stored and executed on one or more of the memoriesand processorsof the processing nodesor may be stored and executed on separate memories and processors that are in communication with the processing nodes. In one example, the interconnection may include multi-channel media such that if one channel ceases to properly function, another channel may be used. Additionally, or alternatively, more than one channel may also allow distributed communication to reduce the possibility of an undesired level of communication congestion among processing nodes.

206 302 300 214 300 302 110 300 3 FIG. In one example system, each parsing engine moduleincludes three primary components: a session control module, a parser module, and the dispatcher moduleas shown in. The session control moduleprovides the logon and logoff functions. It accepts a request for authorization to access the database, verifies it, and then either allows or disallows the access. Once the session control moduleallows a session to begin, a SQL request may be received such as through submission the client deviceand the SQL request is routed to the parser module.

4 FIG. 300 400 300 402 404 302 406 206 208 214 208 206 As illustrated in, the parser modulemay include an interpreter modulethat interprets the SQL request. The parser modulemay also include a syntax checker modulethat checks the request for correct SQL syntax, as well as a semantic checker modulethat evaluates the request semantically. The parser modulemay additionally include a data dictionary checkerto ensure that all of the objects specified in the SQL request exist and that the user has the authority to perform the request. The parsing engine moduleimplements the optimizer moduleto select the least expensive plan to perform the request, and the dispatchercoordinates the runtime execution of executable steps of the query execution plan of the optimizer modulewith the access modules.

212 214 206 214 212 208 In one example, to facilitate implementations of automated adaptive query execution strategies, such as the examples described herein, the WM modulemonitoring takes place by communicating with the dispatcher moduleas it checks the query execution step responses from the access modules. The step responses include the actual cost information, which the dispatcher modulemay then communicate to the WM modulewhich, in turn, compares the actual cost information with the estimated costs of the optimizer module.

102 500 500 502 503 504 506 508 500 504 506 508 510 5 FIG. 5 FIG. 5 FIG. 5 FIG. AI-driven techniques may be implemented in the analytic platformallowing more advanced workload compression techniques to be handled by large language models (“LLMs”).is an example of workload compression to illustrate how an LLM may be used to conduct compression for purposes of allowing various workload-related analyses to be performed downstream. In the example of, a query setmay contain queries Q1 through Q22. One technique in compressing workloads may include clustering queries based on selected characteristics. Queries can be related in a number of manners, what may include, but not be limited to, data (e.g., tables) being accessed, query source, storage type, and query type. In the example of, queries from the query setmay undergo clusteringbased on some predetermined selected characteristic(s), which generates clustering resultsthat includes three query clusters,, and. Queries Q2, Q8, and Q13 are outliers based on the selected characteristic(s) with respect to the query set. The clusters,, andmay undergo compressionin that a query representative of a respective cluster may be selected for analysis. In the example of, queries Q7, Q11, and Q21 are selected as a representative query from their respective clusters. Such compression allows analyses to be performed without the need to consume massive amounts of query data, which typical systems cannot handle to scale.

6 FIG. 600 104 602 104 602 104 604 605 604 Using these compression techniques, LLMs may be implemented in a manner that can execute compression allowing more precise results to be generated enhancing downstream analyses and applications.is an example of a systemthat may be executed in an analytic environment that includes a database management system, such as RDBMS, allowing LLMs to handle workload compression. In one example, customer workloadsmay be executed by the RDBMS, where each customer workloadmay include one or more queries. The RDBMSmay log received queries in which query telemetryis collected in query logs. The query telemetrymay include various types of metadata associated with a query.

604 604 208 604 608 604 608 606 604 606 610 604 606 As described, LLMs may be used in the handling of workload compression. However, in order for an LLM to be able to handle workload compression, the LLM must receive query information in manner that can be processed by the LLM. For example, information in natural language format allows modern LLMs to process the information. Query telemetryis raw database data, which may reside in different formats and storage types. The query telemetrymay also provide additional raw data beyond the query syntax itself, such as the execution plan chosen by the optimizerand runtime metrics. The query telemetryneeds to be converted to a text-based representation allowing an LLM to perform workload compression. In one example, a workload telemetry preprocessor modulemay generate text-based documents based on the query telemetry. Specifically, the workload telemetry preprocessor modulemay generate a workload documentfor each received query based on the query telemetry, with the workload documentsmaintained in a workload document repository. Table 1 below is an example of categories that the telemetrymay be organized into and used as standard subsections in each workload documentto facility natural language processing (“NLP”) by an LLM.

TABLE 1 Categories of Query Metadata Document Subsections Description/Purpose SQL text Statement type, objects referenced, analytic functions, and filtering conditions Plan text (EXPLAIN) Natural language summary of execution plan steps, physical operators, and access methods Scheduling WLM assigned priority tier and selected compute cluster types Resource Usage Runtime metrics including CPU, I/O, memory, spool, and network Latency Response time and Service Level Goal adherence Object metadata Critical portions of object definitions needed for query clustering

7 FIG. 700 702 700 700 210 606 Referring to, an example of workload documents are presented to illustrate how the workload documents may be structured. In one example, workload documentsandillustrate the types of information that may be used in a workload document. Regarding workload document, a query record (“QUERY ID”) may be identified. The workload documentmay also contain a SQL text category of the query and plan text (via the EXPLAIN feature) category. The SQL and EXPLAIN categories provide the names of referenced objects (e.g., tables and columns), which is generally sufficient for query clustering without having to include detailed schema definitions. If included in its entirety, the extra content would bloat the size of the workload documents and exacerbate the limits on LLM context windows. Workload analysis operations running after compression may require such metadata but can retrieve it from the data dictionaryas needed. Even so, this solution recognizes that certain portions of object metadata may be relevant to clustering and reserves a category (“accessed storage”) for them. A prominent example is the underlying storage types of tables referenced (e.g., block storage, open table format, parquet files), with each type supporting dramatically different access methods and performance profiles. While EXPLAIN text may differ somewhat for steps accessing different storage types, a separate subsection in each workload documentis included to emphasize its importance in deciding how to cluster similar queries.

700 702 700 As shown in workload document, categories for “scheduling”, “resource usage” and “latency” may also be included. Workload documentincludes the same subsection categories as workload document, but contains different information regarding the associated query (query id=8675320)

6 FIG. 8 FIG. 8 FIG. 8 FIG. 606 612 614 606 606 616 612 606 800 612 800 802 612 800 804 804 612 606 606 612 800 800 808 800 612 700 702 800 612 800 806 606 Returning to, once the workload documentshave been generated, they may be provided to a core LLM, which may include a classification and compression module, which both classifies and compresses workloads using the workload documents. In order to assist in the understanding the workload documents, prompt templatesmay be used to provide guidance to the LLM, both with respect to the content of each workload document and comparison of the content across the workload documents.is an example of a prompt templatethat may be provided to the core LLM. As shown in, prompt templatemay include a context sectiongiving context of the task to the LLM. The context section may include not only context to databases, also to specific systems (e.g., Teradata VantageCloud). The prompt templatemay also include an instructions section. The instructions sectionprovides instructions to the LLMon what to do with the workload documents. In particular, the instructions provide the goal of the LLM, as well as describe the structure of the workload documents. In order to further assist the LLM, the prompt templatemay include examples of workload documents describing queries with similar characteristics and queries having different characteristics. In the example prompt template, sample workload documentsare included in the prompt templatehaving similarities such that LLMmay understand queries of this degree of similarity are considered “highly similar”. In the example of, the workload documentsandare used as two queries considered to be “different”. The prompt templatemay be tailored to include as many sample workload documents within the limits of input of the context window of the LLM. Lastly, the prompt templatemay include an input section. The input section may include all of the workload documentsrepresenting queries to be considered in workload compression.

800 612 606 606 606 612 606 900 900 606 606 505 900 902 612 800 904 906 904 906 908 910 606 606 904 904 612 908 914 916 9 FIG. Using the prompt template, the LLMmay perform workload compression using the available workload documents. Each LLM has a limit on the number of tokens that it can receive as input. Thus, if the number of total workload documentsexceeds this limit, the number of workloadsmay be broken up to allow the LLMto perform the workload compression. An example of this technique is shown in. In one example, the workload documentsmay be broken into n chunkswith each chunkrepresenting a number T of the workload documents. The number T represents a minimum threshold of information to provide meaningful query clustering for the compression. In scenarios in which the combined size of T workload documentsis too large, the SQL and EXPLAIN text portions of the workload documentsmay be truncated, as necessary. Each chunkmay undergo compressionby the core LLMin accordance with the guidance provided by the prompt templatesresulting in a compressed chunk. A compression evaluationmay be performed on the compressed chunks. The evaluationmay include an additional compression determinationto determine if additional compression is needed based on the intermediate chunks being too large. If additional compression is needed, redistributionof the workloadsmay be performed by randomly redistributing the workloadsof the compressed chunksinto n chunks. This process may occur iteratively until the size of the chunksare small enough to be collectively handled by the LLM. Once additional compression is no longer needed (determination), the final compression resultsthat includes final compressed chunks.

6 FIG. 6 FIG. 914 618 102 620 622 110 620 914 624 624 626 110 628 110 104 630 630 626 104 Referring back to, once the final compressed workloadsare generated, they may be retained in a compressed workload repository. With the compressed workloads a number of analyses may be performed as illustrated in. In one example, the analytic platformmay include a chat-bot LLMallowing follow-on conversationsto be performed between a user via client deviceand the chat-bot LLM. The compressed workloadsmay be used to perform workload analysis. The workload analysismay provide tuning recommendationsto the user via the client device, which may be applied as tuning actionsby a user via client deviceto the RDBMS. In an autonomous mode, tuning actionsbased on the recommendationsmay be automatically applied to the RDBMS.

104 914 632 634 208 104 634 208 The RDBMSmay also implement the compressed workloadsfor index selectionto execute “what-if” simulationsusing the optimizer(not shown) of the RDBMSand dictionary index definitionsto recommend indices and materialized views for the purpose of improving performance which is generally referred to as “index selection”. An established and accurate method for performing index selection involves a series of “what-if” simulations on the potentially large space of alternative index configurations. Each simulation requires calling the optimizerto estimate the benefit of the candidate index configuration for each retrieval query in the workload and the added maintenance overhead for each update statement. More recent solutions for index selection involve the use of deep learning that require the use of a reward function to observe the benefit from applying a given tuning action. In the case of uncompressed large workloads, the time required to compute the reward would make this trial-and-error learning approach impractical.

10 FIG. 6 FIG. 1000 104 1002 110 1000 1004 1002 1003 620 620 1004 is an example of a cloud-service-based version of a workload compressor as seen inusing an LLM. In one example, a database workload servicemay act as a cloud-based service independent of the RDBMSoperating over cloud network. In operation, a user via client devicemay access the features of the database workload servicevia an application programming interface (“API”) layerthrough cloud-networkwith an API call. Alternatively, the chat-bot LLMmay be used to allow user interactions, however the chat-bot LLMoperates through the API layer.

110 1000 1006 605 604 608 606 1008 1008 1010 1002 612 622 110 9 FIG. In operation, the client devicemay initiate the database workload service, which may retrieve workload telemetry, which may include query logscontaining individual query telemetryincluded in various workloads. The workload telemetry preprocessor modulemay generate the workload documentsthat may be compressed by the workload compressor. In one example, the workload compressormay implement a third-party core LLM servicevia cloud network, similar in functionality to the core LLMand may handle the compression of the workloads via the techniques described with regard to. Workload analysis toolsmay be used in either a manual fashion via the client deviceor through autonomous techniques.

11 FIG. 1100 604 608 1102 606 604 1104 606 900 606 1106 900 1108 1110 904 1112 1108 1110 914 1114 915 1116 is an operational flow diagramof example operation of workload compression via an LLM. In one example, query telemetrymay be retrieved by the workload telemetry preprocessor(). A workload documentmay be generated for each query described in the query telemetry(). The workload documentsmay be divided into n chunkswith each chunk containing at least T workload documents(). Each chunkmay be compressed (). If additional compression is needed (), the compressed chunksmay be redistributed () and again compressed (). Once additional compression is no longer needed (), the compressed workload setmay be stored for analysis (). The compressed workload setmay be used for workload analysis ().

While various embodiments of the disclosure have been described, it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible within the scope of the disclosure. Accordingly, the disclosure is not to be restricted except in light of the attached claims and their equivalents.