Managing Operational Resilience of System Assets Using an Artificial Intelligence Model

This application is a continuation-in-part of U.S. patent application Ser. No. 18/889,371 entitled “IDENTIFYING AND REMEDIATING GAPS IN ARTIFICIAL INTELLIGENCE USE CASES USING A GENERATIVE ARTIFICIAL INTELLIGENCE MODEL” filed on Sep. 18, 2024, which is a continuation-in-part of U.S. patent application Ser. No. 18/653,858 entitled “VALIDATING VECTOR CONSTRAINTS OF OUTPUTS GENERATED BY MACHINE LEARNING MODELS” filed on May 2, 2024, which is a continuation-in-part of U.S. patent application Ser. No. 18/637,362 entitled “DYNAMICALLY VALIDATING AI APPLICATIONS FOR COMPLIANCE” filed on Apr. 16, 2024.

This application is further a continuation-in-part of U.S. patent application Ser. No. 18/782,019 entitled “IDENTIFYING AND ANALYZING ACTIONS FROM VECTOR REPRESENTATIONS OF ALPHANUMERIC CHARACTERS USING A LARGE LANGUAGE MODEL” and filed Jul. 23, 2024, which is a continuation-in-part of U.S. patent application Ser. No. 18/771,876 entitled “MAPPING IDENTIFIED GAPS IN CONTROLS TO OPERATIVE STANDARDS USING A GENERATIVE ARTIFICIAL INTELLIGENCE MODEL” and filed Jul. 12, 2024, which is a continuation-in-part of U.S. patent application Ser. No. 18/661,532 entitled “DYNAMIC INPUT-SENSITIVE VALIDATION OF MACHINE LEARNING MODEL OUTPUTS AND METHODS AND SYSTEMS OF THE SAME” and filed May 10, 2024, which is a continuation-in-part of U.S. patent application Ser. No. 18/661,519 entitled “DYNAMIC, RESOURCE-SENSITIVE MODEL SELECTION AND OUTPUT GENERATION AND METHODS AND SYSTEMS OF THE SAME” and filed May 10, 2024, and is a continuation-in-part of U.S. patent application Ser. No. 18/633,293 entitled “DYNAMIC EVALUATION OF LANGUAGE MODEL PROMPTS FOR MODEL SELECTION AND OUTPUT VALIDATION AND METHODS AND SYSTEMS OF THE SAME” and filed Apr. 11, 2024. The content of the foregoing applications is incorporated herein by reference in their entirety.

Artificial intelligence (AI) models often operate based on extensive and enormous training models. The models include a multiplicity of inputs and how each should be handled. When the model receives a new input, the model produces an output based on patterns determined from the data the model was trained on. A large language model (LLM) is a language model notable for its ability to achieve general-purpose language generation and other natural language processing tasks such as classification. LLMs can be used for text generation, a form of generative AI (e.g., GenAI, Gen AI, GAI), by taking an input text and repeatedly predicting the next token or word. LLMs acquire these abilities by learning statistical relationships from text documents during a computationally intensive self-supervised and semi-supervised training process. Generative AI models, such as LLMs, are increasing in use and applicability over time.

Generally, organizations are required to adhere to compliance requirements that are set by the government and various regulatory bodies. Different forms of organizations are subject to compliance with a variety of forms of regulations from an assortment of regulatory bodies. For example, there are no standard sets of laws and regulations or definitions for each jurisdiction in which an AI model is used. Many jurisdictions require compliance with the jurisdictions' specific laws and regulations for all uses of AI within their boundaries. For AI models used across borders (e.g., country, state, and local), this creates an overlapping patchwork of regulations and laws. In many circumstances, the laws and regulations are applied differently according to the specific AI model in use in that jurisdiction. Additionally, regulations are changing and can be expected to change in sometimes significant and subtle ways. However, in parallel with the increased complexity of regulations, regulators are taking stronger actions against non-compliance by imposing large penalties and causing potential loss of reputation for non-compliant parties.

The technologies described herein will become more apparent to those skilled in the art from studying the Detailed Description in conjunction with the drawings. Implementations describing aspects of the invention are illustrated by way of example, and the same references can indicate similar elements. While the drawings depict various implementations for the purpose of illustration, those skilled in the art will recognize that alternative implementations can be employed without departing from the principles of the present technologies. Accordingly, while specific implementations are shown in the drawings, the technology is amenable to various modifications.

Pre-existing LLMs and other generative machine learning models are promising for a variety of natural language processing and generation applications. In addition to generating human-readable, verbal outputs, pre-existing systems can leverage LLMs to generate technical content, including software code, architectures, or code patches based on user prompts, such as in the case of a data analysis or software development pipeline. Based on particular model architectures and training data used to generate or tune LLMs, such models can exhibit different performance characteristics, specializations, performance behaviors, and attributes.

However, users or services of pre-existing software development systems (e.g., data pipelines for data processing and model or application development) do not have intuitive, consistent, or reliable ways to select particular LLM models and/or design associated prompts in order to solve a given problem (e.g., to generate a desired code associated with a particular software application). As such, pre-existing systems risk selection of sub-optimal (e.g., relatively inefficient and/or insecure) generative machine learning models. Furthermore, pre-existing software development systems do not control access to various system resources or models. Moreover, pre-existing development pipelines do not validate outputs of the LLMs for security breaches in a context-dependent, and flexible manner. Code generated through an LLM can contain an error or a bug that can cause system instability (e.g., through loading the incorrect dependencies). Some generated outputs can be misleading or unreliable (e.g., due to model hallucinations or obsolete training data). Additionally or alternatively, some generated data (e.g., associated with natural language text) is not associated with the same severity of security risks. As such, pre-existing software development pipelines can require manual application of rules or policies for output validation depending on the precise nature of generated output, thereby leading to inefficiencies in data processing and application development.

The data generation platform disclosed herein enables dynamic evaluation of machine learning prompts for model selection, as well as validation of the resulting outputs, in order to improve the security, reliability, and modularity of data pipelines (e.g., software development systems). The data generation platform can receive a prompt from a user (e.g., a human-readable request relating to software development, such as code generation) and determine whether the user is authenticated based on an associated authentication token (e.g., as provided concurrently with the prompt). Based on the selected model, the data generation platform can determine a set of performance metrics (and/or corresponding values) associated with processing the requested prompt via the selected model. By doing so, the data generation platform can evaluate the suitability of the selected model (e.g., LLM) for generating an output based on the received input or prompt. The data generation platform can validate and/or modify the user's prompt according to a prompt validation model. Based on the results of the prompt validation model, the data generation platform can modify the prompt such that the prompt satisfies any associated validation criteria (e.g., through the redaction of sensitive data or other details) thereby mitigating the effect of potential security breaches, inaccuracies, or adversarial manipulation associated with the user's prompt.

The selected model(s) encounter further challenges with respect to the compliance of AI models with an array of vector constraints (e.g., guidelines, regulations, standards) related to ethical or regulatory considerations, such as protections against bias, harmful language, and intellectual property (IP) rights. For example, vector constraints can include requirements that require AI applications to produce outputs that are free from bias, harmful language, and/or IP rights violations to uphold ethical standards and protect users. Traditional approaches to regulatory compliance often involve manual interpretation of regulatory texts, followed by ad-hoc efforts to align AI systems with compliance requirements. However, the manual process is subjective, lacks scalability, and is error-prone, which makes the approach increasingly unsustainable in the face of growing guidelines and the rapid prevalence of AI applications.

As such, the inventors have further developed a system (e.g., within the data generation platform) to provide a systematic and automated approach to assess and ensure adherence to guidelines (e.g., preventing bias, harmful language, IP violations). The disclosed technology addresses the complexities of compliance for AI applications. In some implementations, the system uses a meta-model that consists of one or more models to analyze different aspects of AI-generated content. For example, one of the models can be trained to identify certain patterns (e.g., patterns indicative of bias) within the content by evaluating demographic attributes and characteristics present in the content. By quantifying biases within the training dataset, the system can effectively scan content for disproportionate associations with demographic attributes and provide insights into potential biases that can impact the fairness and equity of AI applications. In some implementations, the system generates actionable validation actions (e.g., test cases) that operate as input into the AI model for evaluating AI application compliance. The system evaluates the AI application against the set of validation actions and generates one or more compliance indicators and/or a set of actions based on comparisons between expected and actual outcomes and explanations. In some implementations, the system can incorporate a correction module that automates the process of implementing corrections to remove non-compliant content from AI models. The correction module adjusts the parameters of the AI model and/or updates training data based on the findings of the detection models to ensure that non-compliant content is promptly addressed and mitigated.

Unlike manual processes that rely on humans to interpret guidelines and assess compliance, the system can detect subtleties that traditional methods for content moderation often struggle to identify. The system can parse and analyze text data within the response of the AI model and identify nuanced expressions, connotations, and cultural references that can signal biased or harmful content. Additionally, by standardizing the validation criteria, the system establishes clear and objective criteria for assessing the content of an AI application, thereby minimizing the influence of individual biases or interpretations. The system can process large volumes of content rapidly and consistently, ensuring that all content is evaluated against the same set of standards and guidelines, reducing the likelihood of discrepancies or inconsistencies in enforcement decisions.

In cases where non-compliance is detected, conventional approaches to mapping gaps (e.g., issues) in controls (e.g., a set of expected actions) to operative standards (e.g., obligations, criteria, measures, principles, conditions) heavily rely on manually mapping each gap to one or more operative standards. Gaps represent situations where an expected control is either absent or not functioning properly, such as the failure to establish a specific framework within an organization. Operative standards contain controls that can be based on publications such as regulations, organizational guidelines, best practice guidelines, and others. Using manual processes heavily depends on individual knowledge and thus poses a significant risk for potential bias. This subjectivity can result in inconsistent mappings, as different individuals may understand and apply operative standards such as regulatory requirements in varied ways. Further, the sheer volume of identified gaps complicates traditional compliance efforts. Manually managing such a vast number of gaps is not only labor-intensive but also prone to oversights. Another significant disadvantage of traditional methods is the static nature of the mapping process. Conventional approaches often fail to account for the dynamic and evolving nature of regulatory requirements and organizational controls.

As such, the inventors have further developed a system (e.g., an engine within the data generation platform) to use generative AI (e.g., GAI, GenAI, generative artificial intelligence) models, such as an LLM in the above-described data generation platform, to map gaps in controls to corresponding operative standards. The system determines a set of vector representations of alphanumeric characters represented by one or more operative standards, which contain a first set of actions adhering to constraints in the set of vector representations. The system receives, via a user interface, an output generation request that includes an input with a set of gaps associated with scenarios failing to satisfy operative standards of the set of vector representations. Using the received input, the system constructs a set of prompts for each gap, where the set of prompts for a particular gap includes the set of attributes defining the scenario and the first set of actions of the operative standards. Each prompt can compare the corresponding gap against the first set of actions of the operative standards or the set of vector representations. For each gap, the system maps the gap to one or more operative standards of the set of vector representations by supplying the prompt into the LLM and, in response, receiving from the LLM a gap-specific set of operative standards that include the operative standards associated with the particular gap. The system, as compared to conventional approaches, reduces reliance on individual knowledge, thus minimizing personal biases and resulting in more uniform mappings across different individuals and teams. Additionally, the system can efficiently handle the large volumes of gaps that organizations face, significantly reducing the labor-intensive nature of manual reviews.

In another example, conventional approaches to identifying actionable items from guidelines present several challenges. Typically, conventional methods include either human reviewers or automated systems processing guidelines in a linear fashion. The conventional linear approach often leads to an overwhelming number of actionable items being identified. Furthermore, conventional approaches lack the ability to dynamically adapt to changes in guidelines over time. When new guidelines are introduced or existing ones are updated, conventional systems typically simply add new actionable items without reassessing the overall set of actionable items to ensure that the new actionable items are not redundant or contradictory to previously set actionable items. The conventional approach further fails to account for subtle shifts in interpretation that may arise from changes in definitions or regulatory language, potentially leading to outdated or irrelevant requirements remaining on the list. Consequently, organizations may end up with an inflated and confusing set of actionable items that do not accurately reflect the current landscape of the guidelines (e.g., the current regulatory landscape).

As such, the inventors have further developed a system (e.g., an engine within the data generation platform) to use generative AI models, such as an LLM in the above-described data generation platform, to identify actionable items from guidelines. The system receives, from a user interface, an output generation request that includes an input for generating an output using an LLM. The guidelines are partitioned into multiple text subsets based on predetermined criteria, such as the length or complexity of each text subset. Using the partitioned guidelines, the system constructs a set of prompts for each text subset. Each text subset can be mapped to one or more actions in the first set of actions. Subsequent actions in this second set can be generated based on previous actions. The system generates a third set of actions by aggregating the corresponding second set of actions for each text subset. Unlike conventional linear processes that result in an overwhelming number of redundant actionable items, by heuristically analyzing guidelines, the system can identify common actionable items without the parsing through the guideline documents word by word. The disclosed system reduces the number of identified actionable items to only relevant actionable items. Moreover, the system's dynamic and context-aware nature allows the system to respond to changes in guidelines over time by reassessing and mapping shifts in actionable items as the shifts occur.

Further, conventional approaches to compliance are insufficient in light of increasingly complex and expansive regulations over system assets (e.g., AI applications, hardware assets, other software assets). Conventional approaches to complying with regulations typically involve manual processes, static documentation, and periodic audits. Though the approach is feasible for traditional systems with well-defined boundaries and limited complexity, regulations are becoming progressively more complex.

Conventional approaches to complying with regulations are particularly challenging due to, for example, the broad definitions of “AI system” or “models” in regulations such as the EU AI Act, which encompass a wide range of automated systems that cannot be manually evaluated due to their volume and complexity. For example, the EU AI Act defines “AI system” broadly, including machine learning models, expert systems, and even simpler rule-based systems, meaning many systems previously not considered AI can now fall under regulatory scrutiny. Conversely, for example, the California Senate Bill 1047 (California SB-1047), defines “covered model” on and after Jan. 1, 2027 as any artificial intelligence model trained using a quantity of computing power determined by the Government Operations Agency where the cost of which exceeds one hundred million dollars when calculated using the average market price of cloud compute at the start of training. Similarly, the California Assembly Bill 2013 (California AB 2013), which is now passed and incorporated into Part 4 of Division 3 of the California Civil Code (commencing with Section 3110), defines “artificial intelligence” as any engineered or machine-based system that can infer from the input it receives how to generate outputs that can influence physical or virtual environments. The broad definitions mean that many AI systems, including those previously not considered under regulatory scrutiny, such as basic machine learning models or simple decision trees used in business operations, can now fall under the purview of this regulation. Further, the definitions for similar terms vary in the different regulations (e.g., EU AI Act versus California SB-1047).

In another example, the Digital Operational Resilience Act (DORA) defines “critical” functions (which thus maps to more stringent operational resilience requirements) as those whose disruption would materially impact the financial stability or operational continuity of the institution. Additionally, DORA mandates that financial institutions identify “critical ICT third-party service providers,” or external vendors whose services are “essential” to the institution's “critical” functions. By including any function that could materially impact financial stability or operational continuity, organizations within the financial field must continuously assess and monitor a vast array of processes and dependencies, as most of the system assets of the financial organization could potentially materially impact financial stability. This means that, for financial institutions, the reclassification can lead to increased compliance burdens and necessitate updates to existing documentation and processes. Even long-standing systems that an organization may be previously unaware of needing to monitor and/or document, such as rule-based engines and pattern recognition tools, may now fall under the purview of the new regulations. Further, some regulations require that AI used to ensure compliance must itself comply with the regulation, creating a cyclical challenge where compliance tools must also be regulated.

Moreover, across regulations, definitions at a particular point in time can vary. For example, financial institutions face significant challenges since these organizations are required to maintain extensive documentation for their models under Model Risk Management (MRM) frameworks, which include model development, validation, performance monitoring, and governance processes. The EU AI Act potentially introduces additional documentation requirements, such as transparency reports, risk assessments, and compliance checks specific to AI systems. Similarly, DORA potentially requires detailed documentation of operational resilience measures, such as incident response plans, business continuity strategies, and third-party risk assessments. In another example, the California AB 2013 mandates that developers of Gen AI systems must disclose specific documentation about the datasets used to train or develop these AI systems prior to making the GenAI system public and before any “substantial modifications” are made to the system. Consequently, for compliance purposes, organizational tools and data of any system falling under the applicable regulatory definition of a particular regulation must adhere to the compliance requirements of the particular regulation (e.g., as opposed to standard technical understandings).

Additionally, under particular regulations, system assets such as AI systems are required to be continuously evaluated throughout their lifecycle, and their decisions must be understood and interpreted by humans, necessitating continuous storage of decisions and their rationale. For example, the EU AI Act classifies AI systems into four risk levels: unacceptable risk, high risk, limited risk, and minimal risk. High-risk systems, which include those used in sensitive areas like healthcare, transportation, or law enforcement, require strict compliance with regulations, including transparency, documentation, and human oversight. For high-risk AI systems, companies must implement a risk management system that continuously evaluates the AI system throughout its lifecycle, maintain detailed records of data sources and processing methods, provide clear information about the AI system's operation and limitations, establish protocols for human intervention in critical decision-making processes, and implement robust cybersecurity measures. In another example under California SB-1047, organizations using “covered models” as defined in the regulation are required to implement a comprehensive safety and security protocol, which includes the capability for a full shutdown of the model, and further required to retain unredacted copies of safety protocols and audit reports for as long as the model is in use, plus five years, thus necessitating continuous evaluation. Similarly, even beyond AI applications, the DORA mandates continuous monitoring of information and communication technology (ICT) systems (e.g., system assets) by requiring real-time monitoring systems, incident reporting mechanisms, and risk management frameworks that include continuous assessment of ICT systems. In particular, the DORA expects organizations to know (1) what the organization's information assets are (internal or external, qualitative or quantitative), (2) where they reside in a firm's ICT estate, and (3) how to protect the information assets. Critical ICT assets in the DORA must be mapped to the specific information assets that are linked to the critical ICT asset. Thus, conventional compliance approaches, which often rely on manual periodic processes and static documentation, are insufficient for the continuous monitoring requirement of system assets in particular risk categories.

Further, system assets such as AI systems face added challenges in meeting compliance requirements across multiple interrelated subject matters, such as risk management, data governance, transparency, human oversight, and cybersecurity measures. Each of the evaluated areas demands specialized knowledge, making the documentation process complex and resource-intensive. Documenting compliance across these diverse and interrelated areas involves coordinating efforts across multiple teams, maintaining up-to-date records, and ensuring that all aspects of the system asset adhere to regulatory requirements, which can be a challenging and ongoing task. For example, effective risk management depends on accurate data governance to ensure that data used for risk assessments is reliable and compliant with privacy regulations, while robust cybersecurity measures are necessary to protect the data from breaches, thereby supporting both risk management and data governance efforts. Since the areas are interrelated, duplicating work wastes valuable resources such as CPU usage, storage capacity, and human effort, as multiple teams may redundantly process the same data, run similar compliance checks, and maintain overlapping documentation, leading to inefficiencies and increased operational costs.

Moreover, to monitor compliance of system assets such as AI systems, decisions made by AI systems must be understood and interpreted by humans, which implicitly means that decisions and their rationale should be continuously stored and managed. The requirement presents several challenges, particularly in the context of complex AI models that can operate as “black boxes,” where the decision-making process is not easily interpretable. From the user's perspective, the AI model functions as a “black box,” where the input is fed into the system, and the output prediction is produced without visibility into the underlying logic. The opaque nature of AI systems makes it difficult to trace how specific decisions are made (especially with complex documentation), thereby complicating the identification of gaps in compliance.

As such, the inventors have further developed a system (e.g., an engine within the data generation platform) to use generative AI models, such as the LLMs in the above-described data generation platform, to manage operational resilience of system assets using an AI model. The system (1) uses an existing inventory of system assets and/or (2) uses the regulatory definitions to create an inventory of the organization's system assets (i.e., observed system assets). To use the regulatory definitions to create an inventory of the organization's system assets, the system can obtain 1) a set of operational data indicating system assets (e.g., a software asset and/or a hardware asset, such as an AI application) used on a set of informational assets within the set of operational data and 2) a set of alphanumeric characters defining one or more operative boundaries of a set of expected system assets configured to adhere to constraints of the set of alphanumeric characters (e.g., the EU AI Act, the California SB-1047, the DORA, the California AB 2013). The set of expected system assets include a set of attributes common among each expected system asset in the set of expected system assets (e.g., attributes of “critical” system assets, attributes of ICT systems). Using the common attributes, a first set of AI models determines a set of observed system assets from the set of operational data (e.g., the observed critical system assets, the observed ICT systems), where each particular observed system asset includes a set of features of the particular observed system asset (e.g., a text-based description of the particular observed system asset, an expected input of the particular observed system asset, an expected output of the particular observed system asset, interdependencies of the particular observed system asset, data supporting the system asset).

Using a second set of AI models (which can be the same as or different from the first set of models), the system maps (e.g., associates, links) each information asset in the set of informational assets indicated by one or more observed system assets to the one or more observed system assets. For each particular observed system asset, a third set of AI models identifies a set of criteria of the particular observed system asset within the set of alphanumeric characters, identify a set of gaps of the particular observed system asset by comparing the set of criteria of the particular observed system asset with the set of features of the particular observed system asset. For example, for each of the identified AI use cases in the inventory, the system examines the AI use case based on the applicable regulations (e.g., the EU AI Act, California SB-1047) using rule-based systems and/or one or more AI models. Using the identified set of gaps, the system can generate a set of actions to be performed related to the particular observed system asset configured to satisfy the set of criteria of the particular observed system asset.

Internally, the system can identify new gaps from automatically refreshed assessments and generate an alert based on the gap (e.g., generating an exceptional trigger for potential “Prohibited” or “High-Risk” AI use cases, or “Critical” ICT assets). In some implementations, the system converts existing documentation (e.g., previously aggregated documentation) into new documentation that meets new regulatory requirements. The system can analyze existing documentation in conjunction with the text of relevant regulations to classify the risk category (while disregarding the preassigned risk rating provided by other regulations). Subsequently, the system utilizes the existing documentation to generate new documentation that complies with the updated regulatory requirements.

Unlike conventional approaches that rely on manual processes, static documentation, and periodic audits, the disclosed systems and methods can generate an inventory, identified through a RAG search of operational data, ensuring that all relevant system assets are accounted for, thereby reducing the compliance burden and ensuring that even previously overlooked system assets are included in the analysis. Further, unlike conventional methods that struggle with varying definitions of terms such as “AI system” or other system assets across different regulations, the disclosed system examines each identified system asset based on applicable regulations using rule-based systems and/or AI models. Further, by classifying system assets into risk categories based on the particular regulations, the system ensures that each use case is evaluated according to the specific requirements of the relevant regulatory framework. This dynamic classification and gap identification process allows for real-time adjustments and continuous compliance, reducing the risk of regulatory breaches and associated penalties. Additionally, the system implements a risk management system that continuously evaluates system assets throughout their lifecycle. By generating timestamped compliance reports and aggregating the required documentation, the system provides a dynamic and automated solution for continuous evaluation, thereby reducing the risk of non-compliance. The system addresses the challenges in meeting compliance requirements across multiple interrelated subject matters by using a single system of generative AI model(s) across the various subject matters, which reduces the need for duplicative work. The system's ability to aggregate documentation and generate compliance reports ensures that all interrelated areas are addressed cohesively, which allows organizations to allocate resources more effectively and reduce operational overhead.

Further, unlike conventional methods that struggle with the opaque nature of complex AI models, the system consolidates the compliance requirements and operational data into a set of gaps in compliance and/or compliance actions that are automatically executed. By mapping compliance requirements to the operational data of the AI systems, such as model inputs and outputs, decision-making processes, and performance metrics, the system can pinpoint specific areas where compliance gaps exist. For instance, if a regulation requires documentation of data sources and processing methods, the system can automatically check the existing documentation against this requirement and flag any discrepancies as compliance gaps.

The methods disclosed herein cause a reduction in greenhouse gas emissions compared to traditional methods for operating models. Every year, approximately 40 billion tons of COare emitted around the world. Power consumption by digital technologies account for approximately 4% of this figure. Further, conventional user device and application settings can sometimes exacerbate the causes of climate change. For example, the average U.S. power plant expends approximately 500 grams of carbon dioxide for every kWh generated. The implementations disclosed herein for conserving hardware, software, and network resources can mitigate climate change by reducing and/or preventing additional greenhouse gas emissions into the atmosphere. For example, managing operational resilience using an artificial intelligence model as described herein reduces electrical power consumption compared to traditional methods. In particular, automating computer-executable compliance and monitoring tasks reduces the need for extensive manual intervention and redundant processes, which can consume additional computational power and energy. Continuously monitoring compliance reduces the need for periodic, resource-heavy audits and assessments, which can traditionally require substantial computational power to process large volumes of data. Spikes in power consumption can lead to higher greenhouse gas emissions. Power plants may need to ramp up production quickly to meet sudden increases in demand, and may rely on less efficient and more polluting sources of energy. For example, instead of conducting quarterly or annual compliance reviews that involve extensive data processing and analysis, the disclosed system can perform compliance monitoring on an ongoing basis to process smaller, incremental data updates, leading to a more consistent energy consumption.

Moreover, in the U.S., datacenters are responsible for approximately 2% of the country's electricity use, while globally they account for approximately 200 terawatt Hours (TWh). Transferring 1 GB of data can produce approximately 3 kg of CO. Each GB of data downloaded thus results in approximately 3 kg of COemissions or other greenhouse gas emissions. The storage of 100 GB of data in the cloud every year produces approximately 0.2 tons of COor other greenhouse gas emissions. The continuous monitoring and dynamic compliance management described herein enables the system to detect and address compliance issues as they arise, rather than waiting for the next scheduled review. The proactive approach not only ensures that compliance is maintained more effectively but also reduces the likelihood of significant non-compliance issues that would require extensive corrective actions. By addressing potential problems early, the system lowers the need for resource-intensive remediation efforts, further conserving energy, and obviates the need for wasteful COemissions. Further, keeping system assets such as AI models in compliance with environmental regulations ensures that the system assets themselves adhere to standards that lower their environmental impact. By continuously monitoring and remediating gaps in the energy efficiency of system assets, the system helps reduce the carbon footprint associated with data processing and storage. Compliance with environmental regulations can include requirements for energy efficiency, waste reduction, and sustainable resource use. By meeting these requirements, the system contributes to broader environmental goals, such as reducing greenhouse gas emissions and conserving natural resources. Therefore, the disclosed implementations mitigate climate change and the effects of climate change by reducing the amount of data stored and downloaded in comparison to conventional network technologies.

Attempting to create a system to dynamically identify and remediate gaps in compliance for system assets in view of the available conventional approaches created significant technological uncertainty. Creating such system required addressing several unknowns in conventional approaches in compliance management, such as how to interpret regulations and apply the regulations to the system assets. Regulations vary significantly across different jurisdictions and industries, making it challenging to create a system that can accurately interpret and apply these complex and variable regulatory standards. Similarly, conventional approaches in compliance management did not provide methods of continuously learning and adapting to new regulatory changes and updates.

Conventional approaches rely on periodic reviews and audits, which are not sufficient for the dynamic nature of an organization's system assets, such as AI systems. In view of regulations such as the EU AI act, the California Senate Bill, and the DORA that require continuous compliance, conventional approaches were insufficient because the conventional approaches did not continuously track compliance status and identifying gaps as they arise, but rather during scheduled audits. For example, a conventional system may manually review logs, data processing workflows, and model outputs. The process can include extracting data from various sources, such as databases, log files, and application programming interfaces (APIs), and then manually cross-referencing the data against regulatory requirements. The manual process is not only time-consuming but also prone to human error, and it fails to capture compliance issues that may arise between audits. Conversely, the disclosed system determines how to dynamically meet the requirements of regulations by integrating with the AI models and data pipelines (containing the operational data and guidelines) in real-time, using APIs and event-driven architectures to capture data as it is processed. For instance, the system can automatically scan and interpret regulatory texts, mapping the requirements to specific data processing activities and model behaviors. Further, the system can identify unusual patterns in data access or processing that may suggest a breach of compliance, and can automatically execute computer-executable tasks to remediate breaches in compliance.

To overcome the technological uncertainties, the inventors systematically evaluated multiple design alternatives. For example, the inventors tested various machine learning algorithms to determine which would be most effective for dynamic compliance monitoring given the variable data in a system asset. The inventors experimented with a rule-based approach where predefined rules were manually coded to map system assets to specific regulatory criteria. The method involved creating an extensive database of rules that corresponded to different regulations and manually updating these rules as new regulations emerged. Additionally, the inventors explored a template-based approach, where compliance templates were created for different types of system assets, and these templates were used to guide the compliance monitoring process.

However, the rule-based approach proved to be inflexible and difficult to maintain. As regulations evolved, the manual updating of rules becomes increasingly cumbersome and error-prone, leading to delays in compliance updates and potential gaps in regulatory coverage. Similarly, the template-based approach lacked the granularity needed to address the specific nuances of different system assets due to the variability in regulations. The templates were too generic, resulting in either overly broad compliance checks that can generate numerous false positives or overly narrow checks that can miss compliance issues. Further, given that regulations such as the DORA govern all ICT assets, creating templates for such a large number of different types of system assets was both infeasible and impractical. The variety of jurisdictions, each with its own set of laws and regulations that are subject to change, renders manual compliance efforts equally infeasible and impractical. Additionally, to prove compliance and satisfy audit requirements, a record of compliance must be documented for all time periods under audit.

Thus, the inventors experimented with different methods for dynamically identifying and remediating gaps in compliance. For example, the inventors tested various machine learning models to analyze regulatory texts (e.g., using NLP) and automatically extract relevant compliance criteria to create a system that could adapt to new regulations in real-time. The system can map the extracted criteria to specific system assets using, for example, classification models, such as support vector machines (SVM) and random forests, to categorize system assets based on regulatory requirements. The system can use the criteria to continuously monitor system assets, to identify deviations from expected behavior that could indicate compliance issues.

While the current description provides examples related to LLMs, one of skill in the art would understand that the disclosed techniques can apply to other forms of machine learning or algorithms, including unsupervised, semi-supervised, supervised, and reinforcement learning techniques. For example, the disclosed data generation platform can use model outputs from support vector machine (SVM), k-nearest neighbor (KNN), decision-making, linear regression, random forest, naïve Bayes, or logistic regression algorithms, and/or other suitable computational models.

In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of implementations of the present technology. It will be apparent, however, to one skilled in the art that implementation of the present technology can be practiced without some of these specific details.

The phrases “in some implementations,” “in several implementations,” “according to some implementations,” “in the implementations shown,” “in other implementations,” and the like generally mean the specific feature, structure, or characteristic following the phrase is included in at least one implementation of the present technology and can be included in more than one implementation. In addition, such phrases do not necessarily refer to the same implementations or different implementations.

shows an illustrative environmentfor evaluating machine learning model inputs (e.g., language model prompts) and outputs for model selection and validation, in accordance with some implementations of the present technology. For example, the environmentincludes the data generation platform, which is capable of communicating with (e.g., transmitting or receiving data to or from) a data nodeand/or third-party databases-via a network. The data generation platformcan include software, hardware, or a combination of both and can reside on a physical server or a virtual server (e.g., as described in) running on a physical computer system. For example, the data generation platformcan be distributed across various nodes, devices, or virtual machines (e.g., as in a distributed cloud server). In some implementations, the data generation platformcan be configured on a user device (e.g., a laptop computer, smartphone, desktop computer, electronic tablet, or another suitable user device). Furthermore, the data generation platformcan reside on a server or node and/or can interface with third-party databases-directly or indirectly.

The data nodecan store various data, including one or more machine learning models, prompt validation models, associated training data, user data, performance metrics and corresponding values, validation criteria, and/or other suitable data. For example, the data nodeincludes one or more databases, such as an event database (e.g., a database for storage of records, logs, or other information associated with LLM-related user actions), a vector database, an authentication database (e.g., storing authentication tokens associated with users of the data generation platform), a secret database, a sensitive token database, and/or a deployment database.

An event database can include data associated with events relating to the data generation platform. For example, the event database stores records associated with users' inputs or prompts for generation of an associated natural language output (e.g., prompts intended for processing using an LLM). The event database can store timestamps and the associated user requests or prompts. In some implementations, the event database can receive records from the data generation platformthat include model selections/determinations, prompt validation information, user authentication information, and/or other suitable information. For example, the event database stores platform-level metrics (e.g., bandwidth data, central processing unit (CPU) usage metrics, and/or memory usage associated with devices or servers associated with the data generation platform). By doing so, the data generation platformcan store and track information relating to performance, errors, and troubleshooting. The data generation platformcan include one or more subsystems or subcomponents. For example, the data generation platformincludes a communication engine, an access control engine, a breach mitigation engine, a performance engine, and/or a generative model engine.

A vector database can include data associated with vector embeddings of data. For example, the vector database includes a numerical representation (e.g., arrays of values) that represent the semantic meaning of unstructured data (e.g., text data, audio data, or other similar data). For example, the data generation platformreceives inputs such as unstructured data, including text data, such as a prompt, and utilize a vector encoding model (e.g., with a transformer or neural network architecture) to generate vectors within a vector space that represents meaning of data objects (e.g., of words within a document). By storing information within a vector database, the data generation platformcan represent inputs, outputs, and other data in a processable format (e.g., with an associated LLM), thereby improving the efficiency and accuracy of data processing.

An authentication database can include data associated with user or device authentication. For example, the authentication database includes stored tokens associated with registered users or devices of the data generation platformor associated development pipeline. For example, the authentication database stores keys (e.g., public keys that match private keys linked to users and/or devices). The authentication database can include other user or device information (e.g., user identifiers, such as usernames, or device identifiers, such as medium access control (MAC) addresses). In some implementations, the authentication database can include user information and/or restrictions associated with these users.

A sensitive token (e.g., secret) database can include data associated with secret or otherwise sensitive information. For example, secrets can include sensitive information, such as application programming interface (API) keys, passwords, credentials, or other such information. For example, sensitive information includes personally identifiable information (PII), such as names, identification numbers, or biometric information. By storing secrets or other sensitive information, the data generation platformcan evaluate prompts and/or outputs to prevent breaches or leakage of such sensitive information.

A deployment database can include data associated with deploying, using, or viewing results associated with the data generation platform. For example, the deployment database can include a server system (e.g., physical or virtual) that stores validated outputs or results from one or more LLMs, where such results can be accessed by the requesting user.

The data generation platformcan receive inputs (e.g., prompts), training data, validation criteria, and/or other suitable data from one or more devices, servers, or systems. The data generation platformcan receive such data using communication engine, which can include software components, hardware components, or a combination of both. For example, the communication engineincludes or interfaces with a network card (e.g., a wireless network card and/or a wired network card) that is associated with software to drive the card and enables communication with network. In some implementations, the communication enginecan also receive data from and/or communicate with the data node, or another computing device. The communication enginecan communicate with the access control engine, the breach mitigation engine, the performance engine, and the generative model engine.

In some implementations, the data generation platformcan include the access control engine. The access control enginecan perform tasks relating to user/device authentication, controls, and/or permissions. For example, the access control enginereceives credential information, such as authentication tokens associated with a requesting device and/or user. In some implementations, the access control enginecan retrieve associated stored credentials (e.g., stored authentication tokens) from an authentication database (e.g., stored within the data node). The access control enginecan include software components, hardware components, or a combination of both. For example, the access control engineincludes one or more hardware components (e.g., processors) that are able to execute operations for authenticating users, devices, or other entities (e.g., services) that request access to an LLM associated with the data generation platform. The access control enginecan directly or indirectly access data, systems, or nodes associated with the third-party databases-and can transmit data to such nodes. Additionally or alternatively, the access control enginecan receive data from and/or send data to the communication engine, the breach mitigation engine, the performance engine, and/or the generative model engine.

The breach mitigation enginecan execute tasks relating to the validation of inputs and outputs associated with the LLMs. For example, the breach mitigation enginevalidates inputs (e.g., prompts) to prevent sensitive information leakage or malicious manipulation of LLMs, as well as validate the security or safety of the resulting outputs. The breach mitigation enginecan include software components (e.g., modules/virtual machines that include prompt validation models, performance criteria, and/or other suitable data or processes), hardware components, or a combination of both. As an illustrative example, the breach mitigation enginemonitors prompts for the inclusion of sensitive information (e.g., PII), or other forbidden text, to prevent leakage of information from the data generation platformto entities associated with the target LLMs. The breach mitigation enginecan communicate with the communication engine, the access control engine, the performance engine, the generative model engine, and/or other components associated with the network(e.g., the data nodeand/or the third-party databases-).

The performance enginecan execute tasks relating to monitoring and controlling performance of the data generation platform(e.g., or the associated development pipeline). For example, the performance engineincludes software components (e.g., performance monitoring modules), hardware components, or a combination thereof. To illustrate, the performance enginecan estimate performance metric values associated with processing a given prompt with a selected LLM (e.g., an estimated cost or memory usage). By doing so, the performance enginecan determine whether to allow access to a given LLM by a user, based on the user's requested output and the associated estimated system effects. The performance enginecan communicate with the communication engine, the access control engine, the performance engine, the generative model engine, and/or other components associated with the network(e.g., the data nodeand/or the third-party databases-).

The generative model enginecan execute tasks relating to machine learning inference (e.g., natural language generation based on a generative machine learning model, such as an LLM). The generative model enginecan include software components (e.g., one or more LLMs, and/or API calls to devices associated with such LLMs), hardware components, and/or a combination thereof. To illustrate, the generative model enginecan provide users' prompts to a requested, selected, or determined model (e.g., LLM) to generate a resulting output (e.g., to a user's query within the prompt). As such, the generative model engineenables flexible, configurable generation of data (e.g., text, code, or other suitable information) based on user input, thereby improving the flexibility of software development or other such tasks. The generative model enginecan communicate with the communication engine, the access control engine, the performance engine, the generative model engine, and/or other components associated with the network(e.g., the data nodeand/or the third-party databases-).

Engines, subsystems, or other components of the data generation platformare illustrative. As such, operations, subcomponents, or other aspects of particular subsystems of the data generation platformcan be distributed, varied, or modified across other engines. In some implementations, particular engines can be deprecated, added, or removed. For example, operations associated with breach mitigation are performed at the performance engineinstead of at the breach mitigation engine.