Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method for detecting a model security, the method comprising: obtaining result data computed by using a model for current input data, wherein the result data comprises intermediate result data or output result data; obtaining, in a trusted execution environment, second result data computed by using the model for a plurality of samples, wherein the second result data comprises second intermediate result data or second output result data; obtaining a GAN through training by using the second result data, wherein the GAN comprises a generator and the discriminator, and wherein obtaining the GAN comprises: obtaining, in the trusted execution environment, the discriminator through training based on a generative adversarial network (GAN) framework, the model, and the plurality of samples; generating data to be input to the generator based on the second result data; and obtaining the generator through training based on the second result data, the data to be input to the generator, and the GAN framework; discriminating the result data by using the discriminator and based on comparing respective distributions of the result data and the second result data, to detect whether the model is currently secure; and determining a security detection result of the model.
This invention relates to detecting security vulnerabilities in machine learning models by analyzing their output or intermediate data. The method addresses the challenge of ensuring model integrity and detecting potential security threats, such as adversarial attacks or data poisoning, which can compromise model performance or behavior. The method involves obtaining result data from a model when processing current input data, where this data can be either intermediate or final output. Additionally, in a trusted execution environment, second result data is generated by running the model on a set of predefined samples. This second result data is used to train a generative adversarial network (GAN), which consists of a generator and a discriminator. The discriminator is trained within the trusted environment using the GAN framework, the model, and the sample data. The generator is trained based on the second result data and synthetic input data derived from it. Once trained, the GAN's discriminator evaluates the original result data by comparing its distribution to that of the second result data. This comparison helps determine whether the model is currently secure, with the final output indicating the security detection result. The approach leverages adversarial training to identify deviations that may indicate security risks.
2. The method according to claim 1 , wherein the model is a deep learning model, and the intermediate result data is data computed at an intermediate layer of the model.
A deep learning model processes input data through multiple layers, generating intermediate result data at each layer before producing a final output. This intermediate data represents learned features or transformations of the input. The method involves analyzing this intermediate data to detect anomalies or errors in the model's processing. By examining the intermediate layer outputs, the system can identify deviations from expected patterns, which may indicate issues such as incorrect training, data corruption, or adversarial attacks. The analysis may include statistical comparisons, pattern recognition, or other techniques to assess the integrity of the model's computations. This approach improves model reliability by detecting errors early in the processing pipeline, allowing for corrective actions before the final output is generated. The method is applicable to various deep learning architectures, including neural networks, convolutional networks, and recurrent networks, where intermediate layer data provides insights into the model's internal state.
3. The method according to claim 1 , wherein the second result data is authentic.
A method for verifying the authenticity of data in a distributed system involves generating a first set of data and a second set of data, where the second set is derived from the first. The method includes transmitting the first set to a remote system and receiving a response that includes the second set. The authenticity of the second set is verified by comparing it to an expected value, ensuring the data has not been tampered with during transmission or storage. This process is particularly useful in systems where data integrity is critical, such as blockchain, secure communications, or distributed databases. The verification step confirms that the second set of data matches the expected result, providing assurance that the data remains unaltered. The method may involve cryptographic techniques, such as hashing or digital signatures, to generate and validate the second set of data. By ensuring the second set is authentic, the system can trust the integrity of the data it receives, preventing unauthorized modifications or corruption. This approach is applicable in scenarios where data must be transmitted or stored securely, such as financial transactions, medical records, or sensitive communications. The method enhances security by detecting any discrepancies in the data, allowing for corrective actions to be taken if tampering is detected.
4. The method according to claim 1 , wherein generating data to be input to the generator further comprises: generating data to be input to the generator based on random data.
This invention relates to methods for generating data inputs for a generator, particularly in the context of machine learning or synthetic data generation. The problem addressed is the need for effective and controllable data generation to train or test machine learning models, ensuring diversity and randomness in the input data to improve model robustness. The method involves generating data to be input to a generator, where the data is derived from random data sources. This ensures that the generated data is varied and unpredictable, which is crucial for training models that must generalize well to unseen data. The random data can be sourced from noise distributions, pseudorandom number generators, or other stochastic processes. By incorporating randomness, the method prevents overfitting and biases that may arise from deterministic or repetitive input patterns. The generator itself may be a neural network, such as a generative adversarial network (GAN) or a variational autoencoder (VAE), which produces synthetic data based on the input. The random data serves as a seed or latent variable, allowing the generator to produce diverse outputs. This approach is particularly useful in applications like data augmentation, privacy-preserving data synthesis, and unsupervised learning, where high-quality, varied training data is essential. The method ensures that the generated data retains the statistical properties of the original data distribution while introducing controlled randomness, making it suitable for training models in domains like computer vision, natural language processing, and reinforcement learning.
5. The method according to claim 2 , wherein discriminating the result data by using a discriminator comprises: discriminating whether the result data is true or false by using the discriminator, wherein a discrimination result reflects whether distribution of the result data is consistent with the second result data.
This invention relates to data validation in machine learning systems, specifically addressing the challenge of verifying the consistency of generated data against reference data. The method involves using a discriminator model to assess whether result data (e.g., synthetic or generated data) is true or false by comparing its distribution to a second set of result data (e.g., real or ground-truth data). The discriminator evaluates the statistical consistency between the two datasets, determining if the generated data accurately represents the expected distribution. This approach is particularly useful in generative models, such as GANs (Generative Adversarial Networks), where ensuring the fidelity of synthetic data is critical for applications like data augmentation, anomaly detection, or simulation. The discriminator acts as a binary classifier, distinguishing between plausible and implausible data based on distributional alignment. The method enhances reliability in scenarios where generated data must closely match real-world distributions, improving the robustness of downstream tasks dependent on high-quality synthetic data.
6. The method according to claim 5 , wherein discriminating the result data by using the discriminator, and determining the security detection result of the model comprises: discriminating the result data by using the discriminator to obtain the discrimination result; and determining the security detection result of the model based on the discrimination result.
This invention relates to security detection systems, specifically improving the accuracy of security detection models by using a discriminator to evaluate model outputs. The problem addressed is the challenge of reliably detecting security threats in data, where traditional models may produce false positives or miss threats due to limitations in their training data or architecture. The method involves a security detection model that processes input data to generate result data, which may include threat indicators or other security-related outputs. A discriminator, which is a separate neural network or classifier, is then used to analyze the result data. The discriminator evaluates the quality, relevance, or confidence of the model's output, producing a discrimination result. This discrimination result is then used to determine the final security detection result of the model. For example, if the discriminator identifies inconsistencies or low confidence in the model's output, the system may flag the result for further review or adjust the detection decision accordingly. This approach enhances the reliability of security detection by introducing an additional layer of validation, reducing false positives and improving threat detection accuracy. The discriminator can be trained separately or jointly with the detection model to refine its ability to assess the model's outputs effectively.
7. The method according to claim 5 , wherein the discriminating the result data by using the discriminator, and determining the security detection result of the model comprises: obtaining the output result data; and determining the security detection result of the model based on the output result data and the discrimination result.
This invention relates to security detection systems that use machine learning models to analyze data and identify potential security threats. The problem addressed is improving the accuracy and reliability of security detection results by incorporating a discriminator to evaluate the output of the model. The method involves using a trained machine learning model to process input data and generate output result data. A discriminator, which is a separate component trained to distinguish between valid and invalid model outputs, then evaluates the output result data. The discriminator generates a discrimination result indicating whether the output is likely to be correct or erroneous. The security detection result is then determined based on both the original output result data and the discrimination result from the discriminator. This approach helps filter out false positives or negatives by cross-verifying the model's output with the discriminator's assessment, enhancing the overall reliability of the security detection system. The discriminator may be trained using adversarial techniques or other methods to improve its ability to identify model errors. This method is particularly useful in applications where security threats must be detected with high accuracy, such as intrusion detection, malware analysis, or network security monitoring.
8. The method according to claim 1 , wherein the discriminator is in a predetermined secure execution environment.
A method for enhancing the security of a discriminator in a machine learning system involves operating the discriminator within a predetermined secure execution environment. The discriminator is a component that evaluates the quality or authenticity of generated data, such as synthetic images or text, by distinguishing between real and fake inputs. By placing the discriminator in a secure execution environment, the method ensures that its operations, data, and decision-making processes are protected from unauthorized access, tampering, or reverse engineering. This secure environment may include hardware-based security mechanisms, such as trusted execution environments (TEEs), secure enclaves, or other isolated execution contexts that enforce strict access controls and integrity checks. The method improves the reliability and trustworthiness of the discriminator by preventing adversarial attacks or unauthorized modifications that could compromise its performance. This approach is particularly useful in applications where data integrity and security are critical, such as financial fraud detection, biometric authentication, or confidential data generation. The secure execution environment ensures that the discriminator's internal state and computations remain confidential, even when the system is deployed in untrusted or adversarial environments.
9. The method according to claim 8 , wherein the discriminator is in a user terminal.
A method for improving the accuracy of data classification in a machine learning system involves using a discriminator to evaluate the quality of generated data. The discriminator is a neural network trained to distinguish between real and synthetic data, ensuring that the generated data closely matches the characteristics of real-world data. This method is particularly useful in applications where high-quality synthetic data is needed for training or testing machine learning models, such as in image generation, natural language processing, or data augmentation. The discriminator is integrated into a user terminal, allowing for local processing of data without relying on a central server. This decentralized approach enhances privacy and reduces latency, as the user terminal can perform real-time evaluations of generated data. The discriminator is trained using a dataset that includes both real and synthetic samples, enabling it to learn the distinguishing features between the two. By continuously refining the discriminator, the system ensures that the generated data remains accurate and reliable over time. This method is beneficial in scenarios where data privacy is a concern, as it eliminates the need to transmit sensitive data to a remote server for evaluation. Additionally, the local processing capability allows for faster feedback loops, improving the efficiency of data generation and classification tasks. The discriminator's ability to adapt to new data distributions ensures that the system remains effective even as the underlying data characteristics evolve.
10. The method according to claim 1 , wherein the execution environment comprises a relatively open environment.
A method for operating a computing system in a relatively open execution environment, where the environment lacks strict security controls or isolation mechanisms. The method involves dynamically adjusting system operations to mitigate risks associated with the open environment, such as unauthorized access or data breaches. This includes monitoring system activities, detecting potential threats, and applying security measures in real-time to protect sensitive data and maintain system integrity. The method may also involve isolating critical processes, restricting access to certain resources, and enforcing security policies to prevent unauthorized actions. By adapting to the open environment, the method ensures secure and reliable operation of the computing system despite the lack of inherent security safeguards. The approach is particularly useful in scenarios where strict security controls are impractical or unavailable, such as in public cloud environments, shared computing resources, or legacy systems with limited security features. The method enhances security without requiring extensive modifications to the existing infrastructure, making it suitable for environments where flexibility and adaptability are essential.
11. The method according to claim 1 , wherein the model comprises a risk control engine installed on a user device.
A system and method for risk assessment and control in a computing environment involves a model that includes a risk control engine installed on a user device. The model is designed to evaluate and mitigate risks associated with user activities, such as data processing, network communications, or application execution. The risk control engine operates locally on the user device to analyze potential threats in real-time, such as malware, unauthorized access, or data breaches. It applies predefined risk assessment criteria to determine the likelihood and impact of identified risks. Based on the assessment, the engine enforces security measures, such as blocking suspicious activities, restricting access to sensitive data, or prompting user confirmation before proceeding with high-risk operations. The system may also integrate with remote servers or cloud-based services to update risk profiles, threat intelligence, or security policies dynamically. By operating on the user device, the risk control engine ensures immediate and localized risk management, reducing reliance on centralized systems and improving response times to emerging threats. The method enhances security by combining local analysis with adaptive risk mitigation strategies.
12. The method according to claim 1 , wherein the model comprises a deep learning model and the result data comprises a vector.
A system and method for processing data using a deep learning model to generate vector-based results. The invention addresses the challenge of efficiently transforming input data into structured, vectorized outputs for further analysis or machine learning applications. The deep learning model is trained to analyze input data, such as text, images, or other structured/unstructured inputs, and produce a vector representation that captures relevant features or patterns. This vector output can be used for tasks like classification, clustering, or similarity comparison. The model may include neural network architectures such as transformers, convolutional networks, or recurrent networks, depending on the input data type. The vector output is optimized for downstream processing, ensuring compatibility with other machine learning models or analytical tools. The system may also include preprocessing steps to normalize or format input data before feeding it into the deep learning model. The invention improves data processing efficiency by automating feature extraction and enabling scalable, high-dimensional data representation. Applications include natural language processing, computer vision, recommendation systems, and predictive analytics. The method ensures that the vector output is both informative and computationally efficient, reducing the need for manual feature engineering.
13. The method according to claim 1 , wherein the random data comprises a randomly generated vector.
A method for generating and processing random data in a computational system addresses the need for secure and unpredictable data sequences in cryptographic and security applications. The method involves creating random data, which is then used to enhance security protocols, encryption processes, or other applications requiring high-entropy inputs. Specifically, the random data is structured as a randomly generated vector, which may be a multi-dimensional array or sequence of values. This vector-based approach allows for efficient storage, manipulation, and transmission of the random data, ensuring that the generated values are both statistically independent and uniformly distributed. The method may also include steps to verify the randomness of the vector, such as statistical tests or entropy measurements, to confirm its suitability for security-sensitive operations. By using a vector format, the method enables compatibility with various cryptographic algorithms and systems that rely on structured random inputs. The overall solution improves the reliability and security of systems that depend on random data generation, reducing vulnerabilities associated with predictable or biased random sequences.
14. A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations for detecting a model security, the operations comprising: obtaining result data computed by using a model for current input data, wherein the result data comprises intermediate result data or output result data; obtaining, in a trusted execution environment, second result data computed by using the model for a plurality of samples, wherein the second result data comprises second intermediate result data or second output result data; obtaining a GAN through training by using the second result data, wherein the GAN comprises a generator and the discriminator, and wherein obtaining the GAN comprises: obtaining, in the trusted execution environment, the discriminator through training based on a generative adversarial network (GAN) framework, the model, and the plurality of samples; generating data to be input to the generator based on the second result data; and obtaining the generator through training based on the second result data, the data to be input to the generator, and the GAN framework; discriminating the result data by using the discriminator and based on comparing respective distributions of the result data and the second result data, to detect whether the model is currently secure; and determining a security detection result of the model.
This invention relates to detecting security vulnerabilities in machine learning models by analyzing their output or intermediate data. The problem addressed is ensuring the integrity and security of machine learning models, which can be compromised by adversarial attacks or other malicious manipulations. The solution involves a system that uses a generative adversarial network (GAN) trained in a trusted execution environment to compare the model's current behavior with its expected behavior under normal conditions. The system first obtains result data from the model when processing current input data, which can include either intermediate or final output data. Separately, in a secure trusted execution environment, the system computes second result data by running the model on a plurality of trusted samples. This second result data is used to train a GAN, which consists of a generator and a discriminator. The discriminator is trained in the trusted environment using the GAN framework, the model, and the trusted samples. The generator is trained using data derived from the second result data and the GAN framework. The trained GAN is then used to discriminate the model's current result data by comparing the distributions of the current result data and the trusted second result data. This comparison helps detect whether the model is behaving as expected or if it has been compromised. The system ultimately determines a security detection result, indicating whether the model is currently secure. This approach leverages the GAN's ability to distinguish between normal and anomalous model behavior, providing a robust method for security monitoring.
15. The computer-readable medium according to claim 14 , wherein generating data to be input to the generator further comprises: generating data to be input to the generator based on random data.
The invention relates to computer-readable media and methods for generating data inputs for a generator, particularly in the context of machine learning or artificial intelligence systems. The problem addressed is the need for effective and efficient generation of input data to train or operate a generator, such as a generative model, to produce high-quality outputs. The invention improves upon prior approaches by incorporating random data into the generation process, enhancing the diversity and unpredictability of the input data. This helps prevent overfitting and improves the robustness of the generator's outputs. The generator may be part of a larger system, such as a generative adversarial network (GAN), where the quality of the generated data directly impacts the performance of the adversarial training process. By leveraging random data, the invention ensures that the generator receives varied and representative inputs, leading to more accurate and reliable outputs. The method involves generating input data for the generator based on random data, which can be combined with other data sources or processing steps to further refine the input. This approach is particularly useful in applications requiring synthetic data generation, such as data augmentation, simulation, or training AI models. The invention enhances the flexibility and adaptability of the generator, making it suitable for a wide range of applications where high-quality synthetic data is needed.
16. A computer-implemented system, comprising: one or more computers; and one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform operations for detecting a model security, the operations comprising: obtaining result data computed by using a model for current input data, wherein the result data comprises intermediate result data or output result data; obtaining, in a trusted execution environment, second result data computed by using the model for a plurality of samples, wherein the second result data comprises second intermediate result data or second output result data; obtaining a GAN through training by using the second result data, wherein the GAN comprises a generator and the discriminator, and wherein obtaining the GAN comprises: obtaining, in the trusted execution environment, the discriminator through training based on a generative adversarial network (GAN) framework, the model, and the plurality of samples; generating data to be input to the generator based on the second result data; and obtaining the generator through training based on the second result data, the data to be input to the generator, and the GAN framework; discriminating the result data by using the discriminator and based on comparing respective distributions of the result data and the second result data, to detect whether the model is currently secure; and determining a security detection result of the model.
This invention relates to a computer-implemented system for detecting security vulnerabilities in machine learning models. The system addresses the challenge of ensuring the integrity and security of models by identifying anomalies or deviations in their outputs that may indicate tampering or malicious behavior. The system operates by first obtaining result data generated by a model for current input data, which may include intermediate or final output results. Simultaneously, in a trusted execution environment, the system computes second result data by applying the same model to a plurality of trusted samples. This second result data serves as a reference baseline for normal model behavior. A generative adversarial network (GAN) is then trained using the second result data. The GAN consists of a generator and a discriminator. The discriminator is trained within the trusted execution environment using the GAN framework, the model, and the trusted samples. The generator is trained based on the second result data, synthetic input data derived from the second result data, and the GAN framework. The discriminator is used to compare the distributions of the current result data and the trusted second result data. By analyzing these distributions, the system detects whether the model is currently secure, identifying any deviations that may indicate security breaches. The system then outputs a security detection result, indicating whether the model is secure or compromised. This approach ensures robust model security by leveraging trusted execution environments and adversarial learning techniques.
17. The computer-implemented system according to claim 16 , wherein generating data to be input to the generator further comprises: generating data to be input to the generator based on random data.
The system relates to computer-implemented methods for generating synthetic data, particularly in the context of machine learning or data augmentation. The problem addressed is the need for high-quality synthetic data that can be used to train or test machine learning models, especially when real-world data is scarce, expensive, or difficult to obtain. The system generates synthetic data by leveraging a generator model, which is a type of neural network capable of producing realistic data samples based on learned patterns. The system includes a generator model that creates synthetic data by processing input data. The input data is generated based on random data, such as noise or random vectors, which the generator transforms into structured, meaningful outputs. This random data serves as a seed for the generator, allowing it to produce diverse and varied synthetic samples. The use of random data ensures that the generated outputs are not merely copies of existing data but instead represent new, unique variations that can be used for training or evaluation purposes. The system may also include additional components, such as a discriminator model, which evaluates the quality of the generated data by distinguishing between real and synthetic samples. This adversarial training process helps refine the generator to produce more realistic and high-quality synthetic data. The overall approach improves data availability for machine learning applications while maintaining the integrity and diversity of the generated samples.
Unknown
June 23, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.