A method for assertion-based formal verification includes executing a plurality of formal verification regression runs on a model of an electronic design; for each of the regression runs, using a unique signature function, calculating and saving a unique signature value for each instantiation of a property of a plurality of properties of the model of the electronic design and a status result for that instantiation of the property in that regression run; and signing off a current version of the model of the electronic device and presenting as a status result for each the instantiations of a plurality of the properties of the current version of the model of the electronic design the preferred status result obtained for that instantiation of the property per the same unique signature value that was calculated for that instantiation of the property in previous runs of the plurality of formal verification regression runs.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
2. The method of claim 1, wherein signing off the current version of the model is possible only if that current version of the model does not include a counter example as a status result.
This invention relates to model validation in machine learning systems, specifically addressing the challenge of ensuring model reliability by preventing the approval of models that contain counterexamples—inputs for which the model produces incorrect or undesirable outputs. The method involves a validation process where a model's current version is evaluated before being signed off or deployed. During this evaluation, the system checks whether the model includes any counterexamples as part of its status results. If a counterexample is detected, the model is not approved, ensuring that only validated, error-free versions are released. This approach enhances model accuracy and reliability by enforcing strict quality control, particularly in critical applications where incorrect predictions could have significant consequences. The method integrates seamlessly into existing machine learning workflows, providing an automated safeguard against deploying flawed models. By systematically rejecting models with counterexamples, the system reduces the risk of errors in production environments, improving overall system performance and user trust. The invention is particularly useful in domains such as healthcare, finance, and autonomous systems, where model accuracy is paramount.
4. The system of claim 3, wherein signing off the current version of the model is possible only if that current version of the model does not include a counter example as a status result.
This invention relates to a model management system that enforces strict version control by preventing the signing off of a model version if it contains a counterexample in its status results. The system ensures that only validated, error-free model versions are approved for deployment. The core functionality involves tracking model versions, analyzing their status results for counterexamples, and blocking the signing-off process if any counterexample is detected. This prevents the release of flawed models, improving reliability in applications such as machine learning, software testing, or simulation environments. The system integrates with a broader model management framework that includes version tracking, status monitoring, and approval workflows. By enforcing this validation step, the system reduces the risk of deploying models with known failures, ensuring higher quality and trust in the deployed versions. The invention is particularly useful in automated testing pipelines or continuous integration/continuous deployment (CI/CD) workflows where model integrity is critical. The counterexample detection mechanism may involve pattern matching, rule-based checks, or statistical analysis to identify invalid or inconsistent results. The system may also log and report counterexamples to facilitate debugging and model refinement.
6. The non-transitory computer readable storage medium of claim 5, wherein signing off the current version of the model is possible only if that current version of the model does not include a counter example as a status result.
A system for managing machine learning model versions ensures that only validated models are signed off for deployment. The system tracks model versions and their associated status results, including whether any counter examples (test cases that fail) are present. A model version can only be signed off if it does not contain any counter examples, preventing deployment of unvalidated models. The system may also include a process for generating and storing model versions, where each version is evaluated against test data to determine its status. If a counter example is detected, the model version is flagged and cannot be approved for deployment. This ensures that only models meeting quality standards are released, reducing the risk of deploying faulty or inaccurate models in production environments. The system may integrate with model training pipelines, version control systems, and deployment workflows to automate validation and approval processes. By enforcing strict validation criteria, the system improves reliability and trust in machine learning deployments.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
June 2, 2021
December 6, 2022
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.