An electronic system includes: a product retriever configured to access a product; an image capturer configured to capture one or more images of the product; and one or more processing units configured to generate prompts for input to a neural network, wherein the prompts are configured to prompt the neural network to determine a feature of the product based on at least one of the one or more captured images, determine a suggested testing for the feature of the product, perform a verification to determine whether the suggested testing for the feature of the product can be performed, and generate product testing instruction after the verification is performed.
Legal claims defining the scope of protection, as filed with the USPTO.
. An electronic system, comprising:
. The electronic system of, wherein the prompts comprise a first set of prompts, and wherein the one or more processing units comprises a first prompt generator configured to generate the first set of prompts to cause the neural network to determine the feature of the product based on the at least one of the one or more captured images.
. The electronic system of, wherein the prompts comprise a second set of prompts, and wherein the one or more processing unit comprises a second prompt generator configured to generate the second set of prompts to cause the neural network to perform the verification to determine whether the suggested testing for the feature of the product can be performed.
. The electronic system of, wherein the prompts comprise a third set of prompts, and wherein the one or more processing units comprises a third prompt generator configured to generate the third set of prompts to cause the neural network to generate the product testing instruction.
. The electronic system of, wherein the one or more processing units are configured to obtain OCR data and/or DOM data, and to provide the OCR data and/or the DOM data along with the third set of prompts to cause the neural network to generate the product testing instruction based on at least a part of the OCR data and/or at least a part of the DOM data.
. The electronic system of, wherein the neural network is separate from the one or more processing units, and wherein the one or more processing units are configured to access the neural network.
. The electronic system of, wherein the one or more processing units comprise the neural network.
. The electronic system of, wherein the neural network comprises a plurality of neural network models, and wherein the one or more processing units are configured to access respective ones of the neural network models.
. The electronic system of, wherein the neural network comprises a plurality of neural network models, and wherein the one or more processing units comprise respective ones of the neural network models.
. The electronic system of, wherein the neural network comprises ChatGPT.
. The electronic system of, wherein the one or more processing units are configured to obtain feedback from the neural network indicating that the suggested testing for the feature of the product cannot be performed.
. The electronic system of, wherein the one or more processing units are configured to generate additional prompts based on the feedback to cause the neural network to determine another feature of the product and/or to determine another suggested testing.
. The electronic system of, further comprising:
. The electronic system of, wherein the product testing device is configured to move a cursor without input from a cursor control.
. The electronic system of, wherein the product testing device is configured to make a selection of an object without input from a cursor control.
. The electronic system of, wherein the product testing device is configured to insert a text in a field without input from a keyboard.
. The electronic system of, wherein the product testing device comprises an interpreter configured to interpret the product testing instruction.
. The electronic system of, wherein the one or more processing units are configured to access a first set of items and a second set of items for the neural network or to provide the first set of items and the second set of items for access by the neural network, wherein the first set of items comprises a plurality of action identifiers, and the second set of items comprises a plurality of objects.
. The electronic system of, wherein one of the action identifiers identifies an action to be performed by the product testing device, and one of the object identifiers identifies an object on which the action is to be performed by the product testing device.
. The electronic system of, wherein one of the action identifiers identifies a click action, a fill action, a type action, a press key action, a hover action, a dropdown select action, a checkbox check action, a checkbox uncheck action, a refresh action, a navigate action, a new tab action, a close tab action, a scroll cation, a drag and drop action, or a click and hold action.
. The electronic system of, wherein one of the object identifiers identifies a button, a field, a dropdown menu, a dropdown option, a link, an icon, a checkbox, a header, a window, a text, a modal, or an user interface element.
. The electronic system of, wherein the product testing instruction has a data structure that associates an action identifier with a corresponding object identifier:
. The electronic system of, wherein the action identifier identifies an action to be performed by the product testing device, and the object identifier identifies an object on which the action is to be performed by the product testing device.
. The electronic system of, wherein the action identifier identifies a click action, a fill action, a type action, a press key action, a hover action, a dropdown select action, a checkbox check action, a checkbox uncheck action, a refresh action, a navigate action, a new tab action, a close tab action, a scroll cation, a drag and drop action, or a click and hold action.
. The electronic system of, wherein the object identifier identifies a button, a field, a dropdown menu, a dropdown option, a link, an icon, a checkbox, a header, a window, a text, a modal, or an user interface element.
. The electronic system of, further comprising a non-transitory medium storing the product testing instruction in association with an identity of the product.
. The electronic system of, wherein the product testing device is configured to check if an element is visible after the product testing device performs a testing action.
. The electronic system of, wherein the product testing device is configured to check if an element is not visible after the product testing device performs a testing action.
. The electronic system of, wherein the product testing device is configured to check if a specified page has loaded after the product testing device performs a testing action.
. The electronic system of, wherein the product testing device is configured to:
. The electronic system of, wherein the first image is based on a completion of the product testing task performed during the testing of the product.
. The electronic system of, wherein the product comprises a web page, a web site, a computer application, a mobile device application, or a processor application.
. The electronic system of, wherein the one or more processing units comprise an action retriever configured to obtain past testing information regarding past human-testing action and/or past machine-testing action, and to provide the past testing information to the neural network for guiding an operation of the neural network.
. A method performed by an electronic system, the method comprising:
-. (canceled)
Complete technical specification and implementation details from the patent document.
This application relates generally to product testing, and more specifically, to systems and methods for providing product testing instruction and for providing automated product testing.
Many products require product testing. Supplier of products generally require products be tested before they can be made available to customers. Sometimes, after a product has been made available, it may still be desirable to test the product. Testing of products can be a lengthy and complicated process. There may be many different technical features in the product that need to be tested under different testing parameters. Many product suppliers may not have in-house product testers. These product suppliers will need to outsource product testing to outside product testers. However, finding the right product testers who can adequately perform the product testing project can itself be a time-consuming process, and can be challenging. If the product supplier needs fast turn-around time for the product testing for its product, such out-sourcing technique may not be suitable.
In some cases, product testing device may be provided to perform product testing automatically. However, this requires user to create product testing instruction for execution by the product testing device. Creating product testing instruction is a time-consuming and labor-intensive process. Sometimes it may take a long time to generate product testing instruction. Even if a set of workable product testing instruction is created for a product, the product may change overtime (due to the product provider adjusting a feature of the product, adding a feature of the product, or removing a feature of the product). Thus, the created product testing instruction may not work properly for the changed product. Debugging and adjusting the product testing instruction are also expensive processes.
An electronic system includes: a product retriever configured to access a product; an image capturer configured to capture one or more images of the product; one or more processing units configured to obtain the one or more images captured by the image capturer, determine a feature of the product based on at least one of the one or more captured images, determine a suggested testing for the feature of the product, perform a verification to determine whether the suggested testing for the feature of the product can be performed, and obtain product testing instruction that is generated automatically after the verification is performed.
Optionally, the one or more processing units comprises a first processing unit configured to determine the feature of the product based on the at least one of the one or more captured images.
Optionally, the first processing unit is configured to determine the feature of the product utilizing a first neural network.
Optionally, the first processing unit is configured to determine the feature of the product by accessing the first neural network.
Optionally, the first neural network is a part of the first processing unit.
Optionally, the first neural network comprises ChatGPT.
Optionally, the one or more processing unit comprises a second processing unit configured to perform the verification to determine whether the suggested testing for the feature of the product can be performed.
Optionally, the second processing unit is configured to utilize the first neural network to perform the verification to determine whether the suggested testing for the feature of the product can be performed.
Optionally, the second processing unit is configured to perform the verification to determine whether the suggested testing for the feature of the product can be performed by accessing the first neural network.
Optionally, the first neural network is a part of the first processing unit, and a part of the second processing unit.
Optionally, the second processing unit is configured to utilize a second neural network to perform the verification to determine whether the suggested testing for the feature of the product can be performed.
Optionally, the second processing unit is configured to provide feedback to the first processing unit if the second processing unit determines that the suggested testing for the feature of the product cannot be performed.
Optionally, the first processing unit is configured to determine another feature of the product and/or to determine another suggested testing based on the feedback, and wherein the first processing unit is configured to determine the other feature of the product and/or to determine the other suggested testing utilizing the first neural network.
Optionally, the one or more processing units comprises a third processing unit configured to obtain the product testing instruction.
Optionally, the third processing unit comprises a product testing instruction generator, or is configured to interface with the product testing instruction generator.
Optionally, the third processing unit is configured to obtain OCR data and DOM data, and wherein the third processing unit is configured to obtain the product testing instruction based on at least a part of the OCR data and at least a part of the DOM data.
Optionally, the first processing unit, the second processing unit, and the third processing unit are configured to access the first neural network.
Optionally, the first neural network forms a part of the first processing unit, a part of the second processing unit, and a part of the third processing unit.
Optionally, the electronic system further includes: a product testing device configured to execute the product testing instruction to perform testing of the product based on the product testing instruction; wherein the product testing device is configured to perform the testing of the product by simulating human actions based on the product testing instruction.
Optionally, the product testing device is configured to move a cursor without input from a cursor control.
Optionally, the product testing device is configured to make a selection of an object without input from a cursor control.
Optionally, the product testing device is configured to insert a text in a field without input from a keyboard.
Optionally, the product testing device comprises an interpreter configured to interpret the product testing instruction.
Optionally, the one or more processing units are configured to access a first set of items and a second set of items, wherein the first set of items comprises a plurality of action identifiers, and the second set of items comprises a plurality of objects.
Optionally, one of the action identifiers identifies an action to be performed by the product testing device, and one of the object identifiers identifies an object on which the action is to be performed by the product testing device.
Optionally, one of the action identifiers identifies a click action, a fill action, a type action, a press key action, a hover action, a dropdown select action, a checkbox check action, a checkbox uncheck action, a refresh action, a navigate action, a new tab action, a close tab action, a scroll cation, a drag and drop action, or a click and hold action.
Optionally, one of the object identifiers identifies a button, a field, a dropdown menu, a dropdown option, a link, an icon, a checkbox, a header, a window, a text, a modal, or an user interface element.
Optionally, the product testing instruction has a data structure that associates an action identifier with a corresponding object identifier:
Optionally, the action identifier identifies an action to be performed by the product testing device, and the object identifier identifies an object on which the action is to be performed by the product testing device.
Optionally, the action identifier identifies a click action, a fill action, a type action, a press key action, a hover action, a dropdown select action, a checkbox check action, a checkbox uncheck action, a refresh action, a navigate action, a new tab action, a close tab action, a scroll cation, a drag and drop action, or a click and hold action.
Optionally, the object identifier identifies a button, a field, a dropdown menu, a dropdown option, a link, an icon, a checkbox, a header, a window, a text, a modal, or an user interface element.
Optionally, the electronic system further includes a non-transitory medium storing the product testing instruction in association with an identity of the product.
Optionally, the product testing device is configured to check if an element is visible after the product testing device performs a testing action.
Optionally, the product testing device is configured to check if an element is not visible after the product testing device performs a testing action.
Optionally, the product testing device is configured to check if a specified page has loaded after the product testing device performs a testing action.
Optionally, the product testing device is configured to: obtain a first image that is associated with the testing of the product, obtain a second image, the second image being a reference image that is pre-determined before the first image is obtained, perform a comparison based on the first image and the second image, and determine whether the product passes or fails a product testing task based on a result of the comparison.
Optionally, the first image is based on a completion of the product testing task performed during the testing of the product.
Optionally, the product comprises a web page, a web site, a computer application, a mobile device application, or a processor application.
Optionally, the product testing device is configured to perform machine-based testing of the product.
Optionally, the one or more processing units are implemented as a prompt generator interfacing with a neural network.
Optionally, the electronic system further includes an action retriever configured to obtain past testing information regarding past human-testing action and/or past machine-testing action, and to provide the past testing information to the one or more processing units for guiding an operation of the one or more processing units.
A method performed by an electronic system, includes: accessing a product; capturing one or more images of the product by an image capturer; obtaining the one or more images captured by the image capturer; determining a feature of the product based on at least one of the one or more captured images; determining a suggested testing for the feature of the product; performing a verification to determine whether the suggested testing for the feature of the product can be performed; and obtaining product testing instruction that is generated automatically after the verification is performed.
Optionally, the feature of the product is determined by a first processing unit of the electronic system based on the at least one of the one or more captured images.
Optionally, the feature of the product is determined by the first processing unit utilizing a first neural network.
Optionally, the feature of the product is determined by the first processing unit accessing the first neural network.
Optionally, the first neural network is a part of the first processing unit.
Optionally, the first neural network comprises ChatGPT.
Unknown
November 27, 2025
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.