Described technologies facilitate cloud validation using validation as a service (VaaS). A cloud validation service provider acquires and securely stores certification tests developed by cloud component providers, integrated solution providers, and others. Each test's executable portion tests hardware or software of a candidate cloud. The candidate may be on the premises of an enterprise, or instead be a hosted cloud on the premises of a hoster off the premises of the entity that pays for the hosting. Monitored testing is done using an infrastructure in the candidate cloud or in a public cloud. Results are uploaded to the VaaS provider, which provides an analysis of test results for use in determining whether to validate the candidate cloud. Test execution agents may be VaaS-cloud-resident or candidate-cloud-resident, and may use a mutex to prevent simultaneous execution of tests. Testing may be accomplished even when the candidate cloud has no internet-exposed communication endpoint.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method supporting validation of a candidate cloud, the method suitable to be performed by or on behalf of a validation service provider, comprising: one or more validation servers outside the candidate cloud acquiring from at least one source other than the validation service provider, and storing in a secured location, at least one digital certification test, each digital certification test having an executable portion which is capable, upon execution, of testing hardware that resides in the candidate cloud or testing software that resides in the candidate cloud or testing both; supplying a certification test execution infrastructure which is capable of executing one or more certification test executable portions; monitoring at least one execution of at least one certification test executable portion; receiving at least one certification test result which was generated by execution of one or more certification test executable portions, wherein receiving at least one certification test result comprises receiving one or more results which individually or collectively specify a respective value or status for at least four of the following tested items: throughput measured in input/output operations per second, throughput measured in megabytes per second, transactional throughput, streaming throughput, uptime, virtual machine redistribution impact, load rebalancing impact, patching impact, upgrade installation impact, hardware replacement impact, corrupt storage mitigation impact, power fluctuation impact, power loss impact, server hardware failure impact, concurrent virtual machine deployment impact, backup impact, backup restoration impact, node reboot impact, network interface failure impact, storage capacity threshold alert, virtual machine live migration, storage migration, compute node failure, storage node failure, non-shared disk failure, shared disk failure, network cable failure, network switch failure or error, router failure or error, firewall failure or error, host bus adapter failure; and providing one or more certification test results for use in making a determination whether to validate the candidate cloud.
2. The method of claim 1 , wherein supplying a certification test execution infrastructure comprises supplying the certification test execution infrastructure in a cloud location which is outside the candidate cloud, wherein the certification test executable portion executes in said cloud location outside the candidate cloud, and the certification test executable portion sends requests to endpoints of the candidate cloud which are exposed over the internet.
3. The method of claim 1 , wherein supplying a certification test execution infrastructure comprises furnishing a downloadable test execution agent, wherein the certification test executable portion executes in conjunction with execution of the test execution agent inside the candidate cloud, and the test execution agent makes only one or more outbound connections from the candidate cloud to the one or more validation servers, thereby avoiding reliance on having endpoints of the candidate cloud exposed over the internet.
4. The method of claim 1 , wherein the validation server also acquires at least one digital certification test from the validation service provider, said digital certification test from the validation service provider having an executable portion which is capable upon execution of testing hardware that resides in the candidate cloud or testing software that resides in the candidate cloud or testing both.
5. The method of claim 1 , wherein the validation server also acquires at least one test from at least one of the following: an integrated solution provider, a candidate cloud operator, and wherein said test is not required for validation of the candidate cloud.
6. The method of claim 1 , wherein the following steps are performed automatically without human intervention: monitoring at least one execution of at least one certification test executable portion, receiving at least one certification test result which was generated by execution of one or more certification test executable portions, and providing one or more certification test results for use in making a determination whether to validate the candidate cloud.
7. The method of claim 1 , wherein the validation servers respond to requests from the candidate cloud but do not initiate any traffic inbound to candidate cloud.
8. A system supporting validation testing of a candidate cloud, the system comprising: a validation service interface hosted on a public cloud outside the candidate cloud, the validation service interface including a test upload interface for uploading certification tests; at least one of the following certification test execution infrastructures: a cloud-resident test execution agent and a testing environment configured to run the cloud-resident test execution agent, the testing environment at least partially hosted in the public cloud; a downloadable test execution agent configurable to run in the candidate cloud; and a validation server which includes at least one processor and a memory, the memory containing and configured by software which upon execution (a) coordinates performance of certification tests using at least one test execution agent, (b) receives certification test results generated by said performance, and (c) provides an analysis of the certification test results.
9. The system of claim 8 , wherein the system includes the downloadable test execution agent configurable to run in the candidate cloud, and at least one of the following conditions is satisfied: the downloadable test execution agent is configured to communicate with the validation server using out-bound connectivity from the candidate cloud to the public cloud via the internet; the downloadable test execution agent is fault-resilient; the downloadable test execution agent is configured to automatically download from the public cloud the most recent version of a certification test before executing that certification test.
10. The system of claim 8 , wherein the system includes a validation service store and also includes a secure access token service that limits access to the validation service store, the validation service store containing one or more of the following: uploaded certification tests, resources of the testing environment, certification test results, analysis of the certification test results.
11. The system of claim 10 , wherein the downloadable test execution agent includes code which upon execution obtains a secure access token from the secure access token service and utilizes the secure access token to communicate directly with the validation service store to discover one or more certification tests to execute in the candidate cloud.
12. The system of claim 8 , wherein at least one test execution agent includes code which implements a distributed mutex to ensure only a single certification test executes at a given time.
13. The system of claim 8 , wherein the analysis of the certification test results includes an analytics page which is available to a cloud validation service operator to download and view.
14. The system of claim 8 , wherein the system is configured to run all certification tests using the cloud-resident test execution agent and testing environment.
15. A method supporting validation of a candidate cloud, the method suitable to be performed by or on behalf of a cloud validation service operator, the method comprising: authenticating to a validation service interface; downloading via the validation service interface a test execution agent; giving the test execution agent access to run in the candidate cloud; and obtaining via the validation service interface a result of a certification test after the test execution agent runs in the candidate cloud and executes at least a portion of the certification test in the candidate cloud.
16. The method of claim 15 , wherein the method is performed by an integrated solution provider as the cloud validation service operator, and wherein the test execution agent automatically downloads certification tests, automatically executes in the candidate cloud executable portions of the downloaded certification tests, and automatically uploads results of the certification test executions to a validation service associated with the validation service interface.
17. The method of claim 15 , wherein the method is performed in the candidate cloud without accessing any candidate cloud endpoints from the internet.
18. The method of claim 15 , wherein the candidate cloud is hosted for an entity by a hoster, the candidate cloud is located on a premises of the hoster as opposed to being located on a premises of the entity, and the method is performed by the hoster as the cloud validation service operator.
19. The method of claim 18 , wherein the hoster avoids downloading any package to the entity premises, and avoids setting up any test infrastructure on the entity premises.
20. The method of claim 1 , wherein the candidate cloud does not expose any communication endpoint on the internet.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
January 8, 2018
September 3, 2019
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.