Storage System Io Throttling Utilizing a Reinforcement Learning Framework

2. The apparatus of claim 1 wherein the two or more IO performance metric values for the storage system comprise at least IO operations per second (IOPS) and throughput.

3. The apparatus of claim 1 wherein the current state of the storage system is further characterized by a combination of storage system information, runtime performance information including the two or more IO performance metric values, and IO pattern combination information for a particular time period.

4. The apparatus of claim 1 wherein the state-action record comprises at least a tuple (St, At, Rt+1, St+1) and utilizing the state-action record to update learned experience of the reinforcement learning framework comprises utilizing the state-action record to update at least learned experience Q(Si, Ai) of the reinforcement learning framework, where St denotes the current state, At denotes a selected action of the IO throttling recommendation, Rt+1 denotes a reward for executing the selected action in the storage system, and St+1 denotes the subsequent state.

5. The apparatus of claim 1 wherein the determining, generating and updating are implemented in at least one of server and a host device that are external to the storage system, and applying the IO throttling recommendation to the storage system comprises sending an IO throttling action recommendation to the storage system in response to an IO throttling request received from the storage system.

6. The apparatus of claim 1 wherein the determining, generating, applying and updating are implemented within the storage system, and applying the IO throttling recommendation to the storage system comprises executing an IO throttling action recommendation within the storage system.

7. The apparatus of claim 1 wherein generating the IO throttling recommendation for the storage system comprises determining whether the current state of the storage system matches any of a plurality of state-action records of learned experience maintained by the reinforcement learning framework.

8. The apparatus of claim 7 wherein, responsive to determining that the current state of the storage system does not match any of the plurality of state-action records, randomly selecting an action from an action space, the action space defining a plurality of available IO throttling actions in accordance with an IO throttling policy.

9. The apparatus of claim 7 wherein, responsive to determining that the current state of the storage system matches a given one of the plurality of state-action records, utilizing a probability value of an exploitation and exploration tradeoff parameter to control performance of a particular one of at least first and second different action selections.

10. The apparatus of claim 9 wherein the first action selection comprises selecting a first action specified in the given one of the plurality of state-action records matching the current state of the storage system.

11. The apparatus of claim 9 wherein the second action selection comprises randomly selecting an action from an action space, the action space defining a plurality of available IO throttling actions in accordance with an IO throttling policy.

12. The apparatus of claim 1 wherein the reinforcement learning framework implements a reward function that is configured to control selection of actions that guide the storage system toward one or more specified performance goals.

13. The apparatus of claim 12 wherein the reward function is computed as a weighted combination of first and second functions based at least in part on respective ones of the first and second IO performance metric values.

14. The apparatus of claim 12 wherein the reward function is computed as a weighted combination of a first function indicative of an improvement in average latency relative to an initial latency and a second function indicative of an improvement in average throughput relative to an initial throughput.

16. The computer program product of claim 15 wherein the current state of the storage system is further characterized by a combination of storage system information, runtime performance information including the two or more IO performance metric values, and IO pattern combination information for a particular time period.

17. The computer program product of claim 15 wherein generating the IO throttling recommendation for the storage system comprises determining whether the current state of the storage system matches any of a plurality of state-action records of learned experience maintained by the reinforcement learning framework.

19. The method of claim 18 wherein the current state of the storage system is further characterized by a combination of storage system information, runtime performance information including the two or more IO performance metric values, and IO pattern combination information for a particular time period.

20. The method of claim 18 wherein generating the IO throttling recommendation for the storage system comprises determining whether the current state of the storage system matches any of a plurality of state-action records of learned experience maintained by the reinforcement learning framework.