Data Set Capture Management with Forecasting

1. A computer-implemented method for data set capture management, comprising: under control of one or more computer systems configured with executable instructions, maintaining a collection of data set capture policies including at least one data set capture policy specifying a flexible schedule with respect to capture of a data set; forecasting a utilization load of at least one of a set of resources of a program execution service, the forecasting including determining at least one time period of resource under-utilization with respect to a resource utilization target, the set of resources including a first data store maintaining at least the data set, and a network at least connecting the first data store to a second data store configured at least to store at least one capture of the data set for later inspection; and scheduling at least one capture of the data set based at least in part on said at least one time period of resource under-utilization while satisfying one or more scheduling constraints specified by said at least one data set capture policy, the scheduling arranged at least to optimize utilization of the set of resources of the program execution service, the optimization based at least in part on the collection of data set capture policies and the forecast of the utilization load of said at least one of the set of resources.

2. A computer-implemented method according to claim 1 , wherein specifying a flexible schedule with respect to capture of the data set comprises specifying a window of time during which to perform a capture.

3. A computer-implemented method according to claim 1 , wherein specifying a flexible schedule with respect to capture of the data set comprises specifying a target frequency of captures of the data set.

4. A computer-implemented method according to claim 1 , wherein specifying a flexible schedule with respect to capture of the data set comprises specifying a target probability of loss of at least a portion of the data set from both the first data store and the second data store during a time period.

5. A computer-implemented method according to claim 1 , wherein specifying a flexible schedule with respect to capture of the data set comprises specifying a target cost incurred by performing one or more captures in accordance with the flexible schedule.

6. A computer-implemented method according to claim 1 , wherein the forecast of the utilization load of said at least one of the set of resources of the program execution service is based at least in part on at least one historical pattern of utilization of said at least one of the set of resources by a plurality of users of the program execution service.

7. A computer-implemented method according to claim 1 , wherein the first data store has a low access latency relative to the second data store.

8. A computer-implemented method according to claim 1 , wherein the second data store has a high data durability relative to the first data store.

9. A computer-implemented method according to claim 1 , wherein the optimization further comprises: forecasting at least one time period of resource over-utilization with respect to a resource utilization target; and scheduling said at least one capture of the data set so as to avoid said at least one time period of resource over-utilization while satisfying the one or more scheduling constraints specified by said at least one data set capture policy.

10. A computer-implemented method according to claim 1 , further comprising scheduling at least one pre-capture transfer of at least a portion of the data set to further optimize the utilization of the set of resources of the program execution service.

11. A computing system, comprising: at least one processor; and memory including instructions that, when executed by the at least one processor, cause the computing system to: maintain a collection of data set capture policies including at least one data set capture policy specifying a flexible schedule with respect to capture of a data set; forecast a utilization load of at least one of a set of resources of a program execution service, the forecasting including determining at least one time period of resource under-utilization with respect to a resource utilization target, the set of resources including a first data store maintaining at least the data set, and a network at least connecting the first data store to a second data store configured at least to store at least one capture of the data set for later inspection; and schedule at least one capture of the data set based at least in part on said at least one time period of resource under-utilization while satisfying one or more scheduling constraints specified by said at least one data set capture policy, the scheduling arranged at least to optimize utilization of the set of resources of the program execution service, the optimization based at least in part on the collection of data set capture policies and the forecast of the utilization load of said at least one of the set of resources.

12. The computing system of claim 11 , wherein specifying a flexible schedule with respect to capture of the data set comprises specifying a window of time during which to perform a capture.

13. The computing system of claim 11 , wherein specifying a flexible schedule with respect to capture of the data set comprises specifying a target frequency of captures of the data set.

14. The computing system of claim 11 , wherein specifying a flexible schedule with respect to capture of the data set comprises specifying a target probability of loss of at least a portion of the data set from both the first data store and the second data store during a time period.

15. The computing system of claim 11 , wherein specifying a flexible schedule with respect to capture of the data set comprises specifying a target cost incurred by performing one or more captures in accordance with the flexible schedule.

16. The computing system of claim 11 , wherein the forecast of the utilization load of said at least one of the set of resources of the program execution service is based at least in part on at least one historical pattern of utilization of said at least one of the set of resources by a plurality of users of the program execution service.

17. A non-transitory computer readable storage medium storing one or more sequences of instructions that, when executed by one or more processors, cause a computing system to: maintain a collection of data set capture policies including at least one data set capture policy specifying a flexible schedule with respect to capture of a data set; forecast a utilization load of at least one of a set of resources of a program execution service, the forecasting including determining at least one time period of resource under-utilization with respect to a resource utilization target, the set of resources including a first data store maintaining at least the data set, and a network at least connecting the first data store to a second data store configured at least to store at least one capture of the data set for later inspection; and schedule at least one capture of the data set based at least in part on said at least one time period of resource under-utilization while satisfying one or more scheduling constraints specified by said at least one data set capture policy, the scheduling arranged at least to optimize utilization of the set of resources of the program execution service, the optimization based at least in part on the collection of data set capture policies and the forecast of the utilization load of said at least one of the set of resources.

18. The non-transitory computer readable storage medium of claim 17 , wherein specifying a flexible schedule with respect to capture of the data set comprises specifying a window of time during which to perform a capture.

19. The non-transitory computer readable storage medium of claim 17 , wherein specifying a flexible schedule with respect to capture of the data set comprises specifying a target frequency of captures of the data set.

20. The non-transitory computer readable storage medium of claim 17 , wherein specifying a flexible schedule with respect to capture of the data set comprises specifying a target probability of loss of at least a portion of the data set from both the first data store and the second data store during a time period.