Method and Apparatus for Achieving Consistent Read Latency from an Array of Solid-State Storage Devices

1. A method comprising: storing data in a storage subsystem formed from a plurality of non-volatile solid-state memory devices arranged in an N×M array of non-volatile solid-state memory elements, where M is at least three (3); applying a parity based redundancy scheme to the non-volatile solid-state memory elements, including defining N redundancy groups from among the plurality of non-volatile solid-state memory elements, each said redundancy group including M nonvolatile solid-state memory elements; scheduling operations on the storage subsystem so that erasure is limited to a predetermined sized subset of the non-volatile solid-state memory elements at a time in each said redundancy group; in response to a request to read a first set of data, identifying a first non-volatile solid-state memory element in which at least a portion of the first set of data is stored, the first non-volatile solid-state memory element being a member of a first redundancy group of the N redundancy groups; determining whether the first non-volatile solid-state memory element is busy; in response to a determination that the first non-volatile solid-state memory element is not busy, reading the first set of data from the first non-volatile solid-state memory element; in response to a request to read a second set of data, identifying a second non-volatile solid-state memory element in which at least a portion of the second set of data is stored, the second non-volatile solid-state memory element being a member of a second redundancy group of the N redundancy groups; determining whether the second non-volatile solid-state memory element is busy; and in response to a determination that the second non-volatile solid-state memory element is busy, reading information from the other non-volatile solid-state memory elements in the second redundancy group and reconstructing the second set of data from the information read from the other non-volatile solid-state memories in the second redundancy group.

2. A method as recited in claim 1 , wherein said scheduling operations comprises scheduling operations so that erasure is not allowed on all of the non-volatile solid-state memory elements at a time in each redundancy group of the array.

3. A method as recited in claim 1 , wherein a memory element is considered to be busy if the memory element is performing an erase or write.

4. A method as recited in claim 1 , wherein the plurality of non-volatile solid-state memory elements comprises a plurality of flash memory elements.

5. A method as recited in claim 1 , wherein each said memory element is a memory chip.

6. A method as recited in claim 1 , wherein each said memory element is an independent plane or group of planes of a memory chip.

7. A method as recited in claim 1 , wherein each said memory element is a module comprising a plurality of memory chips.

8. A storage system comprising: a communication interface through which to communicate with an external host via a network; a storage interface through which to access a plurality of non-volatile solid-state memory devices arranged as an N×M array of non-volatile solid-state memory elements, where M is at least three (3); and a processor which, in operation, performs a set of operations including applying a parity based redundancy scheme to the array, including defining N redundancy groups from among the plurality of non-volatile solid-state memory elements, each said redundancy group including M nonvolatile solid-state memory elements; scheduling operations on the storage subsystem so that erasure is limited to a predetermined sized subset of the non-volatile solid-state memory elements at a time in each said redundancy group; in response to a request from the host to read a first set of data, identifying a first non-volatile solid-state memory element in which at least a portion of the first set of data is stored, the first non-volatile solid-state memory element being a member of a first redundancy group of the N redundancy groups; determining whether the first non-volatile solid-state memory element is busy; in response to a determination that the first non-volatile solid-state memory element is not busy, reading the first set of data from the first non-volatile solid-state memory element; in response to a request to read a second set of data, identifying a second non-volatile solid-state memory element in which at least a portion of the second set of data is stored, the second non-volatile solid-state memory element being a member of a second redundancy group of the N redundancy groups; determining whether the second non-volatile solid-state memory element is busy; and in response to a determination that the second non-volatile solid-state memory element is busy, reading information from the other non-volatile solid-state memory elements in the second redundancy group and reconstructing the second set of data from the information read from the other non-volatile solid-state memories in the second redundancy group.

9. A storage system as recited in claim 8 , wherein said scheduling operations comprises scheduling operations so that erasure is not allowed on all of the non-volatile solid-state memory elements at a time in each redundancy group of the array.

10. A storage system as recited in claim 8 , wherein a memory element is considered to be busy if the memory element is performing an erase or write.

11. A storage system as recited in claim 8 , wherein the plurality of non-volatile solid-state memory elements comprises a plurality of flash memory elements.

12. A storage system as recited in claim 8 , wherein each said memory element is a memory chip.

13. A storage system as recited in claim 8 , wherein each said memory element is an independent plane or group of planes of a memory chip.

14. A storage system as recited in claim 8 , wherein each said memory element is a module comprising a plurality of memory chips.