Zero Copy Writes Through Use of Mbufs

1. A method for partitioning data into fixed sized data blocks for storage on a storage device of a storage system having a storage operating system, comprising the following steps: (a) storing data received from a network in one or more data structures, the one or more data structures formed into a variable-length chain of data structures; (b) generating a set of buffer pointers that define a fixed sized block of the data stored in said variable-length chain of data structures; (c) storing said set of buffer pointers in a buffer header associated with the fixed sized block; and (d) repeating steps (b)–(c) until all of said received data is partitioned into fixed sized data blocks.

2. The method of claim 1 , wherein said set of buffer pointers is generated by addressing portions of said received data stored in one or more of said data structures.

3. The method of claim 1 , wherein each generated buffer header stores additional information related to its associated fixed sized data block.

4. The method of claim 3 , wherein said additional information includes at least one of a block number for identifying said associated fixed sized data block and a file identification information for identifying the contents of said associated fixed sized data block.

5. The method of claim 1 , wherein said received data is received by the storage system.

6. The method of claim 1 , wherein said fixed sized data blocks each have a fixed size of 4 kilobytes.

7. The method of claim 1 , further comprising the step: (f) manipulating said fixed sized data blocks, defined by said generated sets of buffer pointers, in a disk storage layer in the storage operating system.

8. The method of claim 1 , further comprising the step: (f) using each generated set of buffer pointers in a scatter/gather method to generate contiguous fixed sized data blocks.

9. The method of claim 8 , further comprising the step: (g) writing said contiguous fixed sized data blocks to a data storage medium.

10. A method for partitioning data into fixed sized data blocks for storage on a storage device of a storage system having a storage operating system, comprising the following steps: (a) storing data received from a network in one or more data structures; (b) generating a set of buffer pointers that define a fixed sized block of the received data; (c) storing said set of buffer pointers in a buffer header associated with the fixed sized block; (d) repeating steps (b)–(c) until all of said received data is partitioned into fixed sized data blocks; (e) manipulating said fixed sized data blocks, defined by said generated sets of buffer pointers, in a disk storage layer in the storage operating system; and (f) calculating at least one parity value in said disk storage layer for each of said fixed sized data blocks defined by said generated sets of buffer pointers.

11. A method for partitioning data into fixed sized data blocks for storage on a storage device of a storage system having a storage operating system, comprising the following steps: (a) storing data received from a network in one or more data structures; (b) generating a set of buffer pointers that define a fixed sized block of the received data; (c) storing said set of buffer pointers in a buffer header associated with the fixed sized block; and (d) repeating steps (b)–(c) until all of said received data is partitioned into fixed sized data blocks, wherein each of said data structures stores a pointer reference count that keeps track of the number of said generated buffer pointers that address portions of its stored data.

12. The method of claim 11 , wherein the storage operating system recycles a data structure when its pointer reference count is decremented to zero.

13. A storage operating system, comprising: (a) a first layer that determines a type of file access request associated with data received by the storage operating system, the received data being stored in one or more mbuf data structures, the one or more mbuf data structures formed into a variable-length chain of mbuf data structures; (b) a second layer that repeats the following steps until all of said received data is partitioned into fixed sized data blocks: (i) generating a set of buffer pointers that define a fixed sized data block by referencing portions of said received data stored in said variable-length chain of mbuf data structures; and (ii) storing said set of buffer pointers in a buffer header; and (c) a third layer that uses each generated set of buffer pointers in a scatter/gather method to generate contiguous fixed sized data blocks.

14. The storage operating system of claim 13 , further comprising: (d) a fourth layer that receives said data from a network and stores said data in said one or more mbuf data structures; and (e) a fifth layer that strips network headers from said received data.

15. The storage operating system of claim 14 , wherein said fourth layer is a media access layer.

16. The storage operating system of claim 14 , wherein said fifth layer comprises network communications and protocol layers.

17. The storage operating system of claim 13 , wherein said third layer writes said contiguous fixed sized data blocks to a data storage medium.

18. The storage operating system of claim 13 , wherein said first layer is a file system protocol layer.

19. The storage operating system of claim 13 , wherein said second layer is a file system layer.

20. The storage operating system of claim 13 , wherein said third layer is a disk driver layer.

21. A storage operating system of a filer that (i) receives data and an associated file access request from a network, (ii) stores said data in one or more mbuf data structures and (iii) partitions said data into fixed sized data blocks, said storage operating system comprising: (a) a first layer that determines the type of file access request associated with said received data; (b) a second layer that repeats the following steps until all of said received data is partitioned into fixed sized data blocks: (i) generating a set of buffer pointers that define a fixed sized data block by referencing portions of said received data stored in said one or more mbuf data structures; and (ii) storing said set of buffer pointers in a buffer header; and (c) a third layer that uses each generated set of buffer pointers in a scatter/gather method to generate contiguous fixed sized data blocks, wherein a fourth layer calculates at least one parity value for each of said fixed sized data blocks defined by said generated sets of buffer pointers.

22. The storage operating system of claim 21 , wherein said sixth layer is a RAID layer.

23. A storage operating system of a filer that (i) receives data and an associated file access request from a network, (ii) stores said data in one or more mbuf data structures and (iii) partitions said data into fixed sized data blocks, said storage operating system comprising: (a) a first layer that determines the type of file access request associated with said received data; (b) a second layer that repeats the following steps until all of said received data is partitioned into fixed sized data blocks: (iii) generating a set of buffer pointers that define a fixed sized data block by referencing portions of said received data stored in said one or more mbuf data structures; and (iv) storing said set of buffer pointers in a buffer header; and (c) a third layer that uses each generated set of buffer pointers in a scatter/gather method to generate contiguous fixed sized data blocks, wherein said storage operating system recycles at least one of said one or more mbuf data structures when none of said generated buffer pointers reference data in said at least one of said one or more mbuf data structures.

24. A memory buffer that receives data sent over a network to an interface of a filer, the memory buffer comprising: a data section; and a header section including a field that indicates the number of buffer pointers in a file system layer of a storage operating system that reference data stored in said data section, wherein the memory buffer is determined to be unused when the buffer pointers reference count equals zero.

25. The memory buffer of claim 24 , wherein the field is configured to store a pointer reference count that keeps track of the number of buffer pointers addressing portions of the data stored in the data section.

26. The memory buffer of claim 25 , wherein the memory buffer is determined to be unused when the pointer reference count equals zero.

27. An apparatus for partitioning data into fixed sized data blocks for storage on a storage device of a storage system having a storage operating system, the apparatus comprising: (a) means for storing data received from a network in one or more data structures, the one or more data structures formed into a variable-length chain of data structures; (b) means for generating a set of buffer pointers that define a fixed sized data block stored in said variable-length chain of data structures; (c) means for storing said set of buffer pointers in a buffer header associated with the fixed sized data block; and (d) means for repeating steps (b)–(c) until all of said received data is partitioned into fixed sized data blocks.

28. A computer-readable medium having instructions for execution on a processor, said instructions for a method of partitioning data into fixed sized data blocks comprising the following steps: (a) storing data received from a network in one or more data structures, the one or more data structures formed into a variable-length chain of data structures; (b) generating a set of buffer pointers that define a fixed sized data block stored in said variable-length chain of data structures; (c) storing said set of buffer pointers in a buffer header; and (d) repeating steps (b)–(c) until all of said received data is partitioned into fixed sized data blocks each having an associated buffer header.

29. A memory buffer, comprising: a data section; and a header section configured to store a pointer reference count that keeps track of the number of buffer pointers referencing data stored in a data section, wherein the memory buffer is determined to be unused when the pointer reference count equals zero.

30. A method for partitioning data into fixed sized data blocks in a storage system, the method comprising the steps of: storing data in one or more data structures, the one or more data structures formed into a variable-length chain of data structures; generating a set of pointers that collectively define a fixed sized block of the data stored in the variable-length chain of data structures; and processing said fixed sized data block using said set of pointers, without first copying said fixed sized data block into a fixed-sized data buffer.

31. The method of claim 30 , further comprising: storing said set of pointers in a header associated with the fixed sized data block.

32. A method for partitioning data into fixed sized data blocks in a storage system, the method comprising the steps of: storing data in one or more data structures, each data structure having a first fixed size; generating a set of pointers that collectively define a fixed sized block of the data stored in the one or more data structures, the fixed sized data block having a second fixed size that is different than the first fixed sized; and processing said fixed sized data block using said set of pointers, without first copying said fixed sized data block into a fixed-sized data buffer.

33. The method of claim 32 , further comprising: storing said set of pointers in a header associated with the fixed sized data block.

34. A network device, comprising: a processor; a network adapter that is adapted to receive data from a network; and a memory that is adapted to store one or more data structures, each data structure being configured to store at least a portion of the data received by the network adapter, the memory being further adapted to store instructions executable by the processor, wherein at least some of the instructions are configured to instruct the processor to perform the steps of: storing data received by the network adapter into one or more data structures, the one or more data structures formed into a variable-length chain of data structures; generating a set of pointers that collectively define a fixed sized block of the data stored in the variable-length chain of data structures; and associating a pointer reference count with the variable-length chain of data structures, the pointer reference count keeping track of the number of buffer pointers addressing portions of the data stored in the variable-length chain of data structures.

35. The network device of claim 34 , wherein the memory is further adapted to store instructions for performing the step of: deallocating at least some of the data structures in the variable-length chain of data structures when the pointer reference count associated with the variable-length chain of data structures equals zero.

36. The network device of claim 34 , wherein the memory is further adapted to store instructions for performing the step of: processing said fixed sized data block using said set of pointers, without first copying said fixed sized data block into a fixed-sized data buffer.

37. A computer system configured to partition data into fixed sized data blocks, the computer system comprising: means for storing data in one or more data structures, the one or more data structures formed into a variable-length chain of data structures; means for generating a set of pointers that collectively define a fixed sized block of the data stored in the variable-length chain of data structures; means for processing said fixed sized data block using said set of pointers, without first copying said fixed sized data block into a fixed-sized data buffer.

38. A computer system configured to partition data into fixed sized data blocks, the computer system comprising: means for storing data in one or more data structures, each data structure having a first fixed size; means for generating a set of pointers that collectively define a fixed sized block of the data stored in the one or more data structures, the fixed sized data block having a second fixed size that is different than the first fixed size; and means for processing said fixed sized data block using said set of pointers, without first copying said fixed sized data block into a fixed-sized data buffer.

39. The computer system of claim 38 , further comprising: means for storing said set of pointers in a header associated with the fixed sized data block.