Parity data in dynamic random access memory (DRAM)

This application is a Continuation of U.S. application Ser. No. 18/108,876, filed Feb. 13, 2023, which issues as U.S. Pat. No. 12,061,518 on Aug. 13, 2024, which is a Continuation of U.S. application Ser. No. 17/130,885, filed Dec. 22, 2020, which issued as U.S. Pat. No. 11,579,964 on Feb. 14, 2023, the contents of which are included herein by reference.

The present disclosure relates generally to managing parity data in dynamic random access memory (DRAM).

Memory devices are typically provided as internal, semiconductor, integrated circuits in computers or other electronic devices. There are many different types of memory including volatile and non-volatile memory. Volatile memory can require power to maintain its data and includes random-access memory (RAM), DRAM, and synchronous dynamic random access memory (SDRAM), among others. Non-volatile memory can provide persistent data by retaining stored data when not powered and can include NAND flash memory, NOR flash memory, read only memory (ROM), Electrically Erasable Programmable ROM (EEPROM), Erasable Programmable ROM (EPROM), and resistance variable memory such as phase change random access memory (PCRAM), resistive random access memory (RRAM), and magnetoresistive random access memory (MRAM), among others.

Memory is also utilized as volatile and non-volatile data storage for a wide range of electronic applications. Non-volatile memory may be used in, for example, personal computers, portable memory sticks, digital cameras, cellular telephones, portable music players such as MP3 players, movie players, and other electronic devices. Memory cells can be arranged into arrays, with the arrays being used in memory devices.

Memory can be part of a memory module (e.g., a dual in-line memory module (DIMM)) used in computing devices. Memory modules can include volatile, such as DRAM, for example, and/or non-volatile memory, such as Flash memory or RRAM, for example. The DIMMs can be used as main memory in computing systems.

The present disclosure includes methods, apparatuses, and systems related to generating, at a controller, parity data based on user data queued for writing to a non-volatile memory device coupled to the controller, receiving the parity data at a DRAM device from the controller and writing the parity data to the DRAM device, receiving the user data at the non-volatile memory device from the controller and writing the user data to the non-volatile memory device, reading the user data from the non-volatile memory device via the controller, and receiving the parity data at the controller from the DRAM device.

The non-volatile memory device and/or the DRAM can be included in a DIMM. The DIMM can be a non-volatile dual in-line memory module (NVDIMM). In a number of embodiments, the non-volatile memory device can be a 3D Cross-point device or a NAND device.

The non-volatile memory device can provide near DRAM speeds with non-volatility, which can eliminate substantial system overhead such as periodic checkpointing and/or dropping system state to a hard drive and/or SSD. The non-volatile memory device also can provide a large memory capacity, for example 1 terabyte (TB), of main memory. In some examples, the memory capacity of the non-volatile memory device can be used for in-memory databases.

In a number of embodiments, non-volatile memory, for instance, 3D Cross-point can have a high defectivity and in some circumstances cannot tolerate a die fail. Die failure can be absorbed using XOR and/or RAID methods. These methods can include one or more dice solely storing parity data of user data written on other dice and responsive to a die fail an XOR operation can be performed with the parity data to restore the user data. However, writing the parity data to one or more dice and/or reconstructing the user data responsive to a die fail can reduce write performance and/or read performance.

A non-volatile memory device coupled to a DRAM device can absorb a die failure without reducing the write performance and/or the read performance of a system. An independent DRAM module to store parity data allows for the full memory bandwidth of the non-volatile memory device to go to serving high performance writes, for example. In a number of embodiments, the non-volatile memory device and the DRAM device can be included in a relatively small footprint.

The controller including an error correction code (ECC) module can generate the parity data by performing an error correction code operation, for example an XOR and/or RAID operation, on the user data. The parity data can be stored in (e.g., written to) the DRAM device and/or in the non-volatile memory device with the user data. In some examples, the parity data can be embedded in the user data in the non-volatile memory device. In a number of embodiments, a bit stream that comprises the user data can also comprise one or more bits of parity data.

User data stored in the non-volatile memory device can be reconstructed using the parity data. The controller can receive (e.g., read) the parity data from the DRAM device and reconstruct the user data in response to a read failure. The read failure can be due to corrupted memory in the non-volatile memory device. In some examples the controller can read and reconstruct the user data in one clock cycle since the parity data is readily available at the DRAM device. For example, when user data is read from the non-volatile memory device, the corresponding parity data is also read from the DRAM device so that if there is a read error, the parity data is readily available and an ECC operation can be executed in one clock cycle.

In a number of embodiments, the parity data can be stored in the non-volatile memory device prior to powering off the DRAM device and/or the DIMM. The parity data can be rewritten to the DRAM device in response to powering on the DRAM device. In some examples, the parity data can be regenerated at the controller and/or received at the DRAM device in response to powering off and powering on the DRAM device. For example, the controller can receive the user data from the non-volatile memory device and perform an XOR operation on the user data in response to powering on the DRAM device and/or the DIMM.

In the following detailed description of the present disclosure, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration how a number of embodiments of the disclosure may be practiced. These embodiments are described in sufficient detail to enable those of ordinary skill in the art to practice the embodiments of this disclosure, and it is to be understood that other embodiments may be utilized and that process, electrical, and/or structural changes may be made without departing from the scope of the present disclosure. As used herein, the designator “M”, “N”, “X”, and “Y” indicates that a number of the particular feature so designated can be included with a number of embodiments of the present disclosure.

As used herein, “a number of” something can refer to one or more of such things. For example, a number of DIMMs can refer to one or more DIMMs. Additionally, designators such as “M”, “N”, “X”, and “Y”, as used herein, particularly with respect to reference numerals in the drawings, indicates that a number of the particular feature so designated can be included with a number of embodiments of the present disclosure.

The figures herein follow a numbering convention in which the first digit or digits correspond to the drawing figure number and the remaining digits identify an element or component in the drawing. Similar elements or components between different figures may be identified by the use of similar digits. As will be appreciated, elements shown in the various embodiments herein can be added, exchanged, and/or eliminated so as to provide a number of additional embodiments of the present disclosure. In addition, the proportion and the relative scale of the elements provided in the figures are intended to illustrate various embodiments of the present disclosure and are not to be used in a limiting sense.

is a functional block diagram of a computing systemincluding an apparatus in the form of a number of memory systems-. . .-N, in accordance with one or more embodiments of the present disclosure. As used herein, an “apparatus” can refer to, but is not limited to, any of a variety of structures or combinations of structures, such as a circuit or circuitry, a die or dice, a module or modules, a device or devices, or a system or systems, for example.

In the embodiment illustrated in, memory systems-. . .-N can include a one or more dual in-line memory modules (DIMM)-, . . . ,-X,-Y. The DIMMs-, . . . ,-X,-Y can include volatile memory and/or non-volatile memory. In a number of embodiments, memory systems-, . . . ,-N can include a multi-chip device. A multi-chip device can include a number of different memory types and/or memory modules. For example, a memory system can include non-volatile or volatile memory on any type of a module.

In, memory system-is coupled to the host via channel-can include DIMMs-, . . . ,-X, where DIMM-is a NVDIMM and-X is DRAM DIMM. In this example, each DIMM-, . . . ,-X,-Y includes a controller. Controllercan receive commands from hostand control execution of the commands on a DIMM. Also, in a number of embodiments, the protocol of the present disclosure could be implemented by a memory device (e.g., a DIMM) without a controller and execution of the commands using the protocol of the present disclosure could be built into the memory device.

The hostcan send commands to the DIMMs-, . . . ,-X,-Y using the protocol of the present disclosure and/or a prior protocol, depending on the type of memory in the DIMM. For example, the host can use the protocol of the present disclosure to communicate on the same channel (e.g., channel-) with a NVDIMM and a prior protocol to communicate with a DRAM DIMM that are both on the same memory system. The host and the NVDIMM can communicate via read ready (R_RDY) signals, read send (R_SEND) signals, write credit increment (WC_INC) signals, and read identification (RID) signals according the protocol of the present disclosure. The read ready (R_RDY) signals, read send (R_SEND) signals, write credit increment (WC_INC) signals, and read identification (RID) signals can be sent via pins that are unused in a prior protocol (e.g. DDR4) or are pins from a prior protocol (e.g. DDR4) that are repurposed (e.g. used differently) so that the present protocol is compatible with the prior protocol. Also, pins can be assigned to the read ready (R_RDY) signals, read send (R_SEND) signals, write credit increment (WC_INC) signals, and read identification (RID) signals in protocols that are being developed (e.g., DDR5).

As illustrated in, a hostcan be coupled to the memory systems-. . .-N. In a number of embodiments, each memory system-. . .-N can be coupled to hostvia a channel. In, memory system-is coupled to hostvia channel-and memory system-N is coupled to hostvia channel-N. Hostcan be a laptop computer, personal computers, digital camera, digital recording and playback device, mobile telephone, PDA, memory card reader, interface hub, among other host systems, and can include a memory access device, e.g., a processor. One of ordinary skill in the art will appreciate that “a processor” can intend one or more processors, such as a parallel processing system, a number of coprocessors, etc.

Hostincludes a host controllerto communicate with memory systems-. . .-N. The host controllercan send commands to the DIMMs-, . . . ,-X,-Y via channels-. . .-N. The host controllercan communicate with the DIMMs-, . . . ,-X,-Y and/or the controlleron each of the DIMMs-, . . . ,-X,-Y to read, write, and erase data, among other operations. A physical host interface can provide an interface for passing control, address, data, and other signals between the memory systems-. . .-N and hosthaving compatible receptors for the physical host interface. The signals can be communicated betweenand DIMMs-, . . . ,-X,-Y on a number of buses, such as a data bus and/or an address bus, for example, via channels-. . .-N.

The host controllerand/or controlleron a DIMM can include control circuitry, e.g., hardware, firmware, and/or software. In one or more embodiments, the host controllerand/or controllercan be an application specific integrated circuit (ASIC) coupled to a printed circuit board including a physical interface. Also, each DIMM-, . . . ,-X,-Y can include buffersof volatile and/or non-volatile memory and an ECC module. Buffercan be used to buffer data that is used during execution of read commands and/or write commands. An ECC operation can be performed on data by the ECC moduleto correct errors prior to being stored in the buffer, for example.

The DIMMs-, . . . ,-X,-Y can provide main memory for the memory system or could be used as additional memory or storage throughout the memory system. Each DIMM-, . . . ,-X,-Y can include one or more arrays of memory cells, e.g., non-volatile memory cells. The arrays can be flash arrays with a NAND architecture, for example. Embodiments are not limited to a particular type of memory device. For instance, the memory device can include RAM, ROM, DRAM, SDRAM, PCRAM, RRAM, and flash memory, among others.

The embodiment ofcan include additional circuitry that is not illustrated so as not to obscure embodiments of the present disclosure. For example, the memory systems-. . .-N can include address circuitry to latch address signals provided over I/O connections through I/O circuitry. Address signals can be received and decoded by a row decoder and a column decoder to access the DIMMs-, . . . ,-X,-Y. It will be appreciated by those skilled in the art that the number of address input connections can depend on the density and architecture of the DIMMs-, . . . ,-X,-Y.

is a block diagram of an apparatus in the form of a dual in-line memory modules (DIMM)in accordance with a number of embodiments of the present disclosure. DIMMcan correspond to DIMMs-, . . . ,-X,-Y in. In, DIMMcan include a controller. Controllercan correspond to controllerin. Controllercan include memory, such as SRAM memory, that can be a bufferand/or an ECC module. Bufferand ECC modulecan correspond to bufferand ECC modulein, respectively. DIMMcan include a non-volatile memory deviceand a DRAM devicecoupled to the controller.

Non-volatile memory devicecan include non-volatile memory arrays. In some examples, the non-volatile memory devicecan be a 3D Cross-point device or a NAND device. The non-volatile memory devicecan provide near DRAM speeds with non-volatility, which can eliminate substantial system overhead such as periodic checkpointing and/or dropping system state to a hard drive and/or SSD. The non-volatile memory devicealso can provide a large memory capacity, for example 1 TB, of main memory. In some examples, the memory capacity of the non-volatile memory devicecan be used for in-memory databases.

In a number of embodiments, non-volatile memory, for instance, 3D Cross-point can have a high defectivity and in some circumstances cannot tolerate a die fail. Die failure can be absorbed using XOR and/or RAID methods. These methods can include one or more dice solely holding parity data of user data written on other dice and responsive to a die fail an XOR operation can be performed with the parity data to restore the user data. However, writing the parity data to one or more dice and/or reconstructing the user data responsive to a die fail can reduce write performance and/or read performance.

DIMMincluding the non-volatile memory deviceand the DRAM devicecan absorb a die failure without reducing the write performance and/or the read performance of the DIMM. The DIMMincluding an independent DRAM deviceto store the parity dataallows for the full memory bandwidth of the non-volatile memory deviceto go to serving high performance writes, for example. In a number of embodiments, the non-volatile memory deviceand the DRAM devicecan be included in a relatively small footprint.

The non-volatile memory devicecan include control circuitry-(e.g., hardware, firmware, and/or software) which can be used to execute commands on non-volatile memory device. The control circuitry-can receive commands from controller. The control circuitry-can be configured to execute commands to read and/or write data in non-volatile memory device.

In a number of embodiments, user datacan be stored on non-volatile memory device. The non-volatile memory devicecan be configured to read the user datain response to receiving a read command from controllerand/or a host (e.g., hostin).

DRAM devicecan include control circuitry-to execute commands on DRAM device. The control circuitry-can receive commands from controllerand/or the host and can execute commands to read and/or write data in DRAM device.

The controllerincluding the ECC modulecan generate the parity databy performing an error correction code operation, for example an XOR and/or RAID operation, on the user data. The parity datacan be stored in the DRAM deviceand/or in the non-volatile memory devicewith the user data. In some examples, the parity datacan be embedded in the user datain the non-volatile memory device. In a number of embodiments, a bit stream that comprises the user datacan also comprise one or more bits of the parity data.

User datastored in the non-volatile memory devicecan be reconstructed using the parity data. The non-volatile memory devicecan transmit the user datato the controllerin response to receiving a read command from the host device. The controllercan receive the parity datafrom the DRAM deviceand reconstruct the user datain response to a read failure (e.g., an error during a read operation of the user datafrom the non-volatile memory device). The read failure can be due to corrupted memory in the non-volatile memory device. The controllercan reconstruct the user databy performing an ECC operation on the parity datausing the ECC module. The controllercan send the user datato the host device and/or the non-volatile memory devicein response to reconstructing the user data.

In some examples the controllercan read and/or reconstruct the user datain one clock cycle since the parity datais readily available in the DRAM device. For example, when user datais read from the non-volatile memory device, the corresponding parity datais also read from the DRAM deviceso that if there is a read error, the parity datais readily available and an ECC operation can be executed in one clock cycle. Reading and/or reconstructing the user datain one clock cycle can increase performance, decrease power consumption and/or decrease processing time of the DIMM.

In a number of embodiments, the parity datacan be stored in the non-volatile memory deviceprior to powering off the DRAM deviceand/or the DIMM. The parity datacan be rewritten to the DRAM devicein response to powering on the DRAM device. In some examples, the parity datacan be regenerated at the controllerand/or received at the DRAM devicein response to powering off and powering on the DRAM device. For example, the controllercan receive the user datafrom the non-volatile memory device and perform an ECC operation, for example an XOR operation, on the user datausing the ECC moduleto regenerate the parity datain response to powering on the DRAM deviceand/or the DIMM.

is a flow diagram of a methodfor storing parity data in DRAM in accordance with a number of embodiments of the present disclosure. At block, the methodcan include generating, at a controller, parity data based on user data queued (e.g., scheduled) for writing to a non-volatile memory device coupled to the controller. The controller including an ECC module can generate the parity data by performing an error correction code operation, for example an XOR and/or RAID operation, on the user data.

At block, the methodcan include receiving the parity data at a DRAM device from the controller and writing the parity data to the DRAM device. The parity data can be stored in the DRAM device and/or in the non-volatile memory device with the user data. In some examples, the parity data can be embedded in the user data in the non-volatile memory device.

At block, the methodcan include receiving the user data at the non-volatile memory device from the controller and writing the user data to the non-volatile memory device. The non-volatile memory device can be a 3D Cross-point device or a NAND device. The non-volatile memory device can be included in a DIMM. In some examples, the DIMM can be a NVDIMM.

At block, the methodcan include reading the user data from the non-volatile memory device via the controller. The non-volatile memory device can transmit the user data to the controller in response to receiving a read command, for example. While executing the read command, a read failure may occur. For example, a read failure may occur due to corrupted memory in the non-volatile memory device.

At block, the methodcan include receiving the parity data at the controller from the DRAM device. The controller can receive the parity data from the DRAM device and reconstruct the user data in response to the read failure. The controller can reconstruct the user data by performing an ECC operation on the parity data using the ECC module. In some examples the controller can read and reconstruct the user data in one clock cycle. For example, when user data is read from the non-volatile memory device, the corresponding parity data is also read from the DRAM device so that if there is a read error, the parity data is readily available and an ECC operation can be executed in one clock cycle. The controller can send the user data to the non-volatile memory device and/or a host device in response to reconstructing the user data.

In a number of embodiments, methodcan further include storing the parity data in the non-volatile memory device prior to powering off the DRAM device and/or the DIMM. The parity data can be rewritten to the DRAM device in response to powering on the DRAM device. In some examples, the parity data can be regenerated at the controller and/or received at the DRAM device in response to powering off and powering on the DRAM device. For example, the controller can receive the user data from the non-volatile memory device and perform an ECC operation, for example an XOR operation, on the user data using the ECC module to regenerate the parity data in response to powering on the DRAM device and/or the DIMM.

Although specific embodiments have been illustrated and described herein, those of ordinary skill in the art will appreciate that an arrangement calculated to achieve the same results can be substituted for the specific embodiments shown. This disclosure is intended to cover adaptations or variations of various embodiments of the present disclosure. It is to be understood that the above description has been made in an illustrative fashion, and not a restrictive one. Combination of the above embodiments, and other embodiments not specifically described herein will be apparent to those of skill in the art upon reviewing the above description. The scope of the various embodiments of the present disclosure includes other applications in which the above structures and methods are used. Therefore, the scope of various embodiments of the present disclosure should be determined with reference to the appended claims, along with the full range of equivalents to which such claims are entitled.

In the foregoing Detailed Description, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the disclosed embodiments of the present disclosure have to use more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment.