Floating Gate Architecture for Deep Neural Network Application

1. A resistive processing unit (RPU) circuit, comprising: at least one floating gate storage device, the floating gate storage device including a floating gate, a control gate and an inject/erase gate; and a feedback circuit connected to the control gate and the floating gate of the floating gate storage device, the feedback circuit being configured to control a voltage potential on the control gate as a function of a voltage potential on the floating gate to thereby maintain a substantially constant floating gate potential of the floating gate storage device during an update mode of operation of the RPU circuit, and the feedback circuit being disabled during a readout mode of operation of the RPU circuit.

2. The RPU circuit of claim 1 , wherein the feedback circuit comprises: a current mirror including a first node for receiving a first current supplied thereto and a second node for generating a second current which is a scaled version of the first current; and a sense device coupled with the current mirror, the sense device being controlled as a function of the voltage potential on the floating gate of the floating gate storage device.

3. The RPU circuit of claim 2 , wherein the current mirror comprises first and second metal-oxide-semiconductor (MOS) transistors and the sense device comprises a third MOS transistor, a first source/drain of each of the first and second MOS transistors being connected with a first voltage supply, a second source/drain and gate of the first MOS transistor being connected together and adapted to receive the first current, a gate of the second MOS transistor being connected with the gate of the first MOS transistor, a second source/drain of the second MOS transistor being connected with a first source/drain of the third MOS transistor and the control gate of the floating gate storage device, a gate of the third MOS transistor being connected with the floating gate of the floating gate storage device, and a second source/drain of the third MOS transistor being connected with a second voltage supply.

4. The RPU circuit of claim 3 , wherein the first voltage supply comprises a programmable voltage source configured to selectively enable the feedback circuit as a function of a first control signal supplied to the feedback circuit.

5. The RPU circuit of claim 3 , wherein the feedback circuit further comprises a switching device coupled with the control gate of the floating gate storage device, the switching device being configured to keep a potential on the control gate at substantially zero during a readout mode of operation of the RPU circuit.

6. The RPU circuit of claim 5 , wherein the switching device comprises a fourth MOS transistor having a first source/drain connected with the control gate of the floating gate storage device, a second source/drain connected with the second voltage supply, and a gate adapted to receive a first control signal for selectively activating the switching device.

7. The RPU circuit of claim 1 , wherein the feedback circuit comprises a first transistor connected in a closed-loop feedback arrangement between the floating gate and the control gate of the floating gate storage device, and wherein the first transistor is configured to vary a voltage potential on the control gate in a manner which keeps a voltage potential on the floating gate substantially constant, independent of stored charge on the floating gate.

8. The RPU circuit of claim 1 , wherein the inject/erase gate of the floating gate storage device is connected with a first voltage source, and wherein during a positive update operation of the RPU circuit, the first voltage source is configured to generate positive pulses to inject charge onto the floating gate of the floating gate storage device, and during a negative update operation, the first voltage source is configured to generate negative pulses to remove charge from the floating gate.

9. The RPU circuit of claim 1 , wherein the floating gate storage device comprises a source and drain, and wherein during an update mode of operation of the RPU circuit, the source and drain of the floating gate storage device are electrically connected together, and during a readout mode of operation of the RPU circuit, the first voltage source is set to a prescribed voltage level and a current flowing through the floating gate storage device is measured, the measured current is proportional to a conductance of the floating gate storage device and indicative of stored charge in the floating gate storage device.

10. The RPU circuit of claim 9 , wherein the measured current flowing through the floating gate storage device corresponds to a conductivity of the floating gate storage device which varies as a function of a voltage potential on the floating gate.

11. The RPU circuit of claim 1 , wherein the RPU circuit is configured such during a forward pass mode of operation, the feedback circuit is disabled and the control gate of the floating gate storage device is held at a constant voltage, and wherein a voltage, V, is applied to a source of the floating gate storage device and a current, I=V·W, where W is a weight representative of channel conductance of the floating gate storage device, is collected from a drain of the floating gate storage device.

12. The RPU circuit of claim 1 , wherein the RPU circuit is configured such that during a backward pass mode of operation, the feedback circuit is disabled and the control gate of the floating gate storage device is held at a constant voltage, and wherein a voltage, V, is applied to a drain of the floating gate storage device and a current, I=V·W, where W is a weight representative of channel conductance of the floating gate storage device, is collected from a source of the floating gate storage device.

13. The RPU circuit of claim 1 , wherein the RPU circuit is configured such that an update mode of operation is performed using two stochastic pulses, one of the pulses being applied to the inject/erase gate of the floating gate storage device, and the other of the pulses being applied to first and second source/drains of the floating gate storage device, whereby charge is injected or removed from the floating gate of the floating gate storage device when the two stochastic pulses coincide with one another, and no change in weight occurs when the two stochastic pulse do not coincide with one another.

14. A method for improving matrix multiplication speed in a neural network, the method comprising: providing at least one resistive processing unit (RPU) circuit, the RPU circuit comprising: at least one floating gate storage device, the floating gate storage device including a floating gate, a control gate and an inject/erase gate; and a feedback circuit connected to the control gate and the floating gate of the floating gate storage; configuring the feedback circuit to control a voltage potential on the control gate as a function of a voltage potential on the floating gate to thereby maintain a substantially constant floating gate potential of the floating gate storage device during an update mode of operation of the RPU circuit; and disabling the feedback circuit during a readout mode of operation of the RPU circuit.

15. The method of claim 14 , further comprising maintaining a potential on the control gate of the floating gate storage device in the RPU circuit at substantially zero during a readout mode of operation of the RPU circuit.

16. The method of claim 14 , further comprising applying positive pulses to the inject/erase gate of the floating gate storage device during a positive update operation of the RPU circuit to inject charge onto the floating gate of the floating gate storage device, and applying negative pulses to the inject/erase gate during a negative update operation of the RPU circuit to remove charge from the floating gate of the floating gate storage device.

17. The method of claim 14 , further comprising connecting a source and a drain of the floating gate storage device together during an update mode of operation of the RPU circuit, and during a readout mode of operation of the RPU circuit, connecting the source and drain of the floating gate storage device to a prescribed voltage level and measuring a current flowing through the floating gate storage device, the measured current being indicative of a value stored in the floating gate storage device.

18. The method of claim 14 , further comprising, during a forward pass mode of operation of the RPU circuit: disabling the feedback circuit in the RPU circuit; maintaining the control gate of the floating gate storage device at a constant voltage; applying a voltage, V, to a source of the floating gate storage device; and collecting a current, I, from a drain of the floating gate storage device, wherein the collected current I is equal to V·W, where W is a weight representative of channel conductance of the floating gate storage device.

19. The method of claim 14 , further comprising, during a backward pass mode of operation of the RPU circuit: disabling the feedback circuit in the RPU circuit; maintaining the control gate of the floating gate storage device at a constant voltage; applying a voltage, V, to a drain of the floating gate storage device; and collecting a current, I, from a source of the floating gate storage device, wherein the collected current I is equal to V·W, where W is a weight representative of channel conductance of the floating gate storage device.

20. The method of claim 14 , further comprising performing an update mode of operation of the RPU circuit using two stochastic pulses, one of the pulses being applied to the inject/erase gate of the floating gate storage device, and the other of the pulses being applied to a source and a drain of the floating gate storage device, whereby charge is injected or removed from the floating gate of the floating gate storage device when the two stochastic pulses coincide with one another, and no change in weight occurs when the two stochastic pulse do not coincide with one another.