Speech-Based Storage Tracking

1. A method for tracking inventory in a refrigerator, the method comprising: receiving, at an electronic device, first audio data representing a first utterance asking where a first physical item is located within the refrigerator in response to a door of the refrigerator being opened; receiving, from the refrigerator, a device identifier associated with the refrigerator; generating first text data representing the first audio data by performing automated speech recognition processing on the first audio data; determining, using natural language understanding processing, that the first text data corresponds to a request for identification of a current location of the first physical item; determining a user account that is associated with the device identifier; determining a refrigerator profile on an inventory management system that is associated with user account, the refrigerator profile including location information, physical item information, and weight information associated with physical items stored within the refrigerator; determining supplemental data corresponding to a change in a weight measurement of the first physical item; analyzing the supplemental data to determine the change is associated with a range of weights of the first physical item; associating the supplemental data with the physical items stored within the refrigerator; sending, to the inventory management system, a first request for data identifying the current location of the first physical item within the refrigerator, wherein sending the request comprises sending the device identifier to the inventory management system such that the refrigerator profile is accessed; determining, based on association between the user account, the refrigerator profile, and the supplemental data, the current location of the first physical item is at a first location; receiving, from the inventory management system, second text data indicating the first location of the first physical item within the refrigerator; generating second audio data by performing text-to-speech processing on the second text data; and sending the second audio data to the refrigerator to cause a response message to be output by a speaker of the refrigerator.

2. The method of claim 1 , further comprising: receiving, from the refrigerator, first weight sensor data representing a first weight determined by a first weight sensor of the refrigerator; receiving third audio data representing a second utterance that indicates that the first physical item was placed within the refrigerator at a second location; generating third text data representing the third audio data by performing the automated speech recognition processing on the third audio data; determining, using the natural language understanding processing, that the third text data indicates that the first physical item was placed at the second location; and sending, to the inventory management system, the third text data and the first weight sensor data such that the inventory management system is able to update the refrigerator profile to have the first physical item be associated with the second location be associated, and update a current weight of the first physical item to be the first weight.

3. The method of claim 1 , further comprising: receiving, from the refrigerator, third audio data representing a second utterance requesting a current location of a second physical item while the door is still open; generating third text data representing the third audio data by performing the automated speech recognition processing on the third audio data; determining, using the natural language understanding processing, that the third text data corresponds to a request for the current location of the second physical item, the second physical item having a second physical item name; sending, to the inventory management system, a second request for the current location of the second physical item associated with the second physical item name; receiving, from the inventory management system, fourth text data representing an additional response message indicating that, for the refrigerator profile, the second physical item is currently located at a second location within the refrigerator; generating fourth audio data representing the fourth text data by performing the text-to-speech processing on the third text data; and sending the fourth audio data to the refrigerator such that the additional response message is played by the speaker to indicate that the second physical item is located at the second location.

4. The method of claim 1 , further comprising: receiving, prior to receiving the first audio data, a notification that the door has been opened based on the refrigerator determining that a door sensor of the refrigerator output a signal indicating that the door was opened; and causing, in response to the notification, the first audio data to be received from the refrigerator such that audio signals of the first utterance are received by a microphone of the refrigerator, the refrigerator generates the first audio data representing the first utterance using the audio signals received by the microphone, and the refrigerator sends the first audio data to the electronic device for automated speech recognition processing.

5. A method, comprising: receiving sensor data representing a weight of a physical item from a sensor of a first device, wherein the sensor is at a first location; determining that the physical item has a range of associated weights; determining the sensor data corresponds to a change in weight of the physical item; associating, based at least in part on the change in weight, the sensor data with the physical item; receiving a device identifier corresponding to the first device; determining a user profile associated with the device identifier; determining, based at least in part on the sensor data and the user profile, the physical item; storing an association between the sensor and the physical item; receiving, from the first device, first audio data representing a first utterance; performing speech processing on the first audio data to determine that the first utterance requests information identifying a current location of the physical item; identifying stored data corresponding to the association; determining, using the stored data, that the current location of the physical item is at the first location; receiving data representing the first location; generating second audio data representing the data; and sending the second audio data to the first device to cause the first device to output audio corresponding to the second audio data.

6. The method of claim 5 , further comprising: determining that the sensor data corresponds to the weight corresponding to the first location; and determining that the physical item has the weight.

7. The method of claim 6 , further comprising: receiving, from the first device, third audio data representing a second utterance; determining that the second utterance indicates additional information associated with the physical item; and sending, to an inventory management system, a request to update the user profile to indicate that the additional information is associated with the first location, such that the user profile includes the additional information and the weight as being associated with the physical item.

8. The method of claim 5 , further comprising: receiving, from the first device, third audio data representing a second utterance; determining that the second utterance indicates a second location where the physical item is stored; and causing the user profile to store an additional association between the physical item and the second location.

9. The method of claim 5 , further comprising: receiving second data representing a second request for additional information associated with the physical item; generating third audio data representing the second data; sending the third audio data to the first device; and sending an instruction to the first device that causes the first device to send fourth audio data representing second audio captured after the second request is output.

10. The method of claim 9 , further comprising: receiving, from the first device, the fourth audio data representing a second utterance; determining, using the fourth audio data, that the second utterance indicates the additional information; and sending, to an inventory management system, a request to update causing the user profile to store an additional association between the additional information and the physical item.

11. A system, comprising: communication circuitry; at least one processor; at least one memory including instructions operable to be executed by the at least one processor to configure the system to: receive sensor data representing a weight of a physical item from a sensor of a first device, wherein the sensor is at a first location; determine the physical item has a range of associated weights; determine the sensor data corresponds to a change in weight of the physical item; associate, based at least in part on the change in weight, the sensor data with the physical item; receive a device identifier from the first device; determine a user profile associated with the device identifier; determine, based at least in part on the sensor data and the user profile, a physical item; store an association between the sensor and the physical item; determine that first audio data representing a first utterance was received from the first device; perform speech processing on the first audio data to determine that the first utterance requests information identifying a current location of the physical item; identify stored data corresponding to the association; determine, using the stored data, that current location of the physical item is at the first location; receive data representing the first location; generate second audio data representing the data; and send the second audio data to the first device to cause the first device to output audio corresponding to the second audio data.

12. The system of claim 11 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further configure the system to: determine that sensor data corresponds to the weight corresponding to the first location; and determine that the physical item has the weight.

13. The system of claim 12 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further configure the system to: determine that third audio data representing a second utterance was received from the first device; determine that the second utterance indicates additional information associated with the physical item; and send, to an inventory management system, a request to update the user profile to indicate that the additional information is associated with the first location, such that the user profile includes the additional information and the weight as being associated with the physical item.

14. The system of claim 11 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further configure the system to: determine that third audio data representing a second utterance was received from the first device; determine that the second utterance indicates a second location where the physical item is stored; and causing the user profile to store an additional association between the physical item and the second location.

15. The system of claim 11 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further configure the system to: receive second data representing a second request for additional information associated with the physical item; generate third audio data representing the second data; send the third audio data to the first device; and send an instruction to the first device that causes the first device to send fourth audio data representing audio captured after the second request is output.

16. The system of claim 15 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further configure the system to: determine that the fourth audio data representing a second utterance was received from the first device; determine, using the fourth audio data, that the second utterance indicates the additional information; and send, to an inventory management system, a request to update causing the user profile to store an additional association between the additional information and the physical item.