A method includes receiving, from a sensor on a robotic device, a captured image representative of an environment of the robotic device when the robotic device is at a location in the environment. The method also includes determining, based at least on the location of the robotic device, a rendered image representative of the environment of the robotic device. The method further includes determining, by applying at least one pre-trained machine learning model to at least the captured image and the rendered image, a property of one or more portions of the captured image.
Legal claims defining the scope of protection, as filed with the USPTO.
2. The method of claim 1, wherein the rendered image is a top-down image of an area at the location of the robotic device.
3. The method of claim 1, wherein the rendered image is a perspective image of an area at the location of the robotic device.
4. The method of claim 1, wherein the rendered image has a wider field of view than the captured image.
7. The method of claim 6, wherein the map includes one or more labels corresponding to the area at the location of the robotic device, and wherein the at least one pre-trained machine learning model is further applied to the one or more labels.
9. The method of claim 1, wherein the captured image and the rendered image are concatenated prior to being input into the at least one pre-trained machine learning model.
11. The method of claim 1, wherein the at least one pre-trained machine learning model comprises an attention mechanism trained to identify a region of the rendered image such that the captured image attends to the region of the rendered image for determining the property of the one or more portions of the environment represented by the captured image.
12. The method of claim 1, wherein one of the at least one pre-trained machine learning model is a model trained on a different environment of the robotic device and further refined based on the environment of the robotic device.
15. The robotic device of claim 14, wherein the sensor is a red green blue (RGB) camera, a red green blue depth (RGB-D) camera, a depth sensor, or a LIDAR sensor.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
March 4, 2021
September 17, 2024
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.