Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method for compensating for autonomous vehicle (AV) system errors, the method comprising: obtaining multiple 3D points for an object; transforming the multiple 3D points to a camera frame; obtaining actual 2D points for the object; establishing a virtual camera corresponding to the actual 2D points; determining an offset compensation matrix between a camera associated with the camera frame and the virtual camera; applying the offset compensation matrix to data points prior to use by vehicle control systems; and controlling operation of the AV with offset compensated data points.
This invention relates to error compensation in autonomous vehicle (AV) systems, specifically addressing discrepancies between 3D object data and 2D camera projections that can lead to navigation and control errors. The method involves obtaining multiple 3D points representing an object in the environment, transforming these points into a camera frame of reference, and acquiring the corresponding actual 2D points captured by the vehicle's camera. A virtual camera is then established based on the actual 2D points, and an offset compensation matrix is calculated to align the camera frame with the virtual camera. This matrix is applied to subsequent data points before they are used by the vehicle's control systems, ensuring accurate spatial alignment and reducing errors in object detection and localization. The AV's operation is then controlled using the compensated data points, improving navigation and safety. The approach mitigates misalignments between sensor data and camera projections, enhancing the reliability of AV decision-making processes.
2. The method of claim 1 , wherein the determining the offset compensation matrix further comprising: executing a camera pose algorithm using the multiple 3D points in the camera frame and the actual multiple 2D points in a virtual camera frame associated with the virtual camera to determine the offset compensation matrix.
This invention relates to computer vision and augmented reality systems, specifically addressing the challenge of accurately aligning virtual content with real-world camera imagery. The method involves compensating for misalignment between a physical camera and a virtual camera by determining an offset compensation matrix. This matrix corrects discrepancies in the spatial relationship between the two cameras, ensuring that virtual objects appear correctly positioned in the real-world scene. The process begins by obtaining multiple 3D points in the physical camera's frame and their corresponding 2D projections in the virtual camera's frame. These points are used to calculate the offset compensation matrix. The method employs a camera pose algorithm, which processes the 3D and 2D point pairs to derive the matrix. This algorithm estimates the transformation required to align the virtual camera's perspective with the physical camera's view, accounting for any positional or rotational offsets. The offset compensation matrix is then applied to adjust the virtual camera's pose, ensuring that virtual content is rendered in the correct position relative to the real-world environment. This correction is essential for applications such as augmented reality, where precise alignment between virtual and real elements is critical for user experience. The method improves accuracy in mixed-reality environments by dynamically compensating for misalignments between the physical and virtual camera systems.
3. The method of claim 1 , wherein the offset compensation matrix includes translational matrices and rotation matrices.
This invention relates to a method for compensating for offsets in a system, particularly in applications involving spatial transformations such as robotics, computer vision, or sensor calibration. The problem addressed is the need to accurately account for positional and rotational discrepancies between components or coordinate systems, which can lead to errors in measurements, control, or navigation. The method involves generating an offset compensation matrix that corrects for misalignments between two or more coordinate systems. This matrix includes both translational matrices, which adjust for linear displacements, and rotation matrices, which correct for angular deviations. The translational matrices account for shifts in position along one or more axes, while the rotation matrices adjust for angular misalignments, ensuring proper alignment between the coordinate systems. The compensation matrix is applied to transform data from one coordinate system to another, eliminating errors caused by physical misalignments or calibration inaccuracies. This ensures precise spatial relationships, improving the accuracy of operations such as robotic arm positioning, sensor data fusion, or augmented reality overlays. The method is particularly useful in systems where high precision is required, such as industrial automation, autonomous vehicles, or medical imaging.
4. The method of claim 1 , wherein the multiple 3D points are in a world coordinate frame.
A method for processing three-dimensional (3D) point data involves transforming multiple 3D points from a world coordinate frame into a target coordinate frame. The transformation is performed using a transformation matrix derived from a set of reference points in the world coordinate frame and corresponding target points in the target coordinate frame. The transformation matrix is computed to minimize the sum of squared errors between the transformed reference points and the target points. This method ensures accurate alignment of 3D point data across different coordinate systems, which is useful in applications such as robotics, computer vision, and augmented reality. The transformation accounts for both rotational and translational differences between the coordinate frames, enabling precise spatial mapping and localization. The method is particularly effective when the reference and target points are known or can be reliably estimated, ensuring robust and efficient coordinate transformation.
5. The method of claim 4 , wherein the multiple 3D points in the world coordinate frame are transformed to an inertial measurement unit (IMU) frame, from the IMU frame to a Light Detection and Ranging (LiDAR) frame, and from the LiDAR frame to the camera frame.
This invention relates to a method for transforming multiple 3D points between different coordinate frames in a sensor fusion system. The method addresses the challenge of accurately aligning data from different sensors, such as inertial measurement units (IMUs), Light Detection and Ranging (LiDAR) systems, and cameras, which operate in distinct coordinate frames. Misalignment between these frames can lead to errors in applications like autonomous navigation, robotics, and augmented reality. The method involves transforming 3D points from a world coordinate frame to an IMU frame, then to a LiDAR frame, and finally to a camera frame. The IMU frame represents the orientation and motion of the system, while the LiDAR frame corresponds to the 3D point cloud data captured by the LiDAR sensor. The camera frame aligns with the 2D or 3D image data from the camera. Each transformation step accounts for the relative positioning and orientation between the respective sensors, ensuring accurate spatial consistency across the system. This multi-step transformation process enables precise sensor fusion, improving the reliability of applications that depend on integrated sensor data. The method may include additional steps, such as calibrating the sensors or compensating for motion artifacts, to further enhance accuracy.
6. The method of claim 1 , wherein the actual 2D points are determined using at least one of image detection, object detection, and corner detection.
This invention relates to a method for determining the positions of actual 2D points in an image or scene, addressing the challenge of accurately identifying and locating key points for applications such as computer vision, robotics, or augmented reality. The method leverages multiple detection techniques to enhance precision and reliability. Specifically, it employs image detection to analyze pixel patterns, object detection to identify and localize objects within the scene, and corner detection to pinpoint distinctive geometric features. By combining these approaches, the method improves the robustness of point localization, reducing errors caused by occlusions, lighting variations, or ambiguous features. The technique is particularly useful in dynamic environments where single detection methods may fail to provide consistent results. The integration of multiple detection modalities ensures that the system can adapt to different scenarios, whether detecting edges, textures, or specific objects. This multi-modal approach enhances the accuracy and reliability of 2D point determination, making it suitable for applications requiring precise spatial awareness.
7. The method of claim 1 , wherein physical properties of the virtual camera and the camera are the same.
A system and method for synchronizing a virtual camera with a physical camera to ensure consistent visual output. The technology addresses the challenge of discrepancies between virtual and physical camera systems, which can lead to mismatched visual effects, calibration errors, and inconsistent user experiences in applications such as augmented reality, virtual reality, and computer vision. The invention ensures that the physical properties of the virtual camera, such as focal length, field of view, lens distortion, and sensor characteristics, are identical to those of the physical camera. This synchronization allows for seamless integration between virtual and real-world imagery, improving accuracy in applications like 3D reconstruction, object tracking, and mixed reality environments. The method involves dynamically adjusting the virtual camera parameters to match the physical camera's real-time measurements, ensuring that any changes in the physical camera's settings are reflected in the virtual counterpart. This approach eliminates the need for manual calibration and reduces errors caused by environmental factors or hardware variations. The system is particularly useful in scenarios requiring high precision, such as medical imaging, autonomous navigation, and industrial automation, where accurate alignment between virtual and physical camera data is critical.
8. The method of claim 7 , wherein intrinsic parameters of the virtual camera and the camera are the same.
This invention relates to virtual camera systems used in computer graphics, particularly for aligning a virtual camera with a real-world camera to ensure consistent visual output. The problem addressed is the misalignment between virtual and real cameras, which can cause discrepancies in rendering and tracking. The solution involves synchronizing intrinsic parameters of both cameras, such as focal length, sensor size, and lens distortion, to ensure identical imaging characteristics. By matching these parameters, the virtual camera accurately replicates the real camera's perspective, enabling precise overlay of virtual objects onto real-world scenes. This alignment is critical for applications like augmented reality, 3D reconstruction, and visual effects, where accurate spatial correspondence between virtual and real elements is required. The method ensures that the virtual camera's settings mirror those of the real camera, eliminating visual inconsistencies and improving the fidelity of the combined output. The approach may be applied in real-time systems or pre-processing pipelines, depending on the application's requirements.
9. The method of claim 1 , wherein the offset compensation matrix includes yaw, pitch and roll matrices.
A method for improving the accuracy of sensor data in navigation systems, particularly for applications requiring precise orientation measurements such as autonomous vehicles, drones, or robotics. The method addresses the challenge of sensor inaccuracies caused by misalignments or biases in inertial measurement units (IMUs), which can lead to errors in determining yaw, pitch, and roll angles. These errors degrade the performance of navigation and control systems that rely on accurate orientation data. The method involves generating an offset compensation matrix to correct these errors. The compensation matrix includes separate yaw, pitch, and roll matrices, each designed to account for specific misalignments or biases in the sensor measurements. By applying these matrices, the method adjusts the raw sensor data to produce corrected orientation measurements. This ensures that the navigation system operates with higher accuracy, reducing drift and improving stability in dynamic environments. The yaw matrix compensates for rotational errors around the vertical axis, the pitch matrix corrects errors around the lateral axis, and the roll matrix addresses errors around the longitudinal axis. The combined compensation matrix is applied to the sensor data in real-time, allowing the system to maintain precise orientation estimates even under varying conditions. This approach enhances the reliability of navigation systems in applications where precise orientation is critical.
10. A vehicle system comprising: a processor configured to: obtain multiple 3D points for an object; transform the multiple 3D points to a camera frame; obtain actual 2D points for the object; establish a virtual camera corresponding to the actual 2D points; determine an offset compensation matrix between a camera associated with the camera frame and the virtual camera; and apply the offset compensation matrix to data points prior to use by vehicle control systems; and a controller configured to control operation of the AV with offset compensated data points.
This invention relates to autonomous vehicle (AV) systems that process 3D point data for object detection and control. The system addresses the challenge of aligning 3D sensor data (e.g., from LiDAR) with 2D camera data to improve accuracy in object localization and vehicle control. A processor obtains multiple 3D points representing an object, transforms these points into a camera coordinate frame, and retrieves corresponding 2D points from camera imagery. A virtual camera is then established based on the 2D points, and an offset compensation matrix is calculated to correct misalignment between the actual camera and the virtual camera. This matrix is applied to the 3D data points before they are used by vehicle control systems, ensuring precise spatial alignment. A controller then uses the compensated data to guide the AV's operation. The system enhances the reliability of sensor fusion by mitigating discrepancies between 3D and 2D data, improving decision-making for autonomous navigation.
11. The vehicle system of claim 10 , wherein the processor further configured to: execute a camera pose algorithm using the multiple 3D points in the camera frame and the actual multiple 2D points in a virtual camera frame associated with the virtual camera to determine the offset compensation matrix.
This invention relates to vehicle systems that use virtual cameras to generate synthetic views of a vehicle's surroundings. The problem addressed is the misalignment between real-world camera data and virtual camera projections, which can lead to inaccuracies in augmented reality displays or autonomous driving systems. The system includes a processor that processes data from one or more real cameras mounted on the vehicle to generate a 3D point cloud representing the vehicle's environment. A virtual camera is defined in a virtual space, and the system projects the 3D points onto a virtual 2D image plane associated with this virtual camera. The processor then compares the actual 2D points from the real camera with the projected 2D points from the virtual camera to identify discrepancies. To correct these discrepancies, the processor executes a camera pose algorithm that calculates an offset compensation matrix. This matrix adjusts the virtual camera's position and orientation to align the virtual projections with the real-world camera data, ensuring accurate overlays or simulations. The system may also include additional sensors, such as LiDAR or radar, to enhance the 3D point cloud accuracy. The invention improves the reliability of virtual camera-based applications in vehicles, such as augmented reality navigation or autonomous driving assistance.
12. The vehicle system of claim 10 , wherein the offset compensation matrix includes translational matrices and rotation matrices.
The invention relates to a vehicle system designed to improve navigation accuracy by compensating for sensor misalignments. The system addresses the problem of errors in vehicle positioning and orientation caused by imperfect alignment between sensors, such as inertial measurement units (IMUs) and cameras, which can lead to inaccurate navigation and mapping. The system includes an offset compensation matrix that corrects these misalignments by applying both translational and rotational adjustments. The translational matrices account for positional offsets between sensors, while the rotation matrices correct angular misalignments. This dual-matrix approach ensures precise alignment of sensor data, enhancing the accuracy of vehicle navigation and localization. The system may integrate with other vehicle components, such as control units or mapping modules, to provide real-time corrections. By dynamically compensating for sensor misalignments, the system improves the reliability of autonomous driving, robotic navigation, and other applications requiring precise spatial awareness. The invention focuses on mitigating errors that arise from physical misalignments, ensuring consistent and accurate sensor fusion for improved vehicle performance.
13. The vehicle system of claim 10 , wherein the multiple 3D points are in a world coordinate frame.
A vehicle system is designed to process and analyze three-dimensional (3D) point data, such as that obtained from sensors like LiDAR or cameras, to enhance navigation, obstacle detection, or autonomous driving capabilities. The system includes a processing unit that receives multiple 3D points representing objects or features in the environment. These points are mapped and processed in a world coordinate frame, which provides a standardized spatial reference system for accurate positioning and tracking of objects relative to a global or fixed reference. The world coordinate frame allows the system to maintain consistency across different sensor inputs and environmental conditions, improving the reliability of spatial data for applications such as path planning, collision avoidance, and environmental mapping. The system may also include additional components, such as sensors, memory, and communication interfaces, to support real-time data acquisition, storage, and transmission. By operating in a world coordinate frame, the system ensures that the 3D points are accurately aligned and scaled, enabling precise spatial analysis and decision-making for vehicle operations.
14. The vehicle system of claim 13 , wherein the multiple 3D points in the world coordinate frame are transformed to an inertial measurement unit (IMU) frame, from the IMU frame to a Light Detection and Ranging (LiDAR) frame, and from the LiDAR frame to the camera frame.
This invention relates to a vehicle system for processing and transforming 3D point data from multiple sensors, particularly for applications in autonomous driving or advanced driver-assistance systems. The system addresses the challenge of accurately aligning and integrating sensor data from different sources, such as Light Detection and Ranging (LiDAR) and cameras, to create a coherent representation of the vehicle's environment. The system includes a method for transforming multiple 3D points from a world coordinate frame to an inertial measurement unit (IMU) frame, then to a LiDAR frame, and finally to a camera frame. The IMU provides motion and orientation data, while the LiDAR sensor captures 3D point cloud data of the surroundings. The camera captures 2D images, which are used in conjunction with the transformed 3D points to enhance environmental perception. The transformation process ensures that the 3D points are accurately mapped across the different sensor frames, allowing for precise spatial alignment and fusion of data. This enables the vehicle to accurately perceive and interpret its environment, supporting functions such as object detection, tracking, and path planning. The system improves the reliability and accuracy of sensor fusion in autonomous vehicles by maintaining consistent coordinate transformations between sensors.
15. The vehicle system of claim 10 , wherein the actual 2D points are determined using at least one of image detection, object detection, and corner detection.
A vehicle system is designed to enhance navigation and object detection by determining the precise locations of 2D points within an environment. The system addresses challenges in accurately identifying and mapping key features in real-world scenarios, which is critical for autonomous driving, obstacle avoidance, and spatial awareness. The system determines the actual 2D points using at least one of image detection, object detection, or corner detection. Image detection involves analyzing visual data to identify specific features or patterns within an image. Object detection further refines this by recognizing and classifying distinct objects within the scene, such as vehicles, pedestrians, or road signs. Corner detection identifies key geometric features, such as intersections or edges, which are essential for mapping and spatial understanding. By integrating these detection methods, the system improves the accuracy and reliability of point localization, enabling more precise navigation and interaction with the environment. This approach ensures that the vehicle can effectively interpret and respond to its surroundings, enhancing safety and operational efficiency.
16. The vehicle system of claim 10 , wherein physical properties and intrinsic parameters of the virtual camera and the camera are the same.
This invention relates to vehicle systems that use virtual cameras to enhance real-world camera functionality. The system addresses the challenge of accurately simulating real-world camera behavior in virtual environments, ensuring consistency between physical and virtual camera outputs. The vehicle system includes a real camera mounted on the vehicle and a virtual camera that replicates the physical camera's properties and intrinsic parameters, such as field of view, focal length, and sensor characteristics. By matching these properties, the virtual camera generates images that are indistinguishable from those captured by the real camera, enabling seamless integration of virtual elements into real-world scenes. This synchronization improves applications like augmented reality, driver assistance systems, and autonomous vehicle navigation, where accurate visual data is critical. The system ensures that virtual overlays or simulations align perfectly with real-world conditions, enhancing reliability and user experience. The invention also includes a processing unit that dynamically adjusts the virtual camera's parameters to maintain consistency with the real camera, accounting for environmental changes or vehicle movements. This approach eliminates discrepancies between real and virtual imagery, providing a unified and accurate visual representation for various automotive applications.
17. The vehicle system of claim 10 , wherein the offset compensation matrix includes yaw, pitch and roll matrices.
A vehicle system is designed to improve navigation accuracy by compensating for misalignments between different sensor systems. The system includes a sensor fusion module that processes data from multiple sensors, such as inertial measurement units (IMUs), global navigation satellite systems (GNSS), and wheel speed sensors, to estimate vehicle motion. A key challenge in such systems is the misalignment or offset between these sensors, which can introduce errors in position, velocity, and orientation estimates. To address this, the system incorporates an offset compensation matrix that corrects for these misalignments. This matrix includes yaw, pitch, and roll matrices, which account for angular deviations in three-dimensional space. By applying these corrections, the system enhances the accuracy of sensor fusion, leading to more reliable navigation and positioning. The offset compensation matrix is dynamically adjusted based on real-time sensor data, ensuring continuous calibration and improved performance under varying conditions. This approach is particularly useful in autonomous vehicles, advanced driver-assistance systems (ADAS), and other applications where precise motion estimation is critical. The system may also include additional features, such as sensor fault detection and adaptive filtering, to further refine the accuracy of the navigation solution.
18. A method for compensating for autonomous vehicle (AV) system errors, the method comprising: obtaining projected 2D points from multiple 3D points for an object; obtaining actual 2D points for the object; calculating an error between the projected 2D points and the actual 2D points; executing a compensation algorithm when a calculated error is equal to or greater than a defined threshold, wherein the compensation algorithm comprises: transforming the multiple 3D points to a camera frame; establishing a virtual camera corresponding to the actual 2D points; determining an offset compensation matrix between a camera associated with the camera frame and the virtual camera; and applying the offset compensation matrix to data points prior to use by vehicle control systems; and controlling operation of the AV with offset compensated data points.
Autonomous vehicles (AVs) rely on accurate sensor data to navigate and make decisions, but errors in 3D point projections from sensors like LiDAR or cameras can lead to misalignment between projected and actual 2D points. This misalignment can degrade AV performance, causing navigation or safety issues. To address this, a method compensates for these errors by first obtaining projected 2D points derived from multiple 3D points of an object and comparing them to actual 2D points captured by the vehicle's sensors. If the error between projected and actual points exceeds a predefined threshold, a compensation algorithm is triggered. The algorithm transforms the 3D points into a camera reference frame, then establishes a virtual camera aligned with the actual 2D points. By comparing the real camera and virtual camera, an offset compensation matrix is generated. This matrix corrects the misalignment by adjusting the 3D points before they are used by the vehicle's control systems. The corrected data ensures more accurate object detection and tracking, improving AV decision-making and safety. The method dynamically compensates for sensor errors, enhancing reliability in real-world driving conditions.
19. The method of claim 18 , wherein the determining the offset compensation matrix further comprising: executing a camera pose algorithm using the multiple 3D points in the camera frame and the actual multiple 2D points in a virtual camera frame associated with the virtual camera to determine the offset compensation matrix.
This invention relates to a method for determining an offset compensation matrix in a camera-based system, particularly for applications involving virtual cameras and real-world camera calibration. The problem addressed is the misalignment between a real camera's captured 2D image points and corresponding 3D points in a virtual camera's frame, which can lead to inaccuracies in augmented reality, robotics, or computer vision tasks. The method involves using a set of 3D points in the real camera's frame and their corresponding 2D points in a virtual camera's frame to compute an offset compensation matrix. This matrix corrects the positional and rotational discrepancies between the real and virtual camera frames. The process includes executing a camera pose algorithm that processes the 3D and 2D point pairs to derive the compensation matrix, ensuring accurate alignment between the real and virtual camera perspectives. This technique is useful in systems where precise spatial mapping between physical and virtual environments is required, such as in augmented reality, autonomous navigation, or 3D reconstruction. The method improves the accuracy of virtual camera projections by compensating for misalignments, thereby enhancing the reliability of applications dependent on precise spatial relationships.
20. The method of claim 19 , wherein the offset compensation matrix includes translational matrices and rotation matrices.
This invention relates to a method for compensating for offsets in a system, particularly in applications requiring precise alignment or positioning, such as robotics, manufacturing, or navigation. The problem addressed is the need to accurately correct positional and rotational discrepancies that arise due to misalignments, manufacturing tolerances, or environmental factors. The method involves generating an offset compensation matrix that accounts for both translational and rotational deviations. The translational matrices adjust for linear displacements in one or more axes, while the rotation matrices correct for angular misalignments. These matrices are applied to input data or control signals to compensate for the offsets, ensuring accurate positioning or alignment. The compensation matrix may be derived from calibration data, sensor measurements, or predefined correction values. The method can be implemented in real-time systems or as part of a pre-processing step to enhance system accuracy. By incorporating both translational and rotational adjustments, the method provides a comprehensive solution for offset compensation, improving the precision of mechanical systems, robotic arms, or navigation devices. The approach is adaptable to various applications where alignment errors must be minimized.
Unknown
December 15, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.