Multiple Camera System with Auto Recalibration

1. A method for a computing device to continually recalibrate a multiple camera system, the method comprises: for each new picture captured by each camera of the multiple camera system, wherein the multiple camera system includes two or more cameras, and wherein each camera of the two or more cameras is physically separated by a distance from another camera of the two or more cameras: generating new image data; generating new calibration data based on the new image data, wherein the new calibration data includes a new set of matching interest point tuples from each camera, and wherein generating the new set of matching interest point tuples includes: analyzing the new image data from each camera to identify a plurality of interest point tuples; removing erroneous interest point tuples from the plurality of interest point tuples based on coordinate system mapping inconsistencies of the multiple camera system to produce the new set of matching interest point tuples; updating existing calibration data regarding the multiple camera system to include the new calibration data to produce updated calibration data; and detecting for decalibration based on the updated calibration data; and when the decalibration is detected: generating recalibration parameters based on one or more of the existing calibration data and the updated calibration data; determining whether the recalibration parameters are valid parameters; when the recalibration parameters are valid parameters, updating the multiple camera system based on the recalibration parameters; and when the recalibration parameters are not valid parameters: generating additional calibration data to determine the recalibration parameters by one or more of: generating one or more additional pictures having sufficient textural diversity; and generating infrared (IR) signal image data, wherein one or more cameras of the multiple camera system is operable to transmit the IR signal.

2. The method of claim 1 , wherein the removing erroneous interest point tuples comprises at least one of: identifying an erroneous interest point tuple of the erroneous interest point tuples when a first interest point of the erroneous interest point tuple of a first camera of the two of more cameras is outside of an epipolar band of a second interest point of the erroneous interest point tuple of a second camera of the two of more cameras; identifying the erroneous interest point tuple when the first interest point matches more than one interest point of the second camera; identifying the erroneous interest point tuple when a distance metric between the first interest point and the second interest point is greater than a distance threshold; and utilizing an essential matrix equation on the plurality of interest point tuples to identify the erroneous interest point tuples.

3. The method of claim 1 , wherein the detecting the decalibration comprises: detecting occurrence of one or more physical events from a set of physical events occurring to the device, wherein the set of physical event includes dropping, rapid acceleration, rapid deceleration, flexing, a temperature increase, a temperature decrease, a use milestone, and an aging milestone.

4. The method of claim 1 , wherein the detecting decalibration comprises: calculating a set of epipolar liner errors for at least some matching interest point tuples of the new set of matching interest point tuples; determining a decalibration metric based on the set of epipolar line errors; when the decalibration metric exceeds a decalibration threshold: determining whether the new picture includes a sufficient level of textural diversity to render a reliable indication of the recalibration event; and when the new picture includes the sufficient level of textural diversity, indicating the decalibration.

5. The method of claim 4 further comprises: when the new picture does not include the sufficient level of textural diversity: capturing an additional picture; calculating an additional set of epipolar line errors for at least some matching interest point tuples of an additional set of matching interest point tuples of the additional picture; determining an additional decalibration metric based on the additional set of epipolar line errors; when the additional decalibration metric exceeds a decalibration threshold: determining whether the picture and the additional picture collectively include the sufficient level of textural diversity to render the reliable indication of the recalibration event; and when the picture and the additional picture collectively include the sufficient level of textural diversity, indicating the decalibration.

6. The method of claim 1 , wherein the generating the recalibration parameters based on the one or more of the existing calibration data and the updated calibration data comprises: using the existing calibration data for recalibration parameter calculations; and calculating extrinsic parameters for the multiple camera system using an estimated essential matrix and the updated calibration data.

7. The method of claim 6 , wherein the calculating the extrinsic parameters comprises: generating the estimated essential matrix based on a set of linear equations, wherein a linear equation of the set of linear equations includes a first coordinate point of a first camera of the two or more cameras, a second coordinate point of a second camera of the two or more cameras, a first matrix of the first camera, a second matrix of the second camera, and a first essential matrix of a first interest point tuple of a set of matching interest point tuples, wherein the first interest point tuple includes the first and second coordinates points; decomposing the estimated essential matrix into a set of solutions, wherein each solution of the set of solutions includes a rotation matrix and a translation vector; for each solution of the set of solutions, triangulating interest point tuples of the calibration data into a three-dimensional coordinate system to produce triangulated points; identifying the solution of the set of solution that provides valid results for the triangulated interest points to produce an identified solution; and establishing the extrinsic parameters as the rotation matrix and translation vector of the identified solution.

8. The method of claim 6 , wherein the calculating the extrinsic parameters comprises: generating the estimated essential matrix based on a set of linear equations, wherein a linear equation of the set of linear equations includes a fundamental matrix, a first matrix of the first camera, a second matrix of the second camera, and a first essential matrix of a first point tuple of the set of matching interest point tuples, wherein an interest point tuple includes the point and the corresponding point; decomposing the estimated essential matrix into a set of solutions, wherein each solution of the set of solutions includes a rotation matrix and a translation vector; for each solution of the set of solutions, triangulating interest point tuples of the calibration data into a three-dimensional coordinate system to produce triangulated interest points; identifying the solution of the set of solution that provides valid results for the triangulated interest points to produce an identified solution; and establishing the extrinsic parameters as the rotation matrix and translation vector of the identified solution.

9. The method of claim 6 further comprises: obtaining camera parameter constraints regarding the two or more cameras; and calculating intrinsic parameters and extrinsic parameters for each camera of the two or more cameras using a Bayesian formulation, the camera parameter constraints, and the calibration data.

10. The method of claim 1 , wherein the determining whether the recalibration parameters are valid parameters comprises: determining that the recalibration parameters converge to values that are within an expected range of parameter values.

11. The method of claim 1 further comprises: when the decalibration is detected, providing a message for display on the computing device, wherein the message indicates decalibration of the multiple camera system.

12. A computing device comprises: a multiple camera system that includes two or more cameras, wherein each camera of the two or more cameras is physically separated by a distance from another camera of the two or more cameras; memory; and a processing module operably coupled to the two or more cameras and the memory, wherein the processing module is operable to continually recalibrate the multiple camera system by: for each new picture captured by each camera of the multiple camera system: generate new image data; store the new image data in the memory; generate new calibration data based on the new image data, wherein the new calibration data includes a new set of matching interest point tuples from each camera, and wherein generating the new set of matching interest point tuples includes: analyzing the new image data from each camera to identify a plurality of interest point tuples; removing erroneous interest point tuples from the plurality of interest point tuples based on coordinate system mapping inconsistencies of the multiple camera system to produce the new set of matching interest point tuples; updating existing calibration data regarding the multiple camera system to include the new calibration data to produce updated calibration data; and store the updated calibration data in the memory; detect for decalibration based on the updated calibration data; and when the decalibration is detected: generate recalibration parameters based on one or more of the existing calibration data and the updated calibration data; determine whether the recalibration parameters are valid parameters; when the recalibration parameters are valid parameters, update the multiple camera system based on the recalibration parameters; and when the recalibration parameters are not valid parameters: generate additional calibration data to determine the recalibration parameters by one or more of: generating one or more additional pictures having sufficient textural diversity; and utilizing infrared (IR) signal image data, wherein one or more cameras of the multiple camera system is operable to transmit the IR signal.

13. The computing device of claim 12 , wherein the processing module is further operable to remove the erroneous interest point tuples by at least one of: identifying an erroneous interest point tuple of the erroneous interest point tuples when a first interest point of the erroneous interest point tuple of a first camera of the two of more cameras is outside of an epipolar band of a second interest point of the erroneous interest point tuple of a second camera of the two of more cameras; identifying the erroneous interest point tuple when the first interest point matches more than one interest point of the second camera; identifying the erroneous interest point tuple when a distance metric between the first interest point and the second point of interest is greater than a distance threshold; and utilizing an essential matrix equation on the plurality of interest point tuples to identify the erroneous interest point tuples.

14. The computing device of claim 12 , wherein the processing module is further operable to detect the decalibration by: detecting occurrence of one or more physical events from a set of physical events occurring to the device, wherein the set of physical event includes dropping, rapid acceleration, rapid deceleration, flexing, a temperature increase, a temperature decrease, a use milestone, and an aging milestone.

15. The computing device of claim 12 , wherein the processing module is further operable to detect the decalibration by: calculating a set of epipolar line errors for at least some matching interest point tuples of the new set of matching interest point tuples; determining a decalibration metric based on the set of epipolar line errors; when the decalibration metric exceeds a decalibration threshold: determining whether the new picture includes a sufficient level of textural diversity to render a reliable indication of the recalibration event; and when the new picture includes the sufficient level of textural diversity, indicating the decalibration.

16. The computing device of claim 15 , wherein the processing module is further operable to: when the new picture does not include the sufficient level of textural diversity: capture an additional picture; calculate an additional set of epipolar liner errors for at least some matching interest point tuples of an additional set of matching interest point tuples of the additional picture; determine an additional decalibration metric based on the additional set of epipolar line errors; when the additional decalibration metric exceeds a decalibration threshold: determine whether the new picture and the additional picture collectively include the sufficient level of textural diversity to render the reliable indication of the recalibration event; and when the new picture and the additional picture collectively include the sufficient level of textural diversity, indicate the decalibration.

17. The computing device of claim 12 , wherein the processing module is further operable to generate the recalibration parameters based on the one or more of the existing calibration data and the updated calibration data by: using the existing calibration data for recalibration parameter calculations; and calculating extrinsic parameters for each camera of the two or more cameras using an estimated essential matrix and the updated calibration data.

18. The computing device of claim 17 , wherein the processing module is further operable to calculate the extrinsic parameters by: generating the estimated essential matrix based on a set of linear equations, wherein a linear equation of the set of linear equations includes a first coordinate point of a first camera of the two or more cameras, a second coordinate point of a second camera of the two or more cameras, a first matrix of the first camera, a second matrix of the second camera, and a first essential matrix of a first interest point tuple of a set of matching interest point tuples, wherein the first interest point tuple includes the first and second coordinates points; decomposing the estimated essential matrix into a set of solutions, wherein each solution of the set of solutions includes a rotation matrix and a translation vector; for each solution of the set of solutions, triangulating interest point tuples of the calibration data into a three-dimensional coordinate system to produce triangulated interest points; identifying the solution of the set of solution that provides valid results for the triangulated interest points to produce an identified solution; and establishing the extrinsic parameters as the rotation matrix and translation vector of the identified solution.

19. The computing device of claim 17 , wherein the processing module is further operable to calculate the extrinsic parameters by: generating the estimated essential matrix based on a set of linear equations, wherein a linear equation of the set of linear equations includes a fundamental matrix, a first matrix of the first camera, a second matrix of the second camera, and a first essential matrix of a first interest point tuple of the set of matching interest point tuples, wherein an interest point tuple includes the interest point and the corresponding interest point; decomposing the estimated essential matrix into a set of solutions, wherein each solution of the set of solutions includes a rotation matrix and a translation vector; for each solution of the set of solutions, triangulating interest point tuples of the calibration data into a three-dimensional coordinate system to produce triangulated interest points; identifying the solution of the set of solution that provides valid results for the triangulated interest points to produce an identified solution; and establishing the extrinsic parameters as the rotation matrix and translation vector of the identified solution.

20. The computing device of claim 17 , wherein the processing module is further operable to: obtain camera parameter constraints regarding the two or more cameras; and calculate intrinsic parameters and extrinsic parameters for each camera of the two or more cameras using a Bayesian formulation, the camera parameter constraints, and the calibration data.

21. The computing device of claim 12 , wherein the processing module is further operable to determine whether the recalibration parameters are valid parameters by: determining that the recalibration parameters converge to values that are within an expected range of parameter values.

22. A computer readable storage device comprises: for each new picture captured by each camera of a multiple camera system wherein the multiple camera system includes two or more cameras: a first memory section that stores operational instructions that, when executed by a computing device, causes the computing device to: generate new image data; and store the new image data in memory of the computing device; generate new calibration data based on the new image data, wherein the calibration data includes a new set of matching interest point tuples from each camera, and wherein generating the set of matching interest point tuples includes: analyzing the new image data from each camera to identify a plurality of interest point tuples; removing erroneous interest point tuples from the plurality of interest point tuples based on coordinate system mapping inconsistencies of the multiple camera system to produce the new set of matching interest point tuples; update existing calibration data regarding the multiple camera system to include the new calibration data to produce updated calibration data; and store the updated calibration data in the memory of the computing device; and a second memory section that stores operational instructions that, when executed by the computing device, causes the computing device to: detect for decalibration based on the updated calibration data; and a third memory section that stores operational instructions that, when executed by the computing device, causes the computing device to: when the decalibration is detected: generate recalibration parameters based on one or more of the existing calibration data and the updated calibration data; determine whether the recalibration parameters are valid parameters; when the recalibration parameters are valid parameters, update the multiple camera system based on the recalibration parameters; and when the recalibration parameters are not valid parameters: generate additional calibration data to determine the recalibration parameters by one or more of: generating one or more additional pictures having sufficient textural diversity; and utilizing infrared (IR) signal image data, wherein one or more cameras of the multiple camera system is operable to transmit the IR signal.

23. The computer readable storage device of claim 22 , wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to remove the erroneous interest point tuples by at least one of: identifying an erroneous interest point tuple of the erroneous interest point tuples when a first interest point of the erroneous interest point tuple of a first camera of the two of more cameras is outside of an epipolar band of a second interest point of the erroneous interest point tuple of a second camera of the two of more cameras; identifying the erroneous interest point tuple when the first interest point matches more than one interest point of the second camera; identifying the erroneous interest point tuple when a distance metric between the first interest point and the second interest point is greater than a distance threshold; and utilizing an essential matrix equation on the plurality of interest point tuples to identify the erroneous interest point tuples.

24. The computer readable storage device of claim 22 , wherein the second memory section further stores operational instructions that, when executed by the computing device, causes the computing device to detect the decalibration by: detecting occurrence of one or more physical events from a set of physical events occurring to the device, wherein the set of physical event includes dropping, rapid acceleration, rapid deceleration, flexing, a temperature increase, a temperature decrease, a use milestone, and an aging milestone.

25. The computer readable storage device of claim 22 , wherein the second memory section further stores operational instructions that, when executed by the computing device, causes the computing device to detect the decalibration by: calculating a set of epipolar liner errors for at least some matching interest points of the new set of matching interest point tuples; determining a decalibration metric based on the set of epipolar line errors; when the decalibration metric exceeds a decalibration threshold: determining whether the new picture includes a sufficient level of textural diversity to render a reliable indication of the recalibration event; and when the new picture includes the sufficient level of textural diversity, indicating the decalibration.

26. The computer readable storage device of claim 25 , wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to: when the new picture does not include the sufficient level of textural diversity: capture an additional picture; calculate an additional set of epipolar liner errors for at least some matching interest points of an additional set of matching interest point tuples of the additional picture; determine an additional decalibration metric based on the additional set of epipolar line errors; when the additional decalibration metric exceeds a decalibration threshold: determine whether the new picture and the additional picture collectively include the sufficient level of textural diversity to render the reliable indication of the recalibration event; and when the picture and the additional picture collectively include the sufficient level of textural diversity, indicate the decalibration.

27. The computer readable storage device of claim 22 , wherein the third memory section further stores operational instructions that, when executed by the computing device, causes the computing device to generate the recalibration parameters based on the one or more of the existing calibration data and the updated calibration data by: using the existing calibration data for recalibration parameter calculations; and calculating extrinsic parameters for each camera of the two or more cameras using an estimated essential matrix and the updated calibration data.

28. The computer readable storage device of claim 27 , wherein the third memory section further stores operational instructions that, when executed by the computing device, causes the computing device to calculate the extrinsic parameters by: generating the estimated essential matrix based on a set of linear equations, wherein a linear equation of the set of linear equations includes a first coordinate point of a first camera of the two or more cameras, a second coordinate point of a second camera of the two or more cameras, a first matrix of the first camera, a second matrix of the second camera, and a first essential matrix of a first interest point tuple of a set of matching interest point tuples, wherein the first interest point tuple includes the first and second coordinates points; decomposing the estimated essential matrix into a set of solutions, wherein each solution of the set of solutions includes a rotation matrix and a translation vector; for each solution of the set of solutions, triangulating interest point tuples of the calibration data into a three-dimensional coordinate system to produce triangulated interest points; identifying the solution of the set of solution that provides valid results for the triangulated interest points to produce an identified solution; and establishing the extrinsic parameters as the rotation matrix and translation vector of the identified solution.

29. The computer readable storage device of claim 27 , wherein the third memory section further stores operational instructions that, when executed by the computing device, causes the computing device to calculate the extrinsic parameters by: generating the estimated essential matrix based on a set of linear equations, wherein a linear equation of the set of linear equations includes a fundamental matrix, a first matrix of the first camera, a second matrix of the second camera, and a first essential matrix of a first interest point tuple of the set of matching interest point tuples, wherein an interest point tuple includes the interest point and the corresponding interest point; decomposing the estimated essential matrix into a set of solutions, wherein each solution of the set of solutions includes a rotation matrix and a translation vector; for each solution of the set of solutions, triangulating interest point tuples of the calibration data into a three-dimensional coordinate system to produce triangulated interest points; identifying the solution of the set of solution that provides valid results for the triangulated interest points to produce an identified solution; and establishing the extrinsic parameters as the rotation matrix and translation vector of the identified solution.

30. The computer readable storage device of claim 27 , wherein the third memory section further stores operational instructions that, when executed by the computing device, causes the computing device to: obtain camera parameter constraints regarding the two or more cameras; and calculate intrinsic parameters and extrinsic parameters for each camera of the two or more cameras using a Bayesian formulation, the camera parameter constraints, and the calibration data.

31. The computer readable storage device of claim 22 , wherein the third memory section further stores operational instructions that, when executed by the computing device, causes the computing device to determine whether the recalibration parameters are valid parameters by: determining that the recalibration parameters converge to values that are within an expected range of parameter values.

32. The computer readable storage device of claim 22 , wherein the third memory section further stores operational instructions that, when executed by the computing device, causes the computing device to: when the decalibration is detected, display a message that indicates decalibration of the multiple camera system.