Image Processing Apparatus, Image Processing System,control Method for Image Processing Apparatus, and Non-Transitory Computer-Readable Storage Medium for Mixed Reality

1. An image processing apparatus comprising: one or more first processors; and a first memory coupled to the one or more first processors and including first instructions stored thereon that, when executed by the one or more first processors, cause the image processing apparatus to function as: an image capturing unit configured to capture a real space; a detection unit configured to detect, from a plurality of image data captured by the image capturing unit, feature points to be used to obtain a position and an orientation of the image capturing unit; an extraction unit configured to extract each of feature amounts of the feature points detected by the detection unit; a generation unit configured to generate compressed image data by compressing a data amount of the image data; a transmission unit configured to transmit, to an external server, the compressed image data and data including position information of the feature points detected by the detection unit and the feature amounts extracted by the extraction unit; and a display unit configured to display composite image data generated in the external server based on the position information and the compressed image data by receiving the composite image data from the external server, wherein the image capturing unit captures a pair of images corresponding to right and left eyes, wherein the first instructions, when executed by the one or more first processors, further cause the image processing apparatus to function as: a determination unit configured to determine similarity of feature points between the pair of images based on the extracted feature amounts; a setting unit configured to set a transmission priority of data including the feature points and the feature amounts based on the similarity of the feature points; and a control unit configured to control a data amount transmitted by the transmission unit, wherein the transmission unit sequentially transmits the data including the position information of the feature points and the feature amounts in a descending order of the transmission priority of the feature points to the external server, and wherein the control unit measures a transmission amount of the compressed image data transmitted by the transmission unit and stops transmission of the data of the feature points and the feature amounts, when the transmission amount of the compressed image data exceeds a threshold.

2. The apparatus according to claim 1 , wherein the first instructions, when executed by the one or more first processors, cause the image processing apparatus to function as: a transmission control unit configured to control transmission of data by the transmission unit based on a data request instruction from the external server.

3. The apparatus according to claim 1 , wherein the first instructions, when executed by the one or more processors, cause the image processing apparatus to further function as: an object detection unit configured to detect object information from the image data, wherein the transmission unit further transmits the object information.

4. The apparatus according to claim 1 , wherein the image processing apparatus is a head mounted display.

5. An image processing system comprising: the image processing apparatus defined in claim 1 ; one or more second processors; a second memory coupled to the one or more second processors and including second instructions stored thereon that, when executed by the one or more second processors, cause the image processing system to function as: a deriving unit configured to derive position and orientation information of the image capturing unit based on position information of feature points transmitted from the image processing apparatus; a virtual image data generation unit configured to generate virtual image data based on the position and orientation information derived by the deriving unit; a composite image data generation unit configured to generate composite image data based on compressed image data and the virtual image data; and the external server configured to transmit the composite image data to the image processing apparatus.

6. The system according to claim 5 , wherein the external server includes a measurement unit configured to measure a position and an orientation of an object in a virtual space by matching, in a time-axis direction, the feature points received from the image processing apparatus.

7. A control method for an image processing apparatus, the method comprising: detecting, from a plurality of image data captured by an image capturing unit configured to capture a real space, feature points to be used to obtain a position and an orientation of the image capturing unit; extracting each of feature amounts of the detected feature points; generating compressed image data by compressing a data amount of the image data; transmitting, to an external server, the compressed image data and data including position information of the feature points detected in the detection step and the feature amounts extracted in the extraction step; and displaying composite image data generated in the external server based on the position information and the compressed image data by receiving the composite image data from the external server, wherein the image capturing unit captures a pair of images corresponding to left and right eyes, wherein the control method further comprises: determining similarity of feature points between the pair of images based on the extracted feature amounts; and setting a transmission priority of data including the feature points and the feature amounts based on the similarity of the feature points; and controlling a transmitted data amount, wherein the data including the position information of the feature points and the feature amounts are sequentially transmitted to the external server in a descending order of the transmission priority of the feature points, wherein a transmission amount of the compressed image data is measured, and wherein transmission of the data of the feature points and the feature amounts is stopped if when the transmission amount of the compressed image data exceeds a threshold.

8. A non-transitory computer-readable storage medium storing a computer program for causing a computer to function as: a detection unit configured to detect, from a plurality of image data captured by an image capturing unit configured to capture a real space, feature points to be used to obtain a position and an orientation of the image capturing unit; an extraction unit configured to extract each of feature amounts of the feature points detected by the detection unit; a generation unit configured to generate compressed image data by compressing a data amount of the image data; a transmission unit configured to transmit, to an external server, the compressed image data and data including position information of the feature points detected by the detection unit and the feature amounts extracted by the extraction unit; and a display unit configured to display composite image data generated in the external server based on the position information and the compressed image data by receiving the composite image data from the external server, wherein the image capturing unit captures a pair of images corresponding to left and right eyes, wherein the computer program further causes the computer to function as: a determination unit configured to determine similarity of feature points between the pair of images based on the extracted feature amounts; and a setting unit configured to set a transmission priority of data including the feature points and the feature amounts based on the similarity of the feature points; and a control unit configured to control a data amount transmitted by the transmission unit, wherein the transmission unit sequentially transmits to the external server the data including the position information of the feature points and the feature amounts in a descending order of the transmission priority of the feature points and wherein the control unit measures a transmission amount of the compressed image data transmitted by the transmission unit, and wherein the control unit measures a transmission amount of the compressed image data transmitted by the transmission unit and stops transmission of the data of the feature points and the feature amounts, when the transmission amount of the compressed image data exceeds a threshold.

9. An image processing apparatus comprising: one or more processors; a memory coupled with the one or more processors and including instructions stored thereon that, when executed by the one or more processors, cause the image processing apparatus to function as: a detection unit configured to detect feature points from a pair of images corresponding to left and right eyes; an extraction unit configured to extract each of feature amounts of the feature points for the pair of images; a determination unit configured to determine similarity of the feature points between the pair of images based on the extracted feature amounts; a setting unit configured to set a transmission priority of data including position information of the feature points and the feature amounts based on the similarity of the feature points; a transmission unit configured to sequentially transmit the data in a descending order of the transmission priority of the feature points to an external server; and a control unit configured to control a data amount transmitted by the transmission unit, wherein the control unit measures a transmission amount of the data transmitted by the transmission unit and stops transmission of the data of the feature points and the feature amounts, when the transmission amount of the data exceeds a threshold.

10. A control method for an image processing apparatus, comprising: detecting feature points from a pair of images corresponding to left and right eyes; extracting each of feature amounts of the feature points for the pair of images; determining similarity of the feature points between the pair of images based on the extracted feature amounts; setting a transmission priority of data including position information of the feature points and the feature amounts based on the similarity of the feature points; sequentially transmitting the data in a descending order of the transmission priority of the feature points to an external server; and controlling a transmitted data amount, wherein a transmission amount of the data is measured, and wherein transmission of the data of the feature points and the feature amounts is stopped when the transmission amount of the data exceeds a threshold.

11. A non-transitory computer-readable storage medium storing a computer program for causing a computer to function as: a detection unit configured to detect feature points from a pair of images corresponding to left and right eyes; an extraction unit configured to extract each of feature amounts of the feature points for the pair of images; a determination unit configured to determine similarity of the feature points between the pair of images based on the extracted feature amounts; a setting unit configured to set a transmission priority of data including position information of the feature points and the feature amounts based on the similarity of the feature points; a transmission unit configured to sequentially transmit the data in a descending order of the transmission priority of feature points to an external server; and a control unit configured to control a data amount transmitted by the transmission unit, wherein the control unit measures a transmission amount of the data transmitted by the transmission unit and stops transmission of the data of the feature points and the feature amounts, when transmission amount of the data exceeds a threshold.