Information Processing Device and Information Processing Method

This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2025-023386, filed on Feb. 17, 2025, the entire contents of which are incorporated herein by reference.

Embodiments described herein relate to an information processing device and an information processing method.

A distance detection device can accurately detect the distance to an object due to evolution of light detection & ranging (LiDAR) technology, and thus is applied to a wide range of fields, such as an autonomous driving device and an object recognition device.

However, in LiDAR, the distance to an object is detected based on results of repeatedly receiving reflected light from the object, and accordingly, the amount of noise included in light received by LiDAR varies depending on the distance to the object, reflectance of the object, the presence of ambient light such as sunlight, and the like. The output level of a light receiving element of LiDAR varies depending on the presence of light reception, but when a threshold for determining the output level is set to a constant level, reflected light from the object cannot be accurately detected due to variation in the amount of noise, and the accuracy of distance detection of the object potentially decreases.

According to one embodiment, an information processing device includes a distance detector configured to generate first point cloud data including distance information of an object included in original point cloud data; a difference extractor configured to extract third point cloud data from a difference between the first point cloud data and previously acquired second point cloud data; and a difference adjuster configured to generate an index to adjust an extraction amount of the difference, wherein the difference extractor extracts the third point cloud data from the difference based on the index.

Embodiments of an information processing device and an information processing method will be described below with reference to the accompanying drawings. The following description is made with focus on a main part of the information processing device, but the information processing device may include components or functions that are not illustrated or described. The following description does not exclude illustrated or described components or functions.

1 FIG. 1 FIG. 1 1 2 3 4 is a block diagram illustrating a schematic configuration of an information processing deviceaccording to a first embodiment. As illustrated in, the information processing deviceaccording to the first embodiment includes a distance detector, a difference extractor, and a difference adjuster.

5 2 5 5 5 5 1 5 1 Original point cloud data (RAW data) output from a light detectoris input to the distance detector. The light detectoremits light to an object and receives light reflected by the object. For example, the light detectorscans the emission direction of light in a one-dimensional or two-dimensional direction and receives reflected light from an object present in a three-dimensional space. The light detectormay also have a function of variably controlling the scanning range of light. The light detectoris, for example, a LiDAR device provided separately from the information processing device. Alternatively, the light detectormay be incorporated in the information processing device.

2 The distance detectorgenerates first point cloud data including object distance information included in the original point cloud data. The density of the first point cloud data becomes coarse as the distance increases, and the density of the first point cloud data becomes dense as the distance decreases. In this manner, the object distance information can be detected by the coarseness and denseness of the first point cloud data. The first point cloud data is data in which the distance information is added to data of each point. More precisely, each point has orientation information (φ and θ components in polar coordinates) corresponding to the irradiation direction of a laser beam from the LiDAR device, and distance information d to an object present in the orientation of the point. A distance image is an image in which the distance information d of each point is arranged as a two-dimensional image. The first point cloud data may include the distance image and may include data obtained by converting the polar coordinates φ and θ and the distance information d of each point into XYZ coordinates.

3 2 5 The difference extractorextracts third point cloud data from the difference between the first point cloud data generated by the distance detectorand second point cloud data acquired in the past (i.e., previously acquired second point cloud data). The second point cloud data acquired in the past may be point cloud data acquired in the past, or may be point cloud data obtained by performing averaging processing on the point cloud data acquired in the past. In the present specification, the difference between the first point cloud data and the second point cloud data is also referred to as difference point cloud data. The first point cloud data is generated based on the original point cloud data output from the light detector.

1 The second point cloud data is a three-dimensional background image of one frame and is data in which a point cloud does not change with time. The information processing deviceaccording to the first embodiment may include a non-illustrated storage configured to store the second point cloud data.

The first point cloud data is a three-dimensional distance image including the background image represented by the second point cloud data. More specifically, the first point cloud data is a distance image obtained by adding the object distance information to the background image. The difference point cloud data between the first point cloud data and the second point cloud data indicates a distance difference for each pixel between the distance image and the background image.

3 The third point cloud data extracted from the difference point cloud data is point cloud data obtained by extracting at least part of the difference (difference point cloud data) between the first point cloud data and the second point cloud data. In other words, the third point cloud data is a difference image between the distance image and the background image. As described later, for example, the difference extractorextracts the third point cloud data from the difference point cloud data by removing noise included in the difference point cloud data.

4 4 4 The difference adjustergenerates an index for adjusting the extraction amount of the difference (difference point cloud data). For example, the difference adjustergenerates the above-described index based on the first point cloud data or two-dimensional image data. Alternatively, the difference adjustermay generate the above-described index based on an object recognition result.

The index is, for example, a threshold for comparing with the difference (difference point cloud data). Alternatively, the index is the size of a voxel (hereinafter referred to as a voxel size). A voxel refers to a three-dimensional unit region in which the first point cloud data and the second point cloud data in a three-dimensional space are divided into sizes in accordance with the object distance. Each voxel includes the divided first point cloud data or second point cloud data. Each voxel has a size in accordance with the object distance corresponding to point cloud data included inside the voxel.

3 3 In a case where the index is the voxel size, the difference extractorcalculates a difference between a plurality of voxels in which the first point cloud data is divided and a plurality of voxels in which the second point cloud data is divided. For example, in a case where the first point cloud data includes point cloud data of an object that is not present in the background image, the first point cloud data includes a new voxel that is not present in the plurality of voxels in which the second point cloud data is divided. By calculating the difference, a voxel that is not present in the plurality of voxels in which the second point cloud data is divided is taken out. In a case where the index is the voxel size, the difference extractorextracts, as the third point cloud data, point cloud data in a voxel obtained by calculating the difference.

1 FIG. 1 6 6 3 3 6 As illustrated in, the information processing deviceaccording to the first embodiment may include an object recognizer. The object recognizerrecognizes an object based on the third point cloud data extracted by the difference extractor. As described above, the third point cloud data is point cloud data extracted from the difference (difference point cloud data) between the first point cloud data constituting the distance image and the second point cloud data constituting the background image, and object information is included in the third point cloud data. Since the difference extractorextracts the third point cloud data exceeding a threshold from the difference point cloud data, the third point cloud data is data in which a noise component is reduced. Thus, the object recognizercan accurately recognize an object from the third point cloud data.

1 The information processing deviceaccording to the first embodiment may only detect presence of an object based on the third point cloud data without performing object recognition.

In this manner, in the first embodiment, an index for adjusting the extraction amount of the difference (difference point cloud data) is generated, and the third point cloud data is extracted from the difference (difference point cloud data) by using the generated index. Accordingly, for example, the index can be adjusted in accordance with the amount of noise included in the first point cloud data, and as a result, the third point cloud data becomes less susceptible to influence of noise. Thus, an object can be accurately recognized from the third point cloud data.

2 FIG. 2 FIG. 1 FIG. 1 FIG. 1 FIG. 1 1 2 4 2 4 4 is a block diagram illustrating a schematic configuration of the information processing deviceaccording to a second embodiment. The information processing deviceaccording to the second embodiment illustrated inhas the same block configuration as in, but the first point cloud data generated by the distance detectoris input to the difference adjuster. The first point cloud data generated by the distance detectoris not necessarily input to the difference adjusterin, and thus an arrow line illustrating an input path of the difference adjusteris omitted in.

2 4 3 2 FIG. Based on the distance information detected by the distance detector, the difference adjusteringenerates, as the index, a threshold for extracting the third point cloud data from the difference point cloud data between the first point cloud data and the second point cloud data. The difference extractorextracts the third point cloud data based on the difference point cloud data exceeding the threshold. The difference point cloud data is data representing the distance difference for each pixel between the distance image and the background image, and the third point cloud data constituting the difference image is point cloud data of pixels having a distance difference equal to or greater than the threshold.

3 FIG. 3 FIG. 3 FIG. 3 FIG. is a diagram illustrating the correspondence relation between the object distance and the threshold in the second embodiment. In, the horizontal axis represents the object distance, and the vertical axis represents the threshold. As illustrated in, the threshold increases as the distance increases, and the threshold decreases as the distance decreases. As the object distance increases, the density of the first point cloud data becomes coarser and the amount of noise increases, and thus the threshold is increased.illustrates an example in which the distance and the threshold have a linear relation, but the relation may be non-linear.

4 FIG. 4 FIG. 4 FIG. 4 FIG. 4 FIG. 4 FIG. 4 4 4 FIGS.A,B, andC 3 1 2 3 1 2 1 2 3 is a diagram for description of processing by the difference extractor. The left part A ofis a diagram illustrating an example of a distance image IGrepresented by the first point cloud data generated by the distance detector. The right part B ofis a diagram illustrating an example of a background image IGrepresented by the second point cloud data. The bottom part C ofis a diagram illustrating a difference image IGbetween the distance image IGin the left part A ofand the background image IGin the right part B of. The distance image IG, the background image IG, and the difference image IGare all point cloud data, but notations of point clouds are omitted infor simplification.

3 3 1 2 4 FIG. 4 FIG. 4 FIG. The difference extractorextracts the third point cloud data (the bottom part of) constituting the difference image IGand exceeding the threshold from the difference point cloud data between the distance image IGrepresented by the first point cloud data in the left part A ofand the background image IGrepresented by the second point cloud data in the right part B of.

5 FIG. 3 3 1 2 1 is a diagram for description of processing operation of the difference extractor. The difference extractorsequentially selects, as a target pixel SPX, each of a plurality of pixels constituting the distance image IGrepresented by the first point cloud data, and deletes the target pixel SPX from candidates of the difference image if at least one pixel the distance difference of which from the target pixel SPX exceeds the threshold is present in a peripheral pixel region PPA centered at a pixel of the background image IGrepresented by the second point cloud data and corresponding to the selected target pixel SPX. Accordingly, a noise component included in the distance image IGcan be efficiently deleted, and the amount of the third point cloud data constituting the difference image can be reduced.

6 FIG.A 6 FIG.B 6 FIG.C is a diagram illustrating the difference image in a case where the peripheral pixel region of the background image is one-by-one pixels,is a diagram illustrating the difference image in a case where the peripheral pixel region is three-by-three pixels, andis a diagram illustrating the difference image in a case where the peripheral pixel region is five-by-five pixels.

6 6 FIGS.A toC As understood from, a larger amount of the third point cloud data included in the difference image can be reduced as the area of the peripheral pixel region of the background image, which is compared with the target pixel of the distance image, is larger.

3 5 FIG. Note that the processing operation of the difference extractordoes not necessarily need to be performed by the method of. For example, whether to include the target pixel in the difference image may be determined depending on whether a distance difference based on the difference between an average value of pixels in the peripheral pixel region and the target pixel exceeds a threshold.

In this manner, in the second embodiment, a threshold for extracting the third point cloud data from the difference point cloud data is adjusted in accordance with the object distance, and thus the third point cloud data becomes less susceptible to influence of noise by, for example, increasing the threshold in a case where the object distance is long as compared to a case where the object distance is small. Accordingly, according to the second embodiment, variation of the extraction accuracy of the third point cloud data can be reduced even when the object distance changes.

7 FIG. 7 FIG. 2 FIG. 1 1 7 is a block diagram illustrating a schematic configuration of the information processing deviceaccording to a third embodiment. As illustrated in, the information processing deviceaccording to the third embodiment includes a luminance detectorin addition to the configuration in.

5 The light detectoroutputs the original point cloud data (RAW data) as well as two-dimensional image data. The two-dimensional image data is data having luminance information for each pixel.

7 7 The luminance detectordetects luminance information of each pixel based on the two-dimensional image data. The luminance detectormay determine the luminance value of a target pixel by, for example, averaging the luminance values of a plurality of pixels around the target pixel.

7 4 4 Based on the luminance information detected for each pixel by the luminance detector, the difference adjustergenerates, as the index, a threshold for extracting the third point cloud data from the difference point cloud data. For example, for a pixel having a high luminance value, the difference adjusterlowers the threshold for a corresponding portion of the difference point cloud data.

8 FIG. 8 FIG. 7 FIG. is a diagram illustrating the correspondence relation between the object luminance and the threshold in the third embodiment. In, the horizontal axis represents the object luminance, and the vertical axis represents the threshold. As illustrated in, the threshold decreases as the luminance increases, and the threshold increases as the luminance decreases. In a case where the luminance is high, distinction from noise included in the difference point cloud data is easy, and thus the threshold is lowered. On the other hand, in a case where the luminance is low, distinction from noise included in the difference point cloud data is difficult, and thus the threshold is increased.

In this manner, in the third embodiment, a threshold for extracting the third point cloud data from the difference point is adjusted in accordance with the luminance, and thus the extraction accuracy of the third point cloud data can be improved.

9 FIG. 9 FIG. 7 FIG. 9 FIG. 7 FIG. 9 FIG. 7 FIG. 1 1 4 4 3 3 is a block diagram illustrating a schematic configuration of the information processing deviceaccording to a fourth embodiment. As illustrated in, the information processing deviceaccording to the fourth embodiment has the same block configuration as in, but the difference adjusterinperforms processing operation different from that of the difference adjusterin, and the difference extractorinperforms processing operation different from that of the difference extractorin.

4 4 9 FIG. Based on the distance information and the luminance information, the difference adjusteringenerates, as the index, a threshold for extracting the third point cloud data from the difference point cloud data. More specifically, the difference adjustervariably controls the correspondence relation between the object distance and the threshold in accordance with the object luminance.

3 3 9 FIG. The difference extractorinextracts the third point cloud data based on the difference point cloud data exceeding the threshold. More specifically, the difference extractorvariably controls the threshold in accordance with the object distance and the object luminance.

10 FIG. 10 FIG. 10 FIG. 10 FIG. 1 2 1 2 1 2 is a diagram illustrating the correspondence relation among the object distance and the object luminance and the threshold in the fourth embodiment. In, the horizontal axis represents the object distance, and the vertical axis represents the threshold.illustrates a correspondence relation win a case of high luminance and a correspondence relation win a case of low luminance. The correspondence relations wand whave mutually different gradients. In other words, the degree of change of the threshold when the object distance changes is different between high luminance and low luminance. The degree of change of the threshold is smaller as the object luminance is higher. The correspondence relations wand winare merely exemplary.

In this manner, in the fourth embodiment, the threshold is variably controlled in accordance with the object distance and the object luminance, and thus an optimum threshold can be set in accordance with the object distance and the object luminance, and the accuracy of extraction of the third point cloud data from the difference point cloud data can be further improved.

11 FIG. 11 FIG. 1 1 2 3 8 4 is a block diagram illustrating a schematic configuration of the information processing deviceaccording to a fifth embodiment. As illustrated in, the information processing deviceaccording to the fifth embodiment includes the distance detector, the difference extractorincluding a voxel generator, and the difference adjuster.

2 The distance detectorgenerates the first point cloud data including the object distance information included in the original point cloud data (RAW data).

8 The voxel generatordivides the first point cloud data into a plurality of voxels based on the index, and also divides the second point cloud data into a plurality of voxels based on the index. The first point cloud data includes the object distance information. Specifically, the first point cloud data is point cloud data of density corresponding to the object distance, and is data obtained by adding the distance information to data of each point. In a case where the first point cloud data includes a plurality of pieces of point cloud data corresponding to a plurality of objects, and each point cloud data is stored in a voxel having a size in accordance with the distance.

The plurality of voxels in which the first point cloud data is divided do not have the same size but each have a size in accordance with the object distance information corresponding to the point cloud data inside the voxel. For example, the voxel size increases in a case where the point cloud data included in a voxel relates to a distant object. On the other hand, the voxel size decreases in a case where the point cloud data included in a voxel relates to a near object.

4 4 The difference adjustergenerates a voxel size correlated with distance as the index. The difference adjustercan adjust, by the index, the degree of change of the voxel size in accordance with distance change.

8 3 The voxel generatorgenerates a plurality of voxels in which the first point cloud data is divided based on the index, and a plurality of voxels in which the second point cloud data is divided based on the index. The difference extractorextracts, as the third point cloud data, point cloud data included in voxels of the difference between the plurality of voxels in which the first point cloud data is divided based on the index and the plurality of voxels in which the second point cloud data is divided based on the index.

The difference between the plurality of voxels in which the first point cloud data is divided based on the index and the plurality of voxels in which the second point cloud data is divided based on the index includes voxels different from the plurality of voxels in which the second point cloud data is divided. Each voxel of the difference includes point cloud data representing an object. Each voxel of the difference has a size corresponding to the distance of an object in the voxel.

12 FIG. 12 FIG. 12 FIG. 4 is a diagram illustrating the index generated by the difference adjusterin the fifth embodiment. In, the horizontal axis represents the object distance, and the vertical axis represents the voxel size. As illustrated in, the index includes information indicating the degree of change of the voxel size in accordance with change of the object distance.

3 4 The voxel size of each voxel extracted by the difference extractoris variably controlled by adjusting the index by the difference adjuster.

In this manner, in the fifth embodiment, the difference between a plurality of voxels in which the first point cloud data is divided based on the index and a plurality of voxels in which the second point cloud data is divided based on the index is calculated, and the third point cloud data in voxels of the difference is extracted. In the fifth embodiment, the degree of change of the voxel size in accordance with change of the object distance is adjusted by the index, and the third point cloud data is extracted by using the voxel size thus adjusted, and thus the extraction accuracy of the third point cloud data can be improved.

13 FIG. 13 FIG. 11 FIG. 1 1 7 7 is a block diagram illustrating a schematic configuration of the information processing deviceaccording to a sixth embodiment. As illustrated in, the information processing deviceaccording to the sixth embodiment includes the luminance detectorin addition to the configuration in. The luminance detectordetects luminance information of each pixel based on the two-dimensional image data that is input together with the original point cloud data.

4 3 8 8 3 Based on the luminance information, the difference adjustergenerates a voxel size correlated with distance as the index. The difference extractorincludes the voxel generator. The voxel generatorgenerates a plurality of voxels in which the first point cloud data is divided based on the index, and a plurality of voxels in which the second point cloud data is divided based on the index. The difference extractorextracts the third point cloud data in voxels of the difference between the plurality of voxels in which the first point cloud data is divided based on the index and the plurality of voxels in which the second point cloud data is divided based on the index.

14 FIG. 14 FIG. 14 FIG. 4 is a diagram illustrating the index generated by the difference adjusterin the sixth embodiment. In, the horizontal axis represents the object luminance, and the vertical axis represents the voxel size. The index includes information indicating the degree of change of the voxel size in accordance with change of the object luminance. As illustrated in, the voxel size as the index decreases as the object luminance increases.

3 The difference extractorextracts, as the third point cloud data, point cloud data included in voxels of the difference between the plurality of voxels in which the first point cloud data is divided based on the index and the plurality of voxels in which the second point cloud data is divided based on the index. Accordingly, the third point cloud data with less noise can be extracted.

1 4 As a modification of the information processing deviceaccording to the sixth embodiment, the difference adjustermay generate the voxel size as the index based on the distance information and the luminance information.

15 FIG. 15 FIG. 15 FIG. 15 FIG. 4 3 4 3 4 3 4 is a diagram illustrating the index generated by the difference adjusterin the modification of the sixth embodiment. In, the horizontal axis represents the object luminance, and the vertical axis represents the voxel size.illustrates an index win a case of high luminance and an index win a case of low luminance. The indexes wand whave mutually different gradients. The degree of change of the voxel size is smaller as the object luminance is higher. The indexes wand winare merely exemplary.

In this manner, in the sixth embodiment, the voxel size as the index is adjusted in accordance with the object luminance, and the third point cloud data in voxels of the difference between a plurality of voxels in which the first point cloud data is divided based on the index and a plurality of voxels in which the second point cloud data is divided based on the index is extracted. Accordingly, the third point cloud data with less noise can be extracted irrespective of the luminance.

16 FIG. 16 FIG. 1 FIG. 16 FIG. 1 FIG. 1 1 4 4 is a block diagram illustrating a schematic configuration of the information processing deviceaccording to a seventh embodiment. The information processing deviceaccording to the seventh embodiment illustrated inhas the same block configuration as in, but the difference adjusterinperforms processing operation different from that of the difference adjusterin.

4 4 4 1 FIG. 16 FIG. Although the difference adjusterinadjusts the index based on, for example, the first point cloud data or the two-dimensional image data, the difference adjusterinadjusts the index based on an adjustment signal α and a function f(distance or luminance). The function f(distance or luminance) is a function that calculates the index by using at least one of distance or luminance as an input parameter. The shape of the function f is freely selected. The index is, for example, a threshold or a voxel size. The difference adjusteradjusts the index based on, for example, Expression (1) below. The adjustment signal α is a real number other than zero.

17 FIG. 17 FIG. is a graph representing Expression (1). In, the horizontal axis represents the object distance, and the vertical axis represents the threshold or the voxel size. The shape of the function f, for example, the gradient of the function f can be adjusted by the adjustment signal α.

1 1 The adjustment signal α may be input from outside the information processing device, or may be a signal generated inside the information processing device.

18 FIG. 18 FIG. 18 FIG. 1 FIG. 1 1 1 9 10 1 is a block diagram illustrating a schematic configuration of the information processing deviceinside which the adjustment signal α is generated. The information processing deviceinincludes a processing block related to generation of the adjustment signal α. Specifically, the information processing deviceinincludes an evaluatorand a test image storagein addition to the configuration of the information processing devicein.

6 3 6 3 The object recognizerrecognizes an object based on the third point cloud data extracted by the difference extractor. For example, the object recognizerincludes an object recognition model, and outputs the result of object recognition from the object recognition model by inputting the third point cloud data extracted by the difference extractorinto the object recognition model.

3 Training images for training the object recognition model, and test images for evaluating the trained object recognition model are input to the difference extractor.

9 6 6 6 The evaluatorevaluates the object recognition accuracy of the object recognizerbased on the object recognition result output from the object recognizerwhen the test images are input to the object recognizer.

4 9 3 9 4 9 The difference adjusteradjusts the adjustment signal α based on the evaluation result of the evaluator, and updates the index based on the adjusted adjustment signal α. The difference extractorrepeats processing of the evaluatorand the difference adjustera predetermined number of times, and then extracts the third point cloud data from the difference point cloud data based on the index of the highest accuracy evaluated by the evaluator.

19 FIG. 18 FIG. 1 3 1 is a flowchart illustrating processing operation of the information processing devicein. First, the difference extractorsets the index by using an initial value α0 of the adjustment signal (step S). The index is, for example, a threshold or a box size.

6 1 2 2 Subsequently, the object recognizertrains an object recognition model by using the index set at step Sand training images (step S). The object recognition model is a model that outputs a recognition result of an object included in an input image by using the input image and the index as input parameters. Since object information included in the training images is known in advance, the object recognition model is trained at step Sby comparing the object recognition result output from the object recognition model with the known object information.

9 3 Subsequently, the evaluatorevaluates the accuracy of the object recognition model by inputting test images not used in the training and the index to the object recognition model (step S).

2 5 4 4 5 2 Subsequently, it is determined whether the processing at steps Sto Sis repeated a predetermined number of times (step S). In a case where the determination at step Sis “NO”, the adjustment signal α is adjusted and the index is updated (step S). Thereafter, the processing at step Sand later is performed by using the updated index.

4 6 In a case where the determination at step Sis “YES”, the index corresponding to the adjustment signal α with highest accuracy is finally selected (step S).

6 In this manner, in the seventh embodiment, the index can be freely adjusted by the adjustment signal α, and thus the index with which the object recognition accuracy of the object recognizeris optimal can be easily searched.

2 3 4 6 7 1 At least a part or all of the functions of each unit (e.g., the distance detector, the difference extractor, the difference adjuster, the object recognizer, and the luminance detector) of the information processing devicein each of the above-described embodiments may be configured using circuitry. For example, each of these units may be realized by a dedicated hardware circuit (e.g., an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA)) designed to perform a specific function. Alternatively, each of these units may be realized by processing circuitry in which a processor such as a Central Processing Unit (CPU) or a Graphics Processing Unit (GPU) executes a program (software) stored in a memory.

The embodiments as described before may be configured as below.

a distance detector configured to generate first point cloud data including distance information of an object included in original point cloud data; a difference extractor configured to extract third point cloud data from a difference between the first point cloud data and previously acquired second point cloud data; and a difference adjuster configured to generate an index to adjust an extraction amount of the difference, wherein the difference extractor extracts the third point cloud data from the difference based on the index. An information processing device comprising:

1 the first point cloud data constitutes a three-dimensional distance image including the background image. The information processing device according to claim, wherein the second point cloud data constitutes a three-dimensional background image of one frame, and

1 2 the difference extractor extracts the third point cloud data based on the difference exceeding the threshold. The information processing device according to claimor, wherein the difference adjuster generates, as the index, a threshold to adjust the extraction amount of the difference based on the distance information included in the first point cloud data, and

3 The information processing device according to claim, wherein the difference adjuster variably controls the threshold in accordance with a distance to the object.

4 The information processing device according to claim, wherein the difference adjuster increases the threshold as the distance to the object increases.

1 2 wherein the difference adjuster generates, as the index, a threshold to adjust the extraction amount of the difference based on the luminance information, and the difference extractor extracts the third point cloud data based on the difference exceeding the threshold. The information processing device according to claimor, further comprising: a luminance detector configured to detect luminance information for each pixel based on two-dimensional image data input together with the original point cloud data,

6 The information processing device according to claim, wherein the difference adjuster variably controls the threshold in accordance with luminance of each pixel.

6 7 The information processing device according to claimor, wherein the difference adjuster decreases the threshold as the luminance increases.

1 2 wherein the difference adjuster generates, as the index, a threshold to adjust the extraction amount of the difference based on the distance information and the luminance information, and the difference extractor extracts the third point cloud data based on the difference exceeding the threshold. The information processing device according to claimor, further comprising: a luminance detector configured to detect luminance information for each pixel based on two-dimensional image data input together with the original point cloud data,

9 The information processing device according to claim, wherein the difference extractor variably controls the threshold in accordance with a distance to the object and luminance.

10 The information processing device according to claim, wherein the difference adjuster variably controls a correspondence relation between the distance to the object and the threshold in accordance with luminance of the object.

1 2 (i) a plurality of voxels in which the first point cloud data is divided based on the index and (ii) a plurality of voxels in which the second point cloud data is divided based on the index, andthe difference adjuster generates a size of the voxel to adjust the extraction amount of the difference as the index. The information processing device according to claimor, wherein the difference extractor extracts the third point cloud data in a voxel of a difference between

12 The information processing device according to claim, wherein the difference adjuster variably controls the size of the voxel in accordance with a distance to the object.

13 The information processing device according to claim, wherein the difference adjuster increases the size of the voxel as the distance to the object increases.

1 2 (i) the plurality of voxels in which the first point cloud data is divided based on the index and (ii) the plurality of voxels in which the second point cloud data is divided based on the index. wherein the difference extractor extracts, based on the luminance information, the third point cloud data in a voxel of a difference between The information processing device according to claimor, further comprising: a luminance detector configured to detect luminance information of each of a plurality of voxels in which the first point cloud data and the second point cloud data are divided based on the index, based on two-dimensional image data input together with the original point cloud data,

15 The information processing device according to claim, wherein the difference adjuster variably controls the index representing a degree of change of a size of the voxel with respect to a change in a distance to the object, based on the luminance information for each voxel.

1 16 The information processing device according to any one of claimsto, further comprising an object recognizer configured to recognize an object based on the third point cloud data.

1 17 the difference extractor extracts the third point cloud data from the difference based on the index adjusted by the difference adjuster. The information processing device according to any one of claimsto, wherein the difference adjuster adjusts the index based on an adjustment signal, and

18 wherein the difference adjuster adjusts the index based on the adjustment signal based on an evaluation result of the evaluator, and the difference extractor repeats processing of the evaluator and the difference adjuster alternately a predetermined number of times, and then extracts the third point cloud data from the difference based on the index with highest accuracy evaluated by the evaluator. The information processing device according to claim, further comprising: an evaluator configured to evaluate accuracy of the third point cloud data extracted by the difference extractor using the first point cloud data of a test image,

extracting third point cloud data based on a difference between newly acquired first point cloud data and previously acquired second point cloud data; and generating an index to adjust an extraction amount of the difference based on the first point cloud data; and extracting the third point cloud data from the difference based on the index. An information processing method comprising:

While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the inventions. Indeed, the novel embodiments described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the embodiments described herein may be made without departing from the spirit of the inventions. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the inventions.