Hierarchical Feature Tracking Using Optical Flow

1. An image processing apparatus, comprising: an image obtaining device that obtains a first image and a second image; a hierarchical tier image creating device that creates a first hierarchical tier image group including a plurality of first hierarchical tier images with different resolutions by subjecting the first image to stepwise reduction processing, and creates a second hierarchical tier image group including a plurality of second hierarchical tier images with different resolutions by subjecting the second image to stepwise reduction processing; a tracking point detecting device that detects a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images; and a sequential detection device that performs the tracking point detection by the tracking point detecting device for hierarchical tier images included in each of the first and second hierarchical tier image groups in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the second image, the tracking point corresponding to a feature point in the first image, wherein the tracking point detecting device includes: a displacement calculating device that repeatedly detects a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images, and calculates a displacement amount representing a distance between a latest tracking point detected as a result of a detection repetition and a tracking point obtained as a result of a detection repetition before the detection repetition in the second image; a detection result outputting device that stops the repetitive tracking point detection by the displacement calculating device when the displacement amount converges on a value of less than a first threshold value or the repetition count of the tracking point position detection reaches not less than a second threshold value, and outputs a value of each of the displacement amount and the repetition count for the point of time of stopping the repetitive detection; and a criterion setting device that sets the first threshold value for the case where the tracking point detection by the tracking point detecting device is performed for the first and second hierarchical tier images in the first and second hierarchical tier image groups, respectively, the first and second hierarchical tier images having a second resolution that is higher than the first resolution, according to the value of each of the displacement amount and the repetition count output from the detection result outputting device.

2. The image processing apparatus according to claim 1 , wherein when the value of the displacement amount output from the detection result outputting device is smaller than a third threshold value, the criterion setting device sets the first threshold value to be smaller than the first threshold value for the case where the displacement amount is not less than the third threshold value.

3. The image processing apparatus according to claim 1 , wherein when the value of the repetition count output from the detection result outputting device is smaller than a fourth threshold value the criterion setting device sets the first threshold value to be smaller than the first threshold value for the case where the repetition count is not less than the fourth threshold value.

4. The image processing apparatus according to claim 1 , further comprising: a hierarchical tier selecting device that selects a hierarchical tier that is a next target for performing the tracking point detection by the tracking point detecting device according to the value of each of the displacement amount and the repetition count output from the detection result outputting device.

5. An image processing apparatus, comprising: an image obtaining device that obtains a first image and a second image; a hierarchical tier image creating device that creates a first hierarchical tier image group including a plurality of first hierarchical tier images with different resolutions by subjecting the first image to stepwise reduction processing, and creates a second hierarchical tier image group including a plurality of second hierarchical tier images with different resolutions by subjecting the second image to stepwise reduction processing; a tracking point detecting device that detects a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images; and a sequential detection device that performs the tracking point detection by the tracking point detecting device for hierarchical tier images included in each of the first and second hierarchical tier image groups in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the second image, the tracking point corresponding to a feature point in the first image, wherein the tracking point detecting device includes: a displacement calculating device that repeatedly detects a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images, and calculates a displacement amount representing a distance between a latest tracking point detected as a result of a detection repetition and a tracking point obtained as a result of a detection repetition before the detection repetition in the second image; and a detection result outputting device that stops the repetitive tracking point detection by the displacement calculating device when the displacement amount converges on a value of less than a first threshold value or the repetition count of the tracking point position detection reaches not less than a second threshold value, and outputs a value of each of the displacement amount and the repetition count for the point of time of stopping the repetitive detection, and wherein the image processing apparatus further comprises a hierarchical tier selecting device that selects a hierarchical tier that is a next target for performing the tracking point detection by the tracking point detecting device according to the value of each of the displacement amount and the repetition count output from the detection result outputting device.

6. The image processing apparatus according to claim 5 , wherein when the value of the displacement amount output from the detection result outputting device is smaller than a fifth threshold value, the hierarchical tier selecting device sets the hierarchical tier that is a next target for performing the tracking point detection by the tracking point detecting device to a hierarchical tier that is two or more levels higher than the current target hierarchical tier.

7. The image processing apparatus according to claim 5 , wherein when the value of the repetition count output from the detection result outputting device is smaller than a sixth threshold value, the hierarchical tier selecting device sets the hierarchical tier that is a next target for performing the tracking point detection by the tracking point detecting device to a hierarchical tier that is two or more levels higher than the current target hierarchical tier.

8. An image processing apparatus, comprising: an image obtaining device that obtains a first image, a second image and a third image, the second image and the third image being taken temporally before and after the first image being taken; a first hierarchical tier image creating device that creates a first hierarchical tier image group including a plurality of first hierarchical tier images with different resolutions by subjecting the first image to stepwise reduction processing, and creates a second hierarchical tier image group including a plurality of second hierarchical tier images with different resolutions by subjecting the second image to stepwise reduction processing; a first tracking point detecting device that detects a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images; and a first sequential detection device that performs the tracking point detection by the first tracking point detecting device for hierarchical tier images included in each of the first and second hierarchical tier image groups in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the second image, the tracking point corresponding to a feature point in the first image; a movement amount calculating device that calculates an amount of movement between the first image and the second image based on the positions of the feature point and the tracking point in the first and second images, and estimates an amount of movement between the first image and the third image; a hierarchical tier count setting device that sets a hierarchical tier count for hierarchical tier images created from the third image, based on the amount of movement between the first image and the third image; a second hierarchical tier image creating device that creates a number of third hierarchical tier images with different resolutions by subjecting the third image to stepwise reduction processing, the number corresponding to the hierarchical tier count set by the hierarchical tier count setting device, to create a third hierarchical tier image group; a second tracking point detecting device that detects a position of a tracking point in a third hierarchical tier image with a predetermined resolution in the third hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and third hierarchical tier images; and a second sequential detection device that performs the tracking point detection by the second tracking point detecting device for hierarchical tier images included in each of the first and third hierarchical tier image groups in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the third image, the tracking point corresponding to a feature point in the first image.

9. The image processing apparatus according to claim 8 , wherein the hierarchical tier count setting device sets the hierarchical tier count so as to become larger as the amount of movement is larger.

10. The image processing apparatus according to claim 8 , wherein the movement amount calculating device creates a motion vector representing a displacement direction and displacement amount of the feature point based on each of the positions of the feature points and the tracking points for the first and second images, and calculates an average value or median value of the motion vectors as the amount of movement.

11. The image processing apparatus according to claim 8 , wherein the movement amount calculating device estimates that the amount of movement between the first image and the second image and the amount of movement between the first image and the third image are equal to each other.

12. An image processing apparatus, comprising: an image obtaining device that obtains a first image, a second image and a third image, the second image and the third image being taken temporally before and after the first image being taken; a hierarchical tier image creating device that creates a first hierarchical tier image group including a plurality of first hierarchical tier images with different resolutions by subjecting the first image to stepwise reduction processing, creates a second hierarchical tier image group including a plurality of second hierarchical tier images with different resolutions by subjecting the second image to stepwise reduction processing, and creates a third hierarchical tier image group including a plurality of third hierarchical tier images with different resolutions by subjecting the third image to stepwise reduction processing; a first tracking point detecting device that detects a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images; a first sequential detection device that performs the tracking point detection by the first tracking point detecting device for hierarchical tier images included in each of the first and second hierarchical tier image groups in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the second image, the tracking point corresponding to a feature point in the first image; a movement amount calculating device that calculates an amount of movement between the first image and the second image based on the positions of the feature point and the tracking point in the first and second images, and estimates an amount of movement between the first image and the third image; a starting hierarchical tier setting device that sets a starting hierarchical tier to start feature point detection for the first image and the third image, based on the amount of movement between the first image and the third image; a second tracking point detecting device that detects a position of a tracking point in a third hierarchical tier image with a predetermined resolution in the third hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and third hierarchical tier images; and a second sequential detection device that performs the tracking point detection by the second tracking point detecting device starting from a hierarchical tier image in the starting hierarchical tier in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the third image, the tracking point corresponding to a feature point in the first image.

13. The image processing apparatus according to claim 12 , wherein the starting hierarchical tier setting device sets a hierarchical tier including a hierarchical tier image with a lower resolution to the starting hierarchical tier as the amount of movement is larger.

14. The image processing apparatus according to claim 12 , wherein the movement amount calculating device estimates that the amount of movement between the first image and the second image and the amount of movement between the first image and the third image are equal to each other.

15. An image processing method, comprising the steps of: an image obtaining step of obtaining a first image and a second image; a hierarchical tier image creating step of creating a first hierarchical tier image group including a plurality of first hierarchical tier images with different resolutions by subjecting the first image to stepwise reduction processing, and creating a second hierarchical tier image group including a plurality of second hierarchical tier images with different resolutions by subjecting the second image to stepwise reduction processing; a tracking point detecting step of detecting a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images; and a sequential detection step of performing the tracking point detection in the tracking point detecting step for hierarchical tier images included in each of the first and second hierarchical tier image groups in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the second image, the tracking point corresponding to a feature point in the first image, wherein the tracking point detecting step includes: a displacement calculating step of repeatedly detecting a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images, and calculating a displacement amount representing a distance between a latest tracking point detected as a result of a detection repetition and a tracking point obtained as a result of a detection repetition before the detection repetition in the second image; a detection result outputting step of stopping the repetitive tracking point detection in the displacement calculating step when the displacement amount converges on a value of less than a first threshold value or the repetition count of the tracking point position detection reaches not less than a second threshold value, and outputting a value of each of the displacement amount and the repetition count for the point of time of stopping the repetitive detection; and a criterion setting step of setting the first threshold value for the case where the tracking point detection in the tracking point detecting step is performed for first and second hierarchical tier images in the first and second hierarchical tier image groups, respectively, the first and second hierarchical tier images having a second resolution that is higher than the first resolution, according to the value of each of the displacement amount and the repetition count output in the detection result outputting step.

16. The image processing method according to claim 15 , wherein the criterion setting step includes, when the value of the displacement amount output in the detection result outputting step is smaller than a third threshold value, setting the first threshold value to be smaller than the first threshold value for the case where the displacement amount is not less than the third threshold value.

17. The image processing method according to claim 15 , wherein the criterion setting step includes, when the value of the repetition count output in the detection result outputting step is smaller than a fourth threshold value, setting the first threshold value to be smaller than the first threshold value for the case where the repetition count is not less than the fourth threshold value.

18. The image processing method according to claim 15 , further comprising: a hierarchical tier selecting step of selecting a hierarchical tier that is a next target for performing the tracking point detection in the tracking point detecting step according to the value of each of the displacement amount and the repetition count output in the detection result outputting step.

19. An image processing method, comprising the steps of: an image obtaining step of obtaining a first image and a second image; a hierarchical tier image creating step of creating a first hierarchical tier image group including a plurality of first hierarchical tier images with different resolutions by subjecting the first image to stepwise reduction processing, and creating a second hierarchical tier image group including a plurality of second hierarchical tier images with different resolutions by subjecting the second image to stepwise reduction processing; a tracking point detecting step of detecting a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images; and a sequential detection step of performing the tracking point detection in the tracking point detecting step for hierarchical tier images included in each of the first and second hierarchical tier image groups in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the second image, the tracking point corresponding to a feature point in the first image, wherein the tracking point detecting step includes: a displacement calculating step of repeatedly detecting a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images, and calculating a displacement amount representing a distance between a latest tracking point detected as a result of a detection repetition and a tracking point obtained as a result of a detection repetition before the detection repetition in the second image; and a detection result outputting step of stopping the repetitive tracking point detection in the displacement calculating step when the displacement amount converges on a value of less than a first threshold value or the repetition count of the tracking point position detection reaches not less than a second threshold value, and outputting a value of each of the displacement amount and the repetition count for the point of time of stopping the repetitive detection, and wherein the image processing method further comprises a hierarchical tier selecting step of selecting a hierarchical tier that is a next target for performing the tracking point detection in the tracking point detecting step according to the value of each of the displacement amount and the repetition count output in the detection result outputting step.

20. The image processing method according to claim 19 , wherein the hierarchical tier selecting step includes, when the value of the displacement amount output in the detection result outputting step is smaller than a fifth threshold value, setting the hierarchical tier that is a next target for performing the tracking point detection in the tracking point detecting step to a hierarchical tier that is two or more levels higher than the current target hierarchical tier.

21. The image processing method according to claim 19 , wherein the hierarchical tier selecting step includes, when the value of the repetition count output in the detection result outputting step is smaller than a sixth threshold value, setting the hierarchical tier that is a next target for performing the tracking point detection in the tracking point detecting step to a hierarchical tier that is two or more levels higher than the current target hierarchical tier.

22. An image processing method, comprising the steps of: an image obtaining step of obtaining a first image, a second image and a third image, the second image and the third image being taken temporally before and after the first image being taken; a first hierarchical tier image creating step of creating a first hierarchical tier image group including a plurality of first hierarchical tier images with different resolutions by subjecting the first image to stepwise reduction processing, and creating a second hierarchical tier image group including a plurality of second hierarchical tier images with different resolutions by subjecting the second image to stepwise reduction processing; a first tracking point detecting step of detecting a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images; a first sequential detection step of performing the tracking point detection in the first tracking point detecting step for hierarchical tier images included in each of the first and second hierarchical tier image groups in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the second image, the tracking point corresponding to a feature point in the first image; a movement amount calculating step of calculating an amount of movement between the first image and the second image based on the positions of the feature point and the tracking point in the first and second images, and estimating an amount of movement between the first image and the third image; a hierarchical tier count setting step of setting a hierarchical tier count for hierarchical tier images created from the third image, based on the amount of movement between the first image and the third image; a second hierarchical tier image creating step of creating a number of third hierarchical tier images with different resolutions by subjecting the third image to stepwise reduction processing, the number corresponding to the hierarchical tier count set by the hierarchical tier count setting step, to create a third hierarchical tier image group; a second tracking point detecting step of detecting a position of a tracking point in a third hierarchical tier image with a predetermined resolution in the third hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and third hierarchical tier images; and a second sequential detection step of performing the tracking point detection in the second tracking point detecting step for hierarchical tier images included in each of the first and third hierarchical tier image groups in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the third image, the tracking point corresponding to a feature point in the first image.

23. The image processing method according to claim 22 , wherein the hierarchical tier count setting step includes setting the hierarchical tier count so as to become larger as the amount of movement is larger.

24. An image processing method, comprising the steps of: an image obtaining step of obtaining a first image, a second image and a third image, the second image and the third image being taken temporally before and after the first image being taken; a hierarchical tier image creating step of creating a first hierarchical tier image group including a plurality of first hierarchical tier images with different resolutions by subjecting the first image to stepwise reduction processing, creating a second hierarchical tier image group including a plurality of second hierarchical tier images with different resolutions by subjecting the second image to stepwise reduction processing, and creating a third hierarchical tier image group including a plurality of third hierarchical tier images with different resolutions by subjecting the third image to stepwise reduction processing; a first tracking point detecting step of detecting a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images; a first sequential detection step of performing the tracking point detection in the first tracking point detecting step for hierarchical tier images included in each of the first and second hierarchical tier image groups in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the second image, the tracking point corresponding to a feature point in the first image; a movement amount calculating step of calculating an amount of movement between the first image and the second image based on the positions of the feature point and the tracking point in the first and second images, and estimating an amount of movement between the first image and the third image; a starting hierarchical tier setting step of setting a starting hierarchical tier to start feature point detection for the first image and the third image, based on the amount of movement between the first image and the third image; a second tracking point detecting step of detecting a position of a tracking point in a third hierarchical tier image with a predetermined resolution in the third hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and third hierarchical tier images; and a second sequential detection step of performing the tracking point detection in the second tracking point detecting step starting from a hierarchical tier image in the starting hierarchical tier in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the third image, the tracking point corresponding to a feature point in the first image.

25. The image processing method according to claim 24 , wherein the starting hierarchical tier setting step including setting a hierarchical tier including a hierarchical tier image with a lower resolution to the starting hierarchical tier as the amount of movement is larger.

26. A non-transitory computer-readable medium tangibly embodying a program of computer-readable instructions executable by a digital processing apparatus to perform an image processing method, said method comprising: an image obtaining step that obtains a first image and a second image; a hierarchical tier image creating step that creates a first hierarchical tier image group including a plurality of first hierarchical tier images with different resolutions by subjecting the first image to stepwise reduction processing, and creates a second hierarchical tier image group including a plurality of second hierarchical tier images with different resolutions by subjecting the second image to stepwise reduction processing; a tracking point detecting step that detects a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images; and a sequential detection step that performs the tracking point detection by the tracking point detecting step for hierarchical tier images included in each of the first and second hierarchical tier image groups in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the second image, the tracking point corresponding to a feature point in the first image, wherein the tracking point detecting step includes: a displacement calculating step that repeatedly detects a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images, and calculates a displacement amount representing a distance between a latest tracking point detected as a result of a detection repetition and a tracking point obtained as a result of a detection repetition before the detection repetition in the second image; a detection result outputting step that stops the repetitive tracking point detection by the displacement calculating function when the displacement amount converges on a value of less than a first threshold value or the repetition count of the tracking point position detection reaches not less than a second threshold value, and outputs a value of each of the displacement amount and the repetition count for the point of time of stopping the repetitive detection; and a criterion setting step that sets the first threshold value for the case where the tracking point detection by the tracking point detecting step is performed for first and second hierarchical tier images in the first and second hierarchical tier image groups, respectively, the first and second hierarchical tier images having a second resolution that is higher than the first resolution, according to the value of each of the displacement amount and the repetition count by the detection result outputting function.

27. The non-transitory computer-readable medium according to claim 26 , wherein when the value of the displacement amount output by the detection result outputting function is smaller than a third threshold value, the criterion setting function sets the first threshold value to be smaller than the first threshold value for the case where the displacement amount is not less than the third threshold value.

28. The non-transitory computer-readable medium according to claim 26 , wherein when the value of the repetition count output by the detection result outputting function is smaller than a fourth threshold value the criterion setting function sets the first threshold value to be smaller than the first threshold value for the case where the repetition count is not less than the fourth threshold value.

29. The non-transitory computer-readable medium according to claim 26 , further comprising: a hierarchical tier selecting step that selects a hierarchical tier that is a next target for performing the tracking point detection by the tracking point detecting step according to the value of each of the displacement amount and the repetition count output by the detection result outputting step.

30. A non-transitory computer-readable medium tangibly embodying a program of computer-readable instructions executable by a digital processing apparatus to perform an image processing method, said method comprising: an image obtaining step that obtains a first image and a second image; a hierarchical tier image creating step that creates a first hierarchical tier image group including a plurality of first hierarchical tier images with different resolutions by subjecting the first image to stepwise reduction processing, and creates a second hierarchical tier image group including a plurality of second hierarchical tier images with different resolutions by subjecting the second image to stepwise reduction processing; a tracking point detecting step that detects a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images; and a sequential detection step that performs the tracking point detection by the tracking point detecting step for hierarchical tier images included in each of the first and second hierarchical tier image groups in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the second image, the tracking point corresponding to a feature point in the first image, wherein the tracking point detecting step includes: a displacement calculating step that repeatedly detects a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images, and calculates a displacement amount representing a distance between a latest tracking point detected as a result of a detection repetition and a tracking point obtained as a result of a detection repetition before the detection repetition in the second image; and a detection result outputting step that stops the repetitive tracking point detection by the displacement calculating function when the displacement amount converges on a value of less than a first threshold value or the repetition count of the tracking point position detection reaches not less than a second threshold value, and outputs a value of each of the displacement amount and the repetition count for the point of time of stopping the repetitive detection, and wherein the image processing method further comprises: a hierarchical tier selecting step that selects a hierarchical tier that is a next target for performing the tracking point detection by the tracking point detecting step according to the value of each of the displacement amount and the repetition count output by the detection result outputting step.

31. The non-transitory computer-readable medium according to claim 30 , wherein when the value of the displacement amount output by the detection result outputting function is smaller than a fifth threshold value, the hierarchical tier selecting function sets the hierarchical tier that is a next target for performing the tracking point detection by the tracking point detecting function to a hierarchical tier that is two or more levels higher than the current target hierarchical tier.

32. The non-transitory computer-readable medium according to claim 30 , wherein when the value of the repetition count output by the detection result outputting function is smaller than a sixth threshold value, the hierarchical tier selecting function sets the hierarchical tier that is a next target for performing the tracking point detection by the tracking point detecting function to a hierarchical tier that is two or more levels higher than the current target hierarchical tier.

33. A non-transitory computer-readable medium tangibly embodying a program of computer-readable instructions executable by a digital processing apparatus to perform an image processing method, said method comprising: an image obtaining step that obtains a first image, a second image and a third image, the second image and the third image being taken temporally before and after the first image being taken; a first hierarchical tier image creating step that creates a first hierarchical tier image group including a plurality of first hierarchical tier images with different resolutions by subjecting the first image to stepwise reduction processing, and creates a second hierarchical tier image group including a plurality of second hierarchical tier images with different resolutions by subjecting the second image to stepwise reduction processing; a first tracking point detecting step that detects a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images; a first sequential detection step that performs the tracking point detection by the first tracking point detecting function for hierarchical tier images included in each of the first and second hierarchical tier image groups in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the second image, the tracking point corresponding to a feature point in the first image; a movement amount calculating step that calculates an amount of movement between the first image and the second image based on the positions of the feature point and the tracking point in the first and second images, and estimates an amount of movement between the first image and the third image; a hierarchical tier count setting step that sets a hierarchical tier count for hierarchical tier images created from the third image, based on the amount of movement between the first image and the third image; a second hierarchical tier image creating step that creates a number of third hierarchical tier images with different resolutions by subjecting the third image to stepwise reduction processing, the number corresponding to the hierarchical tier count set by the hierarchical tier count setting function, to create a third hierarchical tier image group; a second tracking point detecting step that detects a position of a tracking point in a third hierarchical tier image with a predetermined resolution in the third hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and third hierarchical tier images; and a second sequential detection step that performs the tracking point detection by the second tracking point detecting function for hierarchical tier images included in each of the first and third hierarchical tier image groups in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the third image, the tracking point corresponding to a feature point in the first image.

34. The non-transitory computer-readable medium according to claim 33 , wherein the hierarchical tier count setting function sets the hierarchical tier count so as to become larger as the amount of movement is larger.

35. A non-transitory computer-readable medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform an image processing method, said method comprising: an image obtaining step that obtains a first image, a second image and a third image, the second image and the third image being taken temporally before and after the first image being taken; a hierarchical tier image creating step that creates a first hierarchical tier image group including a plurality of first hierarchical tier images with different resolutions by subjecting the first image to stepwise reduction processing, creates a second hierarchical tier image group including a plurality of second hierarchical tier images with different resolutions by subjecting the second image to stepwise reduction processing, and creates a third hierarchical tier image group including a plurality of third hierarchical tier images with different resolutions by subjecting the third image to stepwise reduction processing; a first tracking point detecting step that detects a position of a tracking point in a second hierarchical tier image with a predetermined resolution in the second hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and second hierarchical tier images; a first sequential detection step that performs the tracking point detection by the first tracking point detecting function for hierarchical tier images included in each of the first and second hierarchical tier image groups in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the second image, the tracking point corresponding to a feature point in the first image; a movement amount calculating step that calculates an amount of movement between the first image and the second image based on the positions of the feature point and the tracking point in the first and second images, and estimates an amount of movement between the first image and the third image; a starting hierarchical tier setting step that sets a starting hierarchical tier to start feature point detection for the first image and the third image, based on the amount of movement between the first image and the third image; a second tracking point detecting step that detects a position of a tracking point in a third hierarchical tier image with a predetermined resolution in the third hierarchical tier image group, the tracking point corresponding to a predetermined feature point in a first hierarchical tier image with the predetermined resolution in the first hierarchical tier image group, based on a gradient of an image signal in each of the first and third hierarchical tier images; and a second sequential detection step that performs the tracking point detection by the second tracking point detecting function starting from a hierarchical tier image in the starting hierarchical tier in increasing order of resolution while reflecting the result of tracking point detection in a hierarchical tier image with a lower resolution, to detect a position of a tracking point in the third image, the tracking point corresponding to a feature point in the first image.

36. The non-transitory computer-readable medium according to claim 35 , wherein the starting hierarchical tier setting step sets a hierarchical tier including a hierarchical tier image with a lower resolution to the starting hierarchical tier as the amount of movement is larger.