Image analysis method for sensing moving ball, and sensing device using same

This application is the 35 U.S.C. 371 national stage of international application PCT/KR2022/009932 filed on Jul. 8, 2022 which claims priority to Korean Patent Application No. 10-2021-0096145 filed on Jul. 22, 2021. The entire contents of each of the above-identified applications are hereby incorporated by reference.

The present invention is related to a sensing device and a method for analyzing an image captured by a camera for sensing a ball moving as a player hits the ball in a field of sports with balls.

Recently, virtual simulation systems for sports that are very restrictive to play directly on the field, such as golf and baseball, have become widely popular.

Furthermore, virtual sports simulation systems for net sports, in which both players exchange balls with each other over nets such as tennis, squash, and badminton, are also emerging, making it easier for users to enjoy various sports in popular cultural spaces.

In such a virtual sports simulation, the game is basically played while the player hits the ball, and for simulation on the image of the ball hit by the player, a sensing device capable of effectively sensing the moving ball is required.

In the past, light sensing methods such as infrared sensors and laser sensors were often used, but since a structure equipped with a number of optical sensors to irradiate and receive light must be installed on the player's play space, the player's play was limited and there was a limit to realizing realistic play. In addition, there was a problem that the sensor's structure was frequently damaged during the play process, and above all, there was a problem that the accuracy of sensing was very low.

Due to the limitations and problems of this light sensing method, an image sensing device that acquires a camera image of a moving ball and calculates sensing information of a moving ball through analysis of the acquired image has been widely used.

This image sensing method has the advantage of obtaining fairly accurate sensing information because it analyzes images to obtain sensing information, but since each image of tens to hundreds of frames per second must be processed, there was a problem that it was necessary to have a camera device and a data processing device with quite high data processing capability.

In this regard, prior arts such as Korean patent publication No. 10-2018-0054279, Korean patent publication No. 10-2019-0085152, U.S. Pat. Nos. 7,497,780 and 7,324,663 are disclosed.

The above prior arts commonly disclose a method of analyzing an image based on a region of interest according to designating a peripheral area in which the ball exists in an image including a ball captured by a camera as the region of interest.

The prior art as described above predicts the position where the ball moves through the acquired image and designates a predetermined area around the predicted position as the region of interest. However, in order to predict the position where the ball moves, it is necessary to calculate the physical quantity of the previous ball movements and perform another complex operation to predict the position where the ball moves using the calculated physical quantity, so prior arts have a problem that they have to go through a complicated process to set the region of interest for the ball on the image.

Even if the region of interest is set through such a complex process, there is also a problem that the ball cannot be detected on the region of interest if the calculation for predicting the movement location is insufficient or the prediction is wrong.

It is an object of the present invention to provide an image analysis method for sensing moving ball, and sensing device using same significantly improving the speed of image processing by performing image analysis by applying an analysis object area set immediately before each acquired image to the current image according to a method that setting an analysis object area, which is a partial area, without analyzing the entire image in analyzing an image for sensing a moving ball in sports with balls, detecting the ball within the set analysis object area, calculating the physical quantity of the ball movement, and setting another analysis object area to include the position where the ball will appear on the image to be analyzed based on the position of the ball detected on the image being analyzed.

In accordance with an aspect of the present invention, the above and other objects can be accomplished by the provision of an image analysis method for sensing moving ball performed by a sensing device, comprising: acquiring an image in a field of view of a camera facing a space where a ball moves; detecting a position of the ball from the acquired image; setting an analysis object area to include a position where a ball will appear on a next image based on the detected position of the ball; and detecting the position of the ball within the set analysis object area on the next image of the image used to set the analysis object area.

Preferably, the method further comprises: setting a new analysis object area to include a position where a ball will appear on a next image based on the position of the ball detected within the analysis object area set on the previous image; and detecting the position of the ball within the new analysis object area on the next image.

Preferably, wherein the detecting the position of the ball within the set analysis object area on the next image includes: calculating a three-dimensional area of the range in which the ball can move using information preset as a limit physical quantity by which the ball can move at the detected position of the ball; and setting an area including a part projected from a three-dimensional area of the range in which the ball can move to a surface viewed by the camera as the analysis object area.

Preferably, wherein the acquiring the image by the camera includes acquiring images by each of the plurality of stereoscopic-type cameras interconnected with each other, and wherein detecting the position of the ball within the set analysis object area on the next image includes: calculating a three-dimensional area of the range in which the ball can move using information preset as a limit physical quantity by which the ball can move at the detected position of the ball; and setting each area including a part projected from each three-dimensional area of the range in which the ball can move to the surface viewed by each of the plurality of cameras, as an analysis object area for each camera image.

Preferably, wherein a three-dimensional area of the range in which the ball can move is calculated by reflecting information on a change in a relative positional relationship between the moving ball and the camera for each acquired image, and the analysis object area is calculated and set from the calculated three-dimensional area.

Preferably, wherein a change in a relative positional relationship between the moving ball and the camera is calculated in advance, a constant three-dimensional area having a size that includes both the current ball position and the ball position in the next image is preset at any position, and the analysis object area is calculated and set from the constant three-dimensional area based on the ball position for each acquired image.

Preferably, wherein the limit physical quantity that the ball can move is preset for each type of golf club that hits the ball, and wherein the calculating the three-dimensional area of the range in which the ball can move includes, identifying a type of the golf club in which the user hits the ball, and calculating the three-dimensional area of the range in which the ball can move by applying the limit physical quantity according to the identified golf club type.

In accordance with another aspect of the present invention, there is provided a sensing device that senses a moving ball through image analysis of the ball that is hit and moved, comprising: a camera device that acquires images in a field of view toward a space where the ball moves; and a sensing processor that detects a position of the ball from the acquired images, sets an analysis object area to include a position where a ball will appear on a next image based on the detected position of the ball, and by detecting the position of the ball within the set analysis object area on the next image of the image used to set the analysis object area, performs image analysis for sensing the ball only within the set analysis object area for each acquired image.

Preferably, wherein the sensing processor is configured to calculate a three-dimensional area of the range in which the ball can move by using information preset as a limit physical quantity by which the ball can move at the detected position of the ball, and set an area including a part projected from the three-dimensional area of the range in which the ball can move to a surface viewed by the camera device as the analysis object area.

Preferably, wherein the sensing processor is configured to newly calculate and set the analysis object area for each position of the ball by calculating the three-dimensional area of the range in which the ball can move by reflecting information on a change in a relative positional relationship between the moving ball and the camera for each acquired image.

Preferably, wherein the sensing processor is configured to calculate a change in a relative positional relationship between the moving ball and the camera in advance, set an area of a size that can be used at any position in the field of view of the camera device as the analysis object area, and collectively apply the preset analysis object area to the image to be analyzed.

The image analysis method for sensing moving ball, and sensing device using same according to the present invention have an advantageous effect that the speed of image processing can be significantly improved by performing image analysis by applying an analysis object area set immediately before each acquired image to the current image according to a method that setting an analysis object area, which is a partial area, without analyzing the entire image in analyzing an image for sensing a moving ball in sports with balls, detecting the ball within the set analysis object area, calculating the physical quantity of the ball movement, and setting another analysis object area to include the position where the ball will appear on the image to be analyzed based on the position of the ball detected on the image being analyzed.

An image analysis method for sensing moving ball, and sensing device using same according to the present invention will be more specifically described with reference to the drawings.

First, a configuration of a device for sensing a moving ball according to an embodiment of the present invention is described with reference to.is a block diagram showing configurations of a device for sensing a moving ball according to an embodiment of the present invention.

The sensing device and the image analysis method used same according to the present invention, whether it is a single camera method or a stereoscopic method linked with multiple cameras, can analyze images capturing the movement of the ball by camera(s) to calculate coordinate information of the ball position in a real space, and calculate various physical quantities related to the movement of the ball, such as the speed of the ball, the direction angle of the ball, and the spin of the ball, based on the calculated information. The present invention is applicable to sports using balls, such as golf, baseball, tennis, squash, and bowling.

In the present specification, among various sports using balls as described above, a sensing device used in a golf field is described as an example, however the technology applied to the sensing device used in the golf field may be equally applied to other sports.

is a block diagram showing the configuration of a sensing device that senses a ball moving as a user puts a ball GB on a golf mat GM and hits the ball GB with a golf club GC in a so-called screen golf system.

As shown in, the sensing device for sensing the ball according to an embodiment of the present invention may acquire images of a moving ball as the user hits the ball GB with a golf club GC, and analyze the acquired images to calculate the three-dimensional coordinates of the ball moving in space and information on motion characteristics such as the initial speed, direction angle, and height angle of the ball based on the three-dimensional coordinate information.

The sensing device may be applied to various fields such as analysis of a ball that is hit and moves or virtual golf using a virtual reality-based simulation according to a user's golf swing.

The sensing device according to an embodiment of the present invention may include a camera deviceand a sensing processoras shown in.

The camera devicemay be configured to consecutively acquire images in a field of view including a moving golf ball, and in order to calculate position information on the three-dimensional space for a moving golf ball, it is preferable that a plurality of cameras, for example, the first cameraand the second camera, which acquire images for the same object at different field of views, are synchronized with each other and configured in a stereoscopic manner, as shown in.

By configuring the camera device of the sensing device in a stereoscopic manner in which the plurality of camerasandare synchronized with each other as described above, the two-dimensional information of the corresponding golf ball extracted from the image acquired through the first cameraand the image acquired through the second camerafor the same object may be converted into three-dimensional information.

As shown in, the sensing processormay include an image receiverthat collects images from each of the plurality of camerasand, an image processorthat detects a ball from each image by performing necessary image processing on the images collected by the image receiver, and an operatorthat calculates three-dimensional position information from the two-dimensional position information of the ball detected from each image and is responsible for various operations for sensing.

The sensing processorof the sensing device according to an embodiment of the present invention detects a moving ball from each of the images collected through each of the camerasand, calculates the position information of the detected ball, and transmits the calculated position information to a client, allowing the clientto perform its own functions, such as calculating new information or calculating analysis information using the transmitted position information of the ball.

For example, if the clientis implemented as a simulator used in a screen golf system, the client may receive position information of a golf ball and a golf club from the sensing processorto implement a simulation image of the trajectory of a virtual golf ball flying on a virtual golf course in the simulation image.

If the clientis implemented as a golf swing analysis device, the client may receive position information of a golf ball and a golf club from the sensing processorto provide analysis information on the user's golf swing, diagnosis of swing problems, and lesson information to solve the problems.

The image processormay be configured to perform image processing to obtain a difference image between a reference image and each of the images consecutively acquired by the camerasand, and the operatormay be configured to calculate position information of a moving ball from the image processed by the image processor.

In addition to the method of using the difference image as described above as a method of detecting a moving ball from the acquired image, a template image for the ball may be prepared and the golf ball may be extracted through similarity with the template image for the ball on each acquired image.

Meanwhile, the camera deviceof the sensing device according to an embodiment of the present invention as described above acquires images in a field of view including a space in which the ball moves, and the sensing processormay detect a position of a ball from the acquired images and set an analysis object area to include a position at which a ball will appear on the next frame image based on the ball position detected by the sensing processor in the current frame image. Subsequently, when the sensing processor detect a position of a ball from the next frame image, the position of the ball in the next frame image may be detected within the previously set analysis object area on the next frame image. In this way, image analysis for sensing the ball may be performed only within the analysis object area set for each acquired image frame.

As described above, when the sensing processoranalyzes the images acquired by the camera device, the data processing speed may be significantly improved because the position information of the ball is calculated by analyzing only the analysis object area corresponding to a partial area of the entire image without analyzing the entire image.

Meanwhile, the image analysis method for sensing a moving ball according to an embodiment of the present invention will be described with reference to.

is a flowchart showing a sensing method of the device for sensing a moving ball according to an embodiment of the present invention, andis a flowchart showing more specifically an image analysis method according to an embodiment of the present invention in the flowchart shown in.

As shown in, as the sensing device operates, the camera device acquires images in a field of view toward the space where the ball moves (S).

The sensing processor may analyze a specific area on the acquired image (S) and detect the position of the ball in the specific area on the image (S).

For example, in the case of golf, since golf balls are always hit at a predetermined position such as a golf mat, if the location information of the golf mat is preset, the location of the golf mat can be specified on the image, and the initial ball position can be easily detected by examining an area including the specified location of the golf mat. For other sports, such as baseball, the initial position of the ball can be easily detected in the same way.

Meanwhile, as the sensing processor detects the position of the ball, the sensing device is in a sensing ready state for the ball (S), and by detecting the position where the ball is placed through the consecutively acquired images, whether the ball is hit and moved can be sensed.