Artificial Intelligence-Based Method and System for Controlling Plant Cultivation

The present disclosure relates to a method and system of controlling plant cultivation based on artificial intelligence.

Along with the development of digital technology or communication technology, the development of Information and Communications Technology (ICT) technology is remarkable.

In particular, much research is being conducted on artificial intelligence technology, and attempts are being made to apply the artificial intelligence technology to various fields.

Recently, interest in companion plants has been gradually increasing following companion animals.

However, in the past, small-sized plant cultivation machines were provided so that ordinary people could cultivate plants, but it is not easy for non-experts to cultivate companion plants without problems using only a plant cultivation machine, and thus plant cultivation often fails.

This reduces interest in companion plant cultivation and causes dissatisfaction.

The problem that the present disclosure seeks to solve is to provide an artificial intelligence-based plant cultivation control method and system.

Another problem that the present disclosure seeks to solve is to provide various plant cultivation control information, including information on growth prediction of target plants based on artificial intelligence, to provide convenience in using a plant cultivation machine.

Another problem that the present disclosure seeks to solve is to provide a service that provides users with not only an automatic control function of a plant cultivation machine but also prediction information on the growth process of a target plant, so as to induce enjoyment and interest in using a plant cultivation machine or plant cultivation, and to enable a virtual experience.

The problems that the present disclosure seeks to solve are not limited to the problems mentioned above, and other problems that are not mentioned can be clearly understood by those skilled in the art from the description below.

A method for plant cultivation based on artificial intelligence in a server according to at least one of the various embodiments of the present disclosure for solving the above-described problem may include generating a plant growth prediction model; detecting a first event; obtaining an image of a target plant in a plant cultivation device; generating cultivation information of a target plant for the detected first event and the obtained image based on the plant growth prediction model; and controlling an output of the generated cultivation information of the target plant.

A system for controlling plant cultivation based on artificial intelligence according to at least one of the various embodiments of the present disclosure may include a plant cultivation device; and a computing device communicating with the plant cultivation device and transmit and receive a signal, wherein the computing device comprises a processor that generates a plant growth prediction model, obtains, when an event is detected, an image of a target plant in the plant cultivation device, generates cultivation information of the target plant for the detected event and the obtained image based on the plant growth prediction model, and controls an output of the generated cultivation information of the target plant.

Other specific details of the present disclosure are included in the detailed description and drawings.

According to at least one of the various embodiments of the present disclosure, there is an effect of providing an artificial intelligence-based plant cultivation control method and system.

According to at least one of the various embodiments of the present disclosure, there is an effect of improving the user's convenience of using a plant cultivation machine through various plant cultivation control information including information on growth prediction of a target plant based on artificial intelligence.

According to at least one of the various embodiments of the present disclosure, a virtual experience service can be provided based on an automatic control function of a plant cultivation machine and prediction information on the growth process of a target plant, so that even non-experts can easily and conveniently use a plant cultivation machine, and there is an effect of inducing enjoyment and interest in plant cultivation.

Hereinafter, embodiments related to the present invention will be described in more detail with reference to the drawings. The suffixes “module” and “part” used for components in the following description are given or used interchangeably only for the convenience of writing a specification, and do not have distinct meanings or roles in themselves.

An artificial intelligence (AI) refers to a field that studies artificial intelligence or a methodology for creating it, and Machine Learning (Machine Learning) refers to a field that defines various problems in the field of artificial intelligence and studies a methodology for solving them. Machine Learning is also defined as an algorithm that improves the performance of a task through continuous experience.

An artificial neural network (ANN) is a model used in machine learning, and can refer to the entire model with problem-solving capabilities that consists of artificial neurons (nodes) that form a network by combining synapses. An artificial neural network can be defined by a connection pattern between neurons in different layers, a learning process that updates model parameters, and an activation function that generates output values.

An artificial neural network can include an input layer, an output layer, and optionally one or more hidden layers. Each layer includes one or more neurons, and an artificial neural network can include synapses that connect neurons to neurons. In an artificial neural network, each neuron can output the function value of the activation function for the input signals, weights, and biases input through the synapse.

Model parameters refer to parameters determined through learning, including the weights of synaptic connections and the biases of neurons. In addition, hyperparameters refer to parameters that must be set before learning in machine learning algorithms, including the learning rate, number of repetitions, mini-batch size, and initialization function.

The purpose of learning in an artificial neural network can be seen as determining model parameters that minimize the loss function. The loss function can be used as an indicator to determine the optimal model parameters during the learning process of an artificial neural network.

Machine learning can be classified into supervised learning, unsupervised learning, and reinforcement learning depending on the learning method.

Supervised learning refers to a method of training an artificial neural network when labels for training data are given, and the labels can refer to the correct answer (or result value) that the artificial neural network must infer when training data is input to the artificial neural network. Unsupervised learning can refer to a method of training an artificial neural network when labels for training data are not given. Reinforcement learning can refer to a learning method that trains an agent defined in a certain environment to select an action or action sequence that maximizes cumulative reward in each state.

Among artificial neural networks, machine learning implemented with a deep neural network (DNN) that includes multiple hidden layers is also called deep learning, and deep learning is a part of machine learning. Hereinafter, machine learning is used to mean including deep learning.

Object detection models using machine learning include the single-stage You Only Look Once (YOLO) model and the two-stage Faster Regions with Convolution Neural Networks (R-CNN) model.

You Only Look Once (YOLO) model is a model that can predict objects and the location of the objects in an image by looking at the image only once.

You Only Look Once (YOLO) model divides the original image into grids of the same size. Then, for each grid, the number of bounding boxes designated in a predefined shape centered on the center of the grid is predicted, and the reliability is calculated based on this.

After that, whether the image contains an object or only the background is included, and the location with high object reliability can be selected to identify the object category.

Faster Regions with Convolution Neural Networks (R-CNN) model is a model that can detect objects faster than the RCNN model and Fast RCNN model.

The Faster Regions with Convolution Neural Networks (R-CNN) model is explained in detail.

First, a feature map is extracted from the image through the Convolution Neural Network (CNN) model. Based on the extracted feature map, multiple regions of interest (RoI) are extracted. A RoI pooling is performed for each region of interest.

The RoI pooling is a process of setting a grid to match the pre-determined H×W size of the feature map on which the region of interest is projected, extracting the largest value for each cell included in each grid, and extracting a feature map having the size of H×W.

A feature vector is extracted from a feature map having the size of H×W, and object identification information can be obtained from the feature vector.

An extended Reality (XR) is a general term for a Virtual Reality (VR), an Augmented Reality (AR), and a Mixed Reality (MR). The VR technology provides objects or backgrounds in the real world only as CG images, the AR technology provides virtual CG images on top of real object images, and the MR technology is a computer graphics technology that mixes and combines virtual objects in the real world.

The MR technology is similar to the AR technology in that it shows real objects and virtual objects together. However, there is a difference in that while the AR technology uses virtual objects to complement real objects, the MR technology uses virtual objects and real objects with equal characteristics.

The XR technology can be applied to a Head-Mounted Display (HMD), a Head-Up Display (HUD), mobile phones, tablet PCS, laptops, desktops, TVs, a digital signage, etc., and devices to which the XR technology is applied can be called XR devices.

1 FIG. 100 illustrates an AI deviceaccording to an embodiment of the present disclosure.

100 The AI devicemay be implemented as a fixed device or a movable device, such as a TV, a projector, a mobile phone, a smart phone, a desktop computer, a laptop, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a tablet PC, a wearable device, a set-top box (STB), a DMB receiver, a radio, a washing machine, a refrigerator, a desktop computer, a digital signage, a robot, a vehicle, etc.

1 FIG. 100 110 120 130 140 150 170 180 Referring to, the terminalmay include a communication unit, an input unit, a learning processor, a sensing unit, an output unit, a memory, and a processor.

110 100 100 200 110 a e The communication unitcan transmit and receive data with external devices such as other AI devicestoor AI serversusing wired and wireless communication technology. For example, the communication unitcan transmit and receive sensor information, user input, learning models, control signals, etc. with external devices.

110 At this time, the communication technologies used by the communication unitinclude Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Long Term Evolution (LTE), 5G, Wireless LAN (WLAN), Wireless-Fidelity (Wi-Fi), Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), ZigBee, Near Field Communication (NFC), etc.

120 The input unitcan obtain various types of data.

120 At this time, the input unitmay include a camera for inputting a video signal, a microphone for receiving an audio signal, a user input unit for receiving information from a user, etc. Here, the camera or microphone may be treated as a sensor, and the signal obtained from the camera or microphone may be referred to as sensing data or sensor information.

120 120 180 130 The input unitmay obtain input data to be used when obtaining output using learning data for model learning and a learning model. The input unitmay obtain unprocessed input data, and in this case, the processoror the learning processormay extract input features as preprocessing for the input data.

130 The learning processormay use the learning data to learn a model composed of an artificial neural network. Here, the learned artificial neural network may be referred to as a learning model. The learning model can be used to infer a result value for new input data that is not learning data, and the inferred value can be used as a basis for judgment to perform a certain action.

130 240 200 At this time, the learning processorcan perform AI processing together with the learning processorof the AI server.

130 100 130 170 100 At this time, the learning processorcan include a memory integrated or implemented in the AI device. Alternatively, the learning processorcan be implemented using a memory, an external memory directly coupled to the AI device, or a memory maintained in an external device.

140 100 100 The sensing unitcan obtain at least one of internal information of the AI device, information about the surrounding environment of the AI device, and user information using various sensors.

140 At this time, the sensors included in the sensing unitinclude proximity sensors, light sensors, acceleration sensors, magnetic sensors, gyro sensors, inertial sensors, RGB sensors, IR sensors, fingerprint recognition sensors, ultrasonic sensors, light sensors, microphones, lidar, radar, etc.

150 The output unitcan generate output related to vision, hearing, or touch.

150 At this time, the output unitcan include a display unit that outputs visual information, a speaker that outputs auditory information, a haptic module that outputs tactile information, etc.

170 100 170 120 The memorycan store data that supports various functions of the AI device. For example, the memorycan store input data, learning data, learning models, learning history, etc. acquired from the input unit.

180 100 180 100 The processorcan determine at least one executable operation of the AI devicebased on information determined or generated using a data analysis algorithm or a machine learning algorithm. Then, the processorcan control the components of the AI deviceto perform the determined operation.

180 130 170 100 To this end, the processorcan request, search, receive, or utilize data from the learning processoror the memory, and control the components of the AI deviceto perform a predicted operation or an operation determined to be desirable among the at least one executable operation.

180 180 At this time, if the processorrequires linkage with an external device to perform the determined operation, the processorcan generate a control signal for controlling the external device and transmit the generated control signal to the external device.

180 The processorcan obtain intention information for a user input and determine the user's requirement based on the obtained intention information.

180 At this time, the processormay obtain intention information corresponding to the user input by using at least one of the Speech To Text (STT) engine for converting voice input into a string or the Natural Language Processing (NLP) engine for obtaining intention information of natural language.

130 240 200 At this time, at least one of the STT engine or the NLP engine may be configured with an artificial neural network that is at least partially learned according to a machine learning algorithm. In addition, at least one of the STT engine or the NLP engine may be learned by the learning processor, learned by the learning processorof the AI server, or learned by distributed processing of these.

180 100 170 130 200 The processormay collect history information including the operation content of the AI deviceor the user's feedback on the operation, and store it in the memoryor the learning processor, or transmit it to an external device such as the AI server. The collected history information can be used to update the learning model.

180 100 170 180 100 The processorcan control at least some of the components of the AI deviceto drive the application program stored in the memory. Furthermore, the processorcan operate two or more of the components included in the AI devicein combination with each other to drive the application program.

2 FIG. 200 illustrates an AI serveraccording to an embodiment of the present disclosure.

2 FIG. 200 200 200 100 Referring to, the AI servermay mean a device that trains an artificial neural network using a machine learning algorithm or uses a trained artificial neural network. Here, the AI servermay be composed of multiple servers to perform distributed processing, and may be defined as a 5G network. In this case, the AI servermay be included as a part of the AI deviceand may perform at least a part of the AI processing together.

200 210 230 240 260 The AI servermay include a communication unit, a memory, a learning processor, a processor, etc.

210 100 The communication unitmay transmit and receive data with an external device such as the AI device.

230 231 231 231 240 a The memorymay include a model storage unit. The model storage unitcan store a model (or artificial neural network,) being learned or learned through the learning processor.

240 231 200 100 a The learning processorcan use learning data to learn the artificial neural network. The learning model can be used while being loaded on the AI serverof the artificial neural network, or can be loaded on an external device such as an AI deviceand used.

230 The learning model can be implemented by hardware, software, or a combination of hardware and software. If part or all of the learning model is implemented by software, one or more instructions constituting the learning model can be stored in the memory.

260 The processorcan infer a result value for new input data using the learning model, and generate a response or control command based on the inferred result value.

3 FIG. 1 illustrates an AI systemaccording to an embodiment of the present disclosure.

3 FIG. 1 200 100 100 100 100 100 10 100 100 100 100 100 100 100 a b c d e a b c d e a e. Referring to, the AI systemis connected to at least one of an AI server, a robot, an autonomous vehicle, an XR device, a smartphone, or an appliancewith a cloud network. Here, the robot, the autonomous vehicle, the XR device, the smartphone, or the applianceto which AI technology is applied may be referred to as an AI deviceto

10 10 The cloud networkmay mean a network that constitutes part of a cloud computing infrastructure or exists within a cloud computing infrastructure. Here, the cloud networkmay be configured using a 3G network, a 4G or Long Term Evolution (LTE) network, a 5G network, or the like.

100 100 200 1 10 100 100 200 a e a e That is, each deviceto,constituting the AI systemcan be connected to each other through the cloud network. In particular, each deviceto,can communicate with each other through the base station, but can also communicate with each other directly without going through the base station.

200 The AI servercan include a server that performs AI processing and a server that performs calculations on big data.

200 1 100 100 100 100 100 10 100 100 a b c d e a e. The AI serveris connected to at least one or more of the AI devices constituting the AI system, such as a robot, an autonomous vehicle, an XR device, a smartphone, or a home appliance, through the cloud network, and can assist at least a part of the AI processing of the connected AI devicesto

200 100 100 100 100 a e a e. At this time, the AI servercan train an artificial neural network according to a machine learning algorithm on behalf of the AI deviceto, and can directly store the learning model or transmit the learning model to the AI deviceto

200 100 100 100 100 a e a e. At this time, the AI servercan receive input data from the AI deviceto, infer a result value for the received input data using the learning model, generate a response or control command based on the inferred result value, and transmit it to the AI deviceto

100 100 a e Alternatively, the AI devicetocan directly infer a result value for the input data using the learning model, and generate a response or control command based on the inferred result value.

100 100 100 100 100 a e a e 3 FIG. 1 FIG. Hereinafter, various embodiments of the AI devicetoto which the above-described technology is applied will be described. Here, the AI devicestoillustrated incan be considered as specific examples of the AI deviceillustrated in.

100 c The XR devicecan be implemented as a Head-Mounted Display (HMD), a Head-Up Display (HUD) equipped in a vehicle, a television, a mobile phone, a smart phone, a computer, a wearable device, a home appliance, a digital signage, a vehicle, a fixed robot or a mobile robot, etc. by applying AI technology.

100 100 c c The XR devicecan obtain information about the surrounding space or a real object by analyzing 3D point cloud data or image data acquired through various sensors or from an external device to generate location data and attribute data for 3D points, and can render and output an XR object to be output. For example, the XR devicecan output an XR object including additional information about the recognized object corresponding it to the recognized object.

100 100 100 200 c c c The XR devicecan perform the above-described operations using a learning model composed of at least one artificial neural network. For example, the XR devicecan recognize a real object from 3D point cloud data or image data using the learning model, and provide information corresponding to the recognized real object. Here, the learning model may be learned directly in the XR deviceor learned from an external device such as an AI server.

100 200 c At this time, the XR devicemay generate a result using the learning model directly and perform the operation, but may also transmit sensor information to an external device such as an AI serverand receive the result generated accordingly and perform the operation.

4 FIG. 100 illustrates an AI deviceaccording to an embodiment of the present disclosure.

1 FIG. Descriptions overlapping withare omitted.

4 FIG. 120 121 122 123 Referring to, the input unitmay include a camera (Camera,) for inputting a video signal, a microphone (Microphone,) for receiving an audio signal, and a user input unit (User Input Unit,) for receiving information from a user.

120 Voice data or image data collected by the input unitmay be analyzed and processed as a user control command.

120 100 121 The input unitis for inputting video information (or signal), audio information (or signal), data, or information input from a user. For inputting video information, the AI devicemay be equipped with one or more cameras.

121 151 170 The cameraprocesses image frames such as still images or moving images obtained by an image sensor in a video call mode or a shooting mode. The processed image frames may be displayed on a display unit (Display Unit,) or stored in a memory.

122 100 122 The microphoneprocesses an external acoustic signal into electrical voice data. The processed voice data can be utilized in various ways depending on the function being performed (or the application being executed) in the AI device. Meanwhile, various noise removal algorithms can be applied to the microphoneto remove noise generated in the process of receiving an external acoustic signal.

123 123 180 100 The user input unitis for receiving information from a user, and when information is input through the user input unit, the processorcan control the operation of the AI deviceto correspond to the input information.

123 100 The user input unitmay include a mechanical input means (or a mechanical key, for example, a button located on the front/rear or side of the terminal, a dome switch, a jog wheel, a jog switch, etc.) and a touch input means. As an example, the touch input means may be composed of a virtual key, a soft key, or a visual key displayed on a touch screen through software processing, or may be composed of a touch key placed on a part other than the touch screen.

150 151 152 153 154 The output unitmay include at least one of a display unit, a sound output unit, a haptic module, and an optical output unit.

151 100 151 100 The display unitdisplays (outputs) information processed in the AI device. For example, the display unitmay display execution screen information of an application program running in the AI device, or User Interface (UI) or Graphical User Interface (GUI) information according to such execution screen information.

151 123 100 100 The display unitcan implement a touch screen by forming a mutual layer structure with the touch sensor or by forming it as an integral part. This touch screen can function as a user input unitthat provides an input interface between the AI deviceand the user, and at the same time, can provide an output interface between the terminaland the user.

152 110 170 The audio output unitcan output audio data received from the communication unitor stored in the memoryin a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, etc.

152 The sound output unitmay include at least one of a receiver, a speaker, and a buzzer.

153 153 The haptic modulegenerates various tactile effects that the user can feel. A representative example of the tactile effect generated by the haptic modulemay be vibration.

154 100 100 The light output unitoutputs a signal to notify the occurrence of an event using the light of the light source of the AI device. Examples of events generated by the AI devicemay include message reception, call signal reception, missed call, alarm, schedule notification, email reception, information reception through an application, etc.

Hereinafter, an AI-based plant cultivation control system (hereinafter referred to as a “plant cultivation control system” for convenience of explanation) is described.

Recently, the number of users who want to grow plants directly using a plant cultivation machine is increasing. These users are not only interested in the results of plant cultivation, but also have expectations about the cultivation process of a companion plant, such as a desired plant, just like a pet. Normally, when a seed kit is inserted into a plant cultivation machine, it takes about 4 to 6 weeks until the cultivation period, and the user may feel that this period is long, which may cause them to lose interest in plant cultivation. Therefore, in this disclosure, in addition to the automatic control function of the plant cultivation machine, even non-expert users are provided with prediction information about the future growth process of the target plant during the cultivation period, thereby providing a service to induce enjoyment and interest in using a plant cultivation machine or growing plants, and to enable virtual experiences.

As AI technology is applied or grafted into various fields, AI technology can also be used in a plant cultivation control system such as this disclosure. In this disclosure, by using a model generated based on artificial intelligence, prediction data (e.g., video or image) on the growth process or growth value of a plant by period is provided, thereby attracting the interest of users using a plant cultivation device and increasing convenience.

Meanwhile, a plant cultivation device has one purpose of creating an optimal environment for growing fresh plants in a relatively short period of time. Accordingly, in this disclosure, image data on a target plant (or all plants) in an artificial intelligence-based plant cultivation device is acquired, and a system is built and provided to automatically optimize various types of plant cultivation environments based on a machine learning recognition model, and furthermore, a system capable of preventing diseases that may occur during the cultivation process is provided, thereby increasing user satisfaction with the service.

In the present disclosure, for artificial intelligence-based plant cultivation control, the following embodiments are described.

First, this disclosure can provide prediction information on the growth process or growth value of a target plant by period using a model generated based on artificial intelligence. Such prediction information can be provided as image data. To this end, the present disclosure can provide a period-based growth image of a target plant based on a future prediction generation model described later as an AI-based generation model. In addition, the present disclosure can also provide information on changes in the plant growth status according to adjustment of the plant growth environment values (e.g., external temperature, humidity, soil, light source, ventilation, etc.) set in a plant cultivation device using an AI-based generation model. Information on changes in the plant growth status can also be provided as image data. However, the present disclosure is not limited to the above-described examples.

Meanwhile, the present disclosure can provide an AI service based on a machine learning recognition model for image data of a target plant in a plant cultivation device. For example, the present disclosure can provide an AI service for automatically optimizing the cultivation environment for a target plant in a plant cultivation device, a notification for determining whether or not a target plant is sick or a user guide (e.g., whether pruning is necessary for the target plant), and an AI service for selecting a cultivation/harvest time or a harvestable plant (or leaves or fruits, etc.). Here, image data for the target plant in the plant cultivation device can be acquired through an image acquisition device, and such an image acquisition device may include an image sensor built into the plant cultivation device or an external image sensor of the plant cultivation device or a user's terminal equipped with an image sensor.

The present disclosure can be applied not only to small plant cultivation devices but also to medium-to large-sized plant cultivation devices or systems such as smart farms. However, for convenience of explanation, a plant cultivation device is taken as an example.

5 FIG. is a drawing illustrating an artificial intelligence-based plant cultivation control system according to an embodiment of the present disclosure.

6 FIG. 5 FIG. 260 is a block diagram illustrating the operation of the processorof.

7 FIG. 6 FIG. 630 is a block diagram illustrating the operation of the learning unitof.

500 200 200 260 500 An AI-based plant cultivation control system according to at least one of the various embodiments of the present disclosure may include a plant cultivation deviceand a computing device (e.g., an AI server)that communicates with the plant cultivation device to exchange signals. At this time, the computing devicemay include a processorthat generates a plant growth prediction model, acquires an image of a target plant in the plant cultivation devicewhen an event is detected, generates cultivation information of the target plant for the detected event and the acquired image based on the plant growth prediction model, and controls the output of the generated cultivation information of the target plant.

5 FIG. 100 200 500 Referring to, the plant cultivation control system may be configured to include an AI device, a computing device (AI server), and a plant cultivation device. Depending on the embodiment, the plant cultivation control system may be configured to include one or more additional components in addition to the illustrated configuration.

100 500 The AI devicemay be a terminal of a user who grows plants through a plant cultivator.

100 200 500 The AI devicemay include or be linked to various output interfaces. These output interfaces may include a display that outputs a screen for various information generated by a computing device, a speaker that outputs sound, etc. The display may provide a user interface (UI) necessary for a terminal user to make a request, control, provide a result according to a request, etc. related to the control of the plant cultivator.

100 500 500 Some screens of the AI devicemay output corresponding information or results when a user executes or controls a function through a button provided on the plant cultivator, or when a display is not employed in the plant cultivator.

100 As described above, the AI devicemay be a fixed terminal such as a PC, DTV, or digital signage, or a mobile terminal such as a smartphone, tablet PC, laptop, or wearable device.

100 500 If the AI devicesupports an augmented reality (AR) function or an augmented reality application or program, it may provide various information disclosed in the present disclosure about a target plant in the plant cultivatorto the user as an augmented reality screen. The present disclosure is not limited thereto, and may also be applied to extended reality (XR: extended Reality), including not only augmented reality but also virtual reality (VR: Virtual Reality), mixed reality (MR: Mixed Reality).

500 100 200 The plant cultivatormay be connected to the AI deviceand the computing devicethrough a wired/wireless communication protocol, and may exchange signals containing various information.

500 The plant cultivation devicecan include or detachably attach various sensors on or inside the housing, and can sense or acquire various information through them. These sensors may include an image sensor, a temperature sensor, a humidity sensor, a light sensor, etc. At least one side of the housing may be implemented with a transparent display or made of transparent glass so that the inside can be seen.

6 FIG. 260 200 In, a detailed configuration block of a processorconstituting a computing deviceis illustrated.

260 8 15 FIGS.to The processormay perform or control operations related todescribed below.

6 FIG. 260 610 620 630 670 Referring to, the processormay be configured to include a data collection module, a data classification module, a learning module, and a control module.

260 640 650 660 640 650 660 260 670 The processormay further include a growth prediction module, a cultivation environment processing module, a disease processing module, etc. However, depending on the embodiment, the growth prediction module, the cultivation environment processing module, the disease processing module, etc. may not be included as components of the processor, but the functions performed by the corresponding modules may be performed by the control module.

610 610 100 The data collection modulecan collect various data related to plant growth (cultivation). The data collection modulecan collect plant growth data for artificial intelligence learning from an external source through a communication module (not shown) or can directly receive growth data of a target plant from an AI device.

620 610 630 620 The data classification modulecan classify and process plant growth data collected or received through the data collection moduleso that the learning modulecan learn it. The data classification modulecan preprocess the received data for classification processing.

670 610 230 620 230 5 FIG. The control modulecan control data collected or received through the data collection moduleto be stored in the memoryshown in, or control data classified by the data classification moduleto be stored in the memory.

630 620 The learning modulecan learn plant growth data input through the data classification modulebased on a pre-generated learning model.

7 FIG. 630 is illustrated to explain the operation of the learning moduleaccording to an embodiment of the present disclosure.

630 The operation of the learning modulecan largely include training, that is, a learning process and an inference process.

630 730 The learning modulecan learn plant growth data using a training process, for example, a pre-generated learning model, to generate a future prediction model.

7 FIG. 730 illustrates a future prediction modelfor performing an inference process.

6 7 FIGS.and 630 720 730 Referring to, the learning modulecan generate a virtual plant growth image from the plant growth future prediction image generation modelusing the input data and the future prediction model.

500 The input data may include a target plant image, target plant growth environment information, and future prediction period information of the target plant. The target plant growth environment information may include, for example, the internal temperature, humidity, soil composition, and light source information of the plant cultivation device. The future prediction period information of the target plant may include time information, which is for the user to check the growth status of the target plant at a certain point in the future, and an arbitrary time (for example, one week later, two weeks later, one month later, etc.) may be set.

740 730 The plant growth future prediction image generation model may generate virtual growth prediction image dataof the corresponding time or period based on the aforementioned input data and the future prediction model.

730 In the present disclosure, in relation to the construction or generation of the future prediction model, specific descriptions may refer to known technologies. At this time, the known technologies relate to a model that generates age-based images based on faces. However, the present disclosure does not use known technologies as they are, and instead of face images, images of target plants that are the subject of the present disclosure may be used. In addition, instead of age (age) regarding growth, desired future growth prediction intervals or time information may be used as input in the present disclosure.

According to one embodiment of the present disclosure, multiple output images, i.e., predicted virtual image data, may be generated based on one input image. At this time, each output image may be expected image data for a specific period or may be expected image data for different periods.

630 720 640 7 FIG. When learning is completed in the learning moduleand the image generation modelofis generated, the growth prediction modulemay generate future growth prediction virtual images for the target plant based on the input data as described above.

650 500 650 640 The cultivation environment processing modulecan process cultivation environment information of the target plant in the plant cultivation device. The cultivation environment processing modulecan operate based on the result data of the growth prediction moduledescribed above, that is, the generated virtual image of future growth prediction for the target plant and data related to cultivation environment among the input data thereof.

660 500 660 640 The disease processing modulecan process disease information of the target plant in the plant cultivation device. The disease processing modulecan operate based on the result data of the growth prediction moduledescribed above, that is, the generated virtual image of future growth prediction for the target plant and data related to disease among the input data thereof.

The operation described above can include at least one of generation, transmission, etc. of control information for cultivation environment control or disease treatment control.

670 260 230 670 6 7 FIGS.and The control modulecan control the overall operation of the processorand can also exchange data between the memory. The control modulecan control the operation of the components or specific components illustrated in.

The AI-based plant cultivation control method will be described in more detail with reference to the attached drawings.

8 12 FIGS.to are flowcharts illustrating the AI-based plant cultivation control method according to an embodiment of the present disclosure.

8 FIG. 200 Referring to, an embodiment of a method for controlling AI-based plant cultivation in a computing devicemay be performed as follows.

200 101 The computing devicemay generate a plant growth prediction model (S).

200 103 The computing devicemay detect a first event (S).

200 105 The computing devicemay acquire an image of a target plant in a plant cultivation device (S).

200 107 The computing devicemay generate cultivation information of the target plant for the detected first event and the acquired image based on the plant growth prediction model (S).

200 109 The computing devicecan control the output of the cultivation information of the target plant generated above (S).

200 101 The computing devicecan collect plant growth data, preprocess the collected plant growth data, and learn the preprocessed plant growth data based on a predefined future prediction generation model in relation to step S.

Meanwhile, in the present disclosure, the cultivation information of the target plant may include at least one of plant growth prediction image information, plant cultivation environment setting information, plant disease presence/absence information, cultivation time information, and selection information for a part of a plant that can be cultivated, but is not necessarily limited thereto.

9 FIG. 200 201 Next, referring to, the computing devicecan extract plant growth prediction image information from the cultivation information of the target plant according to the first event detected (S).

200 203 The computing devicecan generate a virtual plant growth prediction image for a predefined period unit based on the extracted plant growth prediction image information (S).

200 205 The computing devicecan control the generated virtual plant growth prediction image to be sequentially output for a period unit (S).

10 FIG. 200 301 Next, referring to, the computing devicecan analyze the image of the target plant in the acquired plant cultivation device according to the detected first event (S).

200 303 The computing devicecan generate status information of the target plant in the analyzed plant cultivation device (S).

200 305 The computing devicecan generate notification information corresponding to the generated status information of the target plant (S).

200 307 The computing devicecan control the generated notification information to be output (S).

11 FIG. 200 401 Next, referring to, the computing devicecan detect a second event (S).

200 403 The computing devicecan extract cultivation time information and cultivation environment information from the cultivation information of the target plant (S).

200 405 The computing devicecan calculate cultivation environment information for cultivation of the target plant at the cultivation time according to the second event detected based on the extracted cultivation time information and cultivation environment information of the target plant (S).

200 407 The computing devicecan generate cultivation environment information control information currently set for the target plant based on the generated cultivation environment information (S).

200 409 The computing devicecan transmit the generated cultivation environment information control information to the plant cultivation machine (S).

8 FIG. 11 FIG. 500 Here, at least one of the events in the aforementionedandcan be received from one of a predetermined button provided on the plant cultivator, a first mobile device having a plant cultivation control application installed, and a second mobile device linked with the first mobile device.

12 FIG. 200 500 501 Finally, referring to, the computing devicecan obtain information on the growth status of the entire target plant in the current plant cultivatorand information on setting the plant cultivation environment (S).

200 503 The computing devicecan derive a virtual plant growth prediction image of a predefined period unit of the corresponding plant based on the acquired growth status information of the entire target plant and the plant cultivation environment setting information (S).

200 505 The computing devicecan compare the derived current virtual plant growth prediction image for the target plant with the virtual plant growth prediction image for the target plant generated according to the previous event by period unit (S).

200 507 The computing devicecan analyze the comparison results and update the current plant cultivation environment setting information (S).

13 FIG. 15 to tare drawings illustrating a user interface related to an AI-based plant cultivation control according to an embodiment of the present disclosure.

13 15 FIGS.to 8 12 FIGS.to may be user interface screens related to the method ofdescribed above.

13 FIG. illustrates a virtual growth prediction image for a target plant by period unit.

13 a FIG.() 200 100 Referring to, a computing devicecan provide a growth prediction image of a target plant for 3 days, 1 week, 2 weeks, and 1 month from the present to an AI device.

13 a FIG.() Referring to, identifier information (e.g., plant A) that can identify a target plant may be provided in the upper area of each image, and period information (e.g., 3 days, 1 week, 2 weeks, 1 month, etc.) may be provided in another area.

100 200 When a signal for selecting identifier information is received from the AI device, the computing devicemay provide a list of plants for which growth prediction images can be provided, and may change and provide a growth prediction image of a plant selected from the plant list.

100 200 Meanwhile, when a signal for selecting period information is received from the AI device, the computing devicemay provide a list of selectable periods, and may change and provide a growth prediction image of a period selected from the period list.

13 a FIG.() 200 According to the embodiment, four virtual growth prediction images for a period unit for one target plant are provided on each user interface screen of, but when a request for changing period information is received on one of the growth prediction images, the computing devicemay automatically change the period of the remaining growth prediction images accordingly and provide them as growth prediction images corresponding to the corresponding period.

13 a FIG.() 13 b FIG.() Meanwhile, unlike as shown in, each user interface screen provides virtual growth prediction images for one period, i.e., the same period, but the target plants corresponding to each image may be different, as shown in.

13 FIG. 14 FIG. Unlike, in, a virtual growth prediction image of a target plant may be provided with one user interface.

14 a FIG.() In the graph shown in, the horizontal axis may represent the period and the vertical axis may represent the height (growth level).

14 a FIG.() 200 In, the computing devicecan provide status information by providing a solid line connecting each virtual growth prediction image for the target plant and a dotted line indicating a level predicted to be able to grow to the maximum (Max) when the optimal cultivation environment setting is followed.

14 a FIG.() 200 In, the computing devicecan provide a virtual growth prediction image after each period has elapsed based on the current cultivation environment setting.

200 14 a FIG.() The computing devicecan indicate that growth promotion is possible for the target plant by changing the cultivation environment setting on the graph shown in.

100 200 200 14 a FIG.() When the user selects a dotted line or solid line or requests a cultivation environment change setting through the AI devicein, the computing devicecan provide the current cultivation environment setting information corresponding to the solid line and the cultivation environment setting information corresponding to the dotted line together so that they can be compared. At this time, the computing devicecan blank out items that do not require change, and can differentiate items that require change or are necessarily requested to be changed (e.g., disease concerns, etc.).

200 14 a FIG.() According to the embodiment, the computing devicecan control the requested cultivation environment change setting to be made from the corresponding period when a specific period is selected inand the requested cultivation environment change setting is requested.

14 FIG. 13 b FIG.() 14 a FIG.() (b) can be viewed as a user interface screen corresponding to, and all of the contents disclosed indescribed above can be applied equally.

13 b FIG.() 14 b FIG.() 500 Meanwhile,andcan be adopted, for example, when there are multiple plants being grown in one or the same plant cultivation device.

15 FIG. is a user interface illustrated to explain the control of adjusting the flowering/harvesting time.

15 a FIG.() 13 14 FIGS.and 200 Referring to, unlike, the computing deviceis a control scenario for adjusting the flowering/harvesting time, not promoting the growth of the target plant.

200 100 15 a FIG.() The computing devicemay provide a message guiding that the solid line can be moved when the solid line is selected by the AI devicein the user interface of.

15 a FIG.() can be viewed as an example of a case where the solid line is moved downward by providing a guidance message such as “Do you want to adjust the flowering/harvesting time?” as illustrated, for example. However, it is not limited thereto.

15 a FIG.() 200 In other words, when a solid line is selected in the user interface illustrated in, the computing devicecan guide that the solid line can move, and can provide the solid line to move in any direction, up, down, left, or right.

200 15 b FIG.() When the solid line moves to a different location than before, the computing devicecan determine the moving location relative to the reference line (solid line) and provide a corresponding guidance message again, as in. At this time, the guidance message can include guidance on changes in flowering/harvesting time due to movement, guidance on expected flowering/harvesting time, guidance on changes in cultivation environment settings due to changes, etc.

15 b FIG.() 200 When a positive signal or an agreement signal is received in the guidance message according to, the computing devicecan extract cultivation environment setting information to be changed accordingly, and finally change the cultivation environment settings.

15 FIG. In, one target plant is used as an example for convenience of explanation, but as described above, this can be applied to multiple target plants as well, and batch setting change processing can also be performed as needed.

16 FIG. is a drawing illustrating a scenario for providing information on target plant cultivation based on artificial intelligence according to an embodiment of the present disclosure.

16 a FIG.() 500 illustrates a plant cultivation machinein which multiple plants are being cultivated.

16 b FIG.() 16 a FIG.() 100 200 As in, when the AI devicefocuses (or captures) a predetermined space or a predetermined plant in, the computing devicecan select a target area and provide status information on each target plant included in the selected target area.

16 b FIG.() 200 100 Referring to, the computing devicecan provide marking information on a target plant that is in a bad condition and needs pruning, marking information on a target plant that can be harvested in two days, and marking information on a target plant that can be harvested today as an augmented reality view (AR view) output from the AI device.

16 b FIG.() 16 b FIG.() 100 The augmented reality view ofmay include differentiated content so that the user can easily recognize and identify the state of each target plant through the AI device. For the convenience of identification, the augmented reality view ofmay provide various additional information such as text, audio, and emoji.

200 16 b FIG.() The computing devicemay separately monitor target plants viewed in the augmented reality view, as shown in, and target plants selected or accessed in the augmented reality view.

The order of operations described in the present disclosure is not necessarily bound to the order described in the drawings or in the specification, and some operations may be performed together or in a different order than that depicted depending on the embodiment.

According to at least one of the various embodiments of the present disclosure described above, the convenience of using a plant cultivation machine can be improved through various plant cultivation control information including information on growth prediction Of a target plant based on artificial intelligence, and a virtual experience service can be provided based on the automatic control function of the plant cultivation machine and prediction information on the future growth process of the target plant during the cultivation period, so that even non-experts can use the plant cultivation machine easily and conveniently, and also maximize the service quality by inducing enjoyment and interest in plant cultivation.

According to one embodiment of the present invention, the above-described method can be implemented as a code that can be read by a processor in a medium in which a program is recorded. Examples of the medium that can be read by a processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

The display device described above is not limited to the configuration and method of the embodiments described above, and the embodiments can be configured by selectively combining all or part of each embodiment so that various modifications can be made.

According to the artificial intelligence-based plant cultivation control according to the present disclosure, not only the virtual growth prediction information of the plant but also various information or information on control required for plant cultivation are provided, so that even non-experts can easily and conveniently perform plant cultivation, thereby maximizing service satisfaction, and it can be applied or grafted onto various plant cultivation devices, so it has industrial applicability.