Information Processing Apparatus and Control Method Thereof

The present invention relates to a management technique of a learning model and a learning data.

In recent years, systems using machine learning techniques have been put to practical use in various fields. Known learning models and learning data used in such systems have been often created by companies, universities, and the like. However, in recent years, it has also become possible for a general user to create a learning model or learning data for the user's own purpose. For this reason, there are an enormous number of learning models and learning data.

A service via the Internet that can release and acquire learning models and learning data created by general users is assumed. A user who uses the service (e.g., a user who desires to create a customized model) needs to select a learning model or learning data suitable for the user's own purpose from among released learning models and learning data. However, when even a general user can create and release a learning model, an enormous number of learning models and learning data exist, which are to be updated daily. Therefore, it is difficult for the user who uses the service to grasp the status of the learning model and the learning data.

Japanese Patent Laid-Open No. 2022-178892 (Patent Document 1) discloses a method of searching for and finding target learning data based on a tag applied to learning data. Japanese Patent Laid-Open No. 2022-61191 (Patent Document 2) discloses a method of newly outputting an inference result and notifying a user of the result when a saved learning model is updated.

However, with the technique described in Patent Document 1, it is difficult to grasp the update status of the learning data. With the technique described in Patent Document 2, it is necessary to save the learning model and the inference result into a database every time the learning model is updated, and therefore the operation cost increases.

According to one aspect of the present invention, an information processing apparatus comprises: a model management unit that manages, in association with a first learning model, information on a second learning model used for creation of the first learning model and information on a first data set used for creation of the first learning model; a data management unit that manages, in association with the first data set, information on a second data set used for creation of the first data set; a detection unit that detects that a third learning model created by relearning on the second learning model has been added to a management target by the model management unit or that a third data set created by data manipulation on the second data set has been added to a management target by the data management unit; a setting unit that sets a notification condition for notifying that the third learning model or the third data set has been added; and a notification unit that performs notification if the notification condition set by the setting unit is satisfied.

The present invention facilitates acquisition of a learning model and learning data suitable for a user's purpose.

Further features of the present invention will become apparent from the following description of exemplary embodiments (with reference to the attached drawings).

Hereinafter, embodiments will be described in detail with reference to the attached drawings. Note, the following embodiments are not intended to limit the scope of the claimed invention. Multiple features are described in the embodiments, but limitation is not made to an invention that requires all such features, and multiple such features may be combined as appropriate. Furthermore, in the attached drawings, the same reference numerals are given to the same or similar configurations, and redundant description thereof is omitted.

As the first embodiment of an information processing apparatus according to the present invention, a server apparatus that manages a learning model and learning data will be described below as an example.

The server apparatus according to the present embodiment manages a model (learned model) and a data set together with traceability information (past update history of the model and the data set). When the model and the data set managed by the server apparatus are updated, the server apparatus notifies a target user that the update has been performed based on the notification setting registered in advance.

is a view illustrating an overall configuration of the system. A server apparatusis an information processing apparatus that operates as a server apparatus that manages a learning model and learning data. For example, it is arranged as a virtual server on a cloud server. User apparatusestoare information processing apparatuses that operate as user apparatuses that acquire the learning model and the learning data from the server apparatus and use the learning model and the learning data. As illustrated in, the virtual server is configured in such a manner that communication is possible with a plurality of user apparatuses, and is configured in such a manner that the user apparatuses can be provided with the learning model and the learning data in response to a request from the user apparatuses. The user apparatuses are terminal apparatuses with which a user browses and operates screens, and are, for example, a personal computer (PC) and a tablet terminal.

is a view illustrating a hardware configuration of the server apparatus. An His a CPU, and controls various devices connected to a system bus H. An His a ROM, and stores a basic input/output system (BIOS) program and a boot program. An His a RAM, and is used as a main storage apparatus of the H, which is a CPU. An His an interface (I/F), and performs data communication with an external apparatus. For example, the interface may be a communication interface such as Ethernet (registered trademark), or may be a general-purpose interface such as USB or serial communication. The interface may be a wired connection interface or a wireless connection interface.

is a view illustrating a hardware configuration of the user apparatus. An His a CPU, and controls various devices connected to a system bus H. An His a ROM, and stores a BIOS program and a boot program. An His an input apparatus, and performs processing related to input of various types of information. Examples thereof include a touch panel, a keyboard, a mouse, and a robot controller. An His a display apparatus, and performs processing related to display of various types of information. For example, it displays a processing result processed by the user apparatus itself or a processing result processed by the server apparatus and transmitted to the user apparatus. Note that the display apparatus may be of any type, such as a liquid crystal display apparatus, a projector, or an LED indicator.

An His a RAM, and is used as a main storage apparatus of the H, which is a CPU. An His a hard disk, and is used to store an application program, data, a library, and the like. An His a media drive, and enables data writing/reading to/from a removable storage medium. Thereby, it enables data to be moved to an external apparatus (digital still camera, PC, and tablet terminal). An His an interface (I/F), and performs data communication with an external apparatus. For example, the interface may be a communication interface such as Ethernet (registered trademark), or may be a general-purpose interface such as USB or serial communication. The interface may be a wired connection interface or a wireless connection interface.

As a case of the task handled in the first embodiment, an object detection task for inputting an image will be described here. The object detection task is a task of detecting a specific object in an image and inferring a bounding box (BB) surrounding the specific object when image data is input. However, the type of the task is not limited to object detection. For example, the present invention can be applied to various tasks such as a task of estimating and dividing a region and a classification task of classifying a subject (a person, a car, or the like).

is a view illustrating a functional configuration of the server apparatus. The server apparatusincludes a control unitand a storage unit. Details of each configuration will be described below.

The control unitincludes a model management unit, a data management unit, a traceability information management unit, a registration unit, a notification setting unit, and a notification unit. Each of these functional units can be implemented, for example, by the CPU executing various programs. However, some or all of them may be implemented by hardware such as an application specific integrated circuit (ASIC).

The model management unitgenerates management information for a model (learned model), and manages the management information in association with the model. Information as illustrated inis generated and applied to the model to be managed.

The “model ID” is model-specific identification information (ID). The “creation user ID” is ID of a user who performed learning of the model. The “category” is information on an object targeted by the model, information necessary for the user to search for the model, and the like. The “task” is a type of processing targeted by the model, such as object detection and image generation.

The “initial model” describes ID of a model that is an initial parameter at the time of learning the model. For example, as illustrated in, the initial model for a modelis a model with a model ID “md_0001” and the model with the model ID “md_0001” is a model. That is, it is indicated that the modelis a model obtained as a result of (additional) learning with the modelas an initial model. The “learning data set” is ID of a data set used for learning the model. The “evaluation data” is ID of data used when the model is evaluated.

The data management unitgenerates management information for a data set used for learning and evaluation, and manages the management information in association with the data set. Information as illustrated inis generated and applied to the data set to be managed.

The “data set ID” (hereinafter written as “data ID”) is data set-specific ID. The “number of images” is the number of images held by the data set. The “number of GT” is the number of ground truth (GT) applied to the data set. The “category” is information on an object targeted by the data set or information necessary for the user to search for data. The “task” is a type of processing targeted by the data set, such as object detection and image generation.

The “subset” describes ID of a data set from which the data set is created. For example, as illustrated in, the subset for a data setis a data set with a data ID “ds_0002” and the model with the data ID “ds_0002” is a data set. That is, it is indicated that the data setis a data set created based on the data set. The “creation user ID” is ID of a user who has created the data set.

The traceability information management unitmanages traceability information on the model and the data set. Here, the traceability information is information indicating how each model and data set to have been created in the past (i.e., past history of the model and the data set).

is a view illustrating an example of traceability information. For example, it is indicated that a modelwith the model ID “md_0001” is created by learning using the data set with the data ID “ds_0001”. It is indicated that a modelwith a model ID “md_0002” is created by learning using the data set with the data ID “ds_0001” and a data set with a data ID “ds_0003”, of the model ID “md_0001” as an initial model.

These pieces of traceability information are created based on the model management information and the data management information described above. As described above, the traceability information indicates the past history of each model and data set. Therefore, by referring to the traceability information, it is possible to extract information regarding the model and the data set used for creation of each model and data set.

The registration unitregisters, updates, and deletes the model and the data set. When the registration unitregisters, updates, and deletes the model or the data set, the traceability information management unitupdates the traceability information. That is, the traceability information management unitupdates the traceability information based on the model management information and the data management information related to the registered, updated, and deleted model and data set.

The notification setting unitsets notification information requested by the user.

is a view illustrating an example of notification information. The “notification ID” is notification-specific ID. The “user ID” describes the ID of a user who has set the notification information or the user who is a transmission destination of the notification information. The “related model/data” describes model ID of a model or data ID of a data set related to the notification information.

The “notification status” describes a status (e.g., data change) in which notification of notification information is made. For example, notification informationwith notification ID “rep_0001” is notification information set by the user of user ID “user_1001”, and it is indicated that the notification informationis notified when a model additionally learned for the model ID “md_0001” is registered. The “notification option” is for setting an option (e.g., detailed condition setting of the notification status) included in the notification.

The notification unitgenerates notification content based on the traceability information and the notification information set by the notification setting unitand notifies the user of the notification content. Details of a procedure for generating the notification content will be described later with reference to.

is a flowchart of registration of a model and data and notification setting. Here, a case where a certain user collects a dog image and creates and registers a dog detection model is assumed. However, the server apparatusneeds not necessarily perform all the steps described in this flowchart. The image is not limited to the dog, and may be a person, a car, or the like that is a detection target, and the detection model is not limited to the dog detection. The flowchart ofis started when the user (user 1) registers (transmits), to the server apparatus, the dog detection model and the dog image data created by operating the user apparatus.

In S, the model management unitgenerates model management information on the dog detection model transmitted from the user. Here, the model management information on the modelillustrated inis created as the model management information on the dog detection model. In the model management information, the model ID “md_0001” is applied, and the creation user ID “user_1001”, the category “dog”, and the task “object detection” are registered. There is no registration of the initial model, and the learning data set describes the data ID “ds_0001” of the data set registered together with the model. Here, the data ID “ds_1001” is described as a data set used as evaluation data.

In S, the data management unitgenerates data management information on the data set of the dog image transmitted from the user. Here, data management information on a data setillustrated inis created as the data management information on the dog image data. In the data management information, the data ID “ds_0001” is applied, and the number of images “1000”, the number of GT “1200”, the category “dog”, and the task “object detection” are registered. Here, it is assumed that the data included in the data set to be registered is created/collected by the user 1 himself. Therefore, since there is no data associated with the data set, “none” or a blank is registered in the subset. Then, the creation user ID describes the user ID “user_1001” of the user 1.

In S, the traceability information management unitgenerates traceability information. Here, traceability information indicated in the modeland data(corresponding to the dog detection model and the dog image data) ofis generated. From the generated traceability information, it is indicated that the modelof the model ID “md_0001” is created using the data set of the data ID “ds_0001” as learning data.

In S, the registration unitregisters the dog detection model and the dog image data transmitted from the user.

In S, the notification setting unitsets notification information requested by the user. Here, the notification informationofis set as the notification setting set by the user 1. That is, the user 1 assumes a status of “desiring to receive a notification when another user uses the dog detection model registered by himself as an initial model and registers a new detection model”.

Here, the notification settingis applied with the notification ID “rep_0001”, the user ID describes “user_1001”, and the related model/data describes “md_0001”. The “additional learning” is designated in the notification status.

Through the above processing procedure, registration of the learning model and the learning data and notification setting are performed. Although an example of collectively performing registration of the model and the data and notification setting has been described here, registration of only the model, registration of only the data, or only setting of notification may be performed.

is a flowchart of the notification processing. Specifically, it is a processing procedure for notifying the user when the model or data saved in the server apparatusis updated. Here, a status in which a user 2 creates an updated dog detection model and dog data using the dog detection model and the dog data described inis assumed. More specifically, the usercreates a data set (Chihuahua image data) including an image of “Chihuahua” that is his own pet, and additionally learns (relearns) the dog detection model to create a “Chihuahua detection model”. The flowchart ofis started when the user (user 2) registers (transmits), to the server apparatus, the Chihuahua detection model and the Chihuahua image data created by operating the user apparatus.

In S, the model management unitgenerates model management information on the Chihuahua detection model transmitted from the user, and the data management unitgenerates data management information on the Chihuahua image data transmitted from the user.

Here, the model management information on the modelillustrated inis created as the model management information on the Chihuahua detection model. In the model management information, model ID “md_0003” is applied, the model ID “md_0001” is described as an initial model, and the data ID “ds_0001” and “ds_0003” are described in the learning data set.

Data management information on a data setillustrated inis created as data management information on the Chihuahua image data. In the data management information, the data ID “ds_0003” is applied, and the number of images “”, the number of GT “”, the category “dog”, and the task “object detection” are registered. Here, it is assumed that the data included in the data set to be registered is created/collected by the user 2 himself. Therefore, since there is no data associated with the data set, “none” or a blank is registered in the subset. Then, the creation user ID describes the user ID “user_1002” of the user 2.

In S, the traceability information management unitgenerates traceability information from the model management information and the data management information generated in S. Here, traceability information indicated in the modeland data(corresponding to the Chihuahua detection model and the Chihuahua image data) ofis generated. From the generated traceability information, it is indicated that the modelof the model ID “md_0003” is created using the data sets of the data ID “ds_0001” and “ds_0003” as learning data. That is, it is detected that the modelcreated by relearning of the modelhas been added to the management target.

In S, the notification unitgenerates a notification based on the traceability information generated in Sand the notification information () set by the notification setting unit. In the notification information, it is set to notify the userof the user ID “user_1001” when the model of the model ID “md_0001” is additionally learned. Since the above-described Chihuahua detection model (model ID “md_0003”) is created using the model of the model ID “md_0001” as an initial model, the notification is generated.

In S, the notification unitnotifies the user of the notification generated in S. In the example of the generated notification, the user 1 (e.g., the user apparatus) is notified that the model of the model ID “md_0003”, which is a relearning model using the model of the model ID “md_0001” as an initial model, has been registered.

The above-described processing enables the user 1 to easily know that the dog detection model (model ID “md_0001”) created by himself has been relearned by the user 2 and the Chihuahua detection model (model ID “md_0003”) has been created.

While in the example ofdescribed above, the notification in a case where the model is updated (relearned) is described, the notification may be made in a case where data is updated. For example, the user 2 assumes a status of desiring to receive a notification when the data set of the data ID “ds_0001” used when the Chihuahua detection model was created is changed is assumed. Here, it is assumed that the user ID, the model ID, the related model/data, the notification status, and the notification option indicated in the notification informationofare set. Here, “update of 1000 sheets or more (of data)” is set as a notification option. Note that a setting that the notification is made by a change of one sheet may be set, but the notification is made by a change of a certain number of sheets or more may be set so that the frequency of the notification is not excessive.

It is assumed that the user 1 modifies (cleanses) the GT of the data set of the data ID “ds_0001”. When registering the cleansed data, the user 1 writes a subset “ds_0001 (cleansing)” as the data setof, and registers that the data set is based on the data set of the data ID “ds_0001”. By this, the notification setting unitgenerates a notification based on the notification informationinand the data management information on the data setof. Specifically, the user 2 is notified that there has been a change in the data set of the data ID “ds_0001”.