Information processing device, method for generating learning model, and computer-readable storage medium that stores program for generating learning model

The present disclosure relates to an information processing device, a method for generating a learning model, and a computer-readable storage medium that stores a program for generating a learning model.

Japanese Laid-Open Patent Publication No. 2010-33377 discloses a search method for assigning an attribute of a search request to a query. In this method, training data is generated based on a click log. The click log includes a query that has been input for web search and a click URL selected from a list of search results. The training data is used for machine learning that estimates a search request attribute of the query.

To create a machine learning model used for search engine, actual click logs are generally used for training data as described above. However, for example, when the number of search targets is relatively large as in a shopping website that offers a variety of products, the log of a target with a relatively low search frequency may be unable to be sufficiently obtained. As a result, a desired prediction accuracy may be unable to be obtained.

An object of the present disclosure is to provide an information processing device for generating a learning model, a method for generating a learning model, and a computer-readable storage medium that stores a program for generating a learning model that provide a relatively high prediction accuracy even when the number of search targets is relatively large.

This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key characteristics or essential characteristics of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

An information processing device according to an aspect of the present disclosure is an information processing device that generates a learning model. The learning model is configured to output attribute information of a target based on a search query that has been input to search for the target. The information processing device includes one or more processors and one or more memories. At least one of the one or more memories stores target data. The target data includes target datasets. Each of the target datasets includes, as a target data field, identification information of the target and the attribute information of the target. At least one of the one or more processors is configured to execute a process that updates some of parameters included in the learning model by giving the target data to one or more training tasks and executing the one or more training tasks.

A method according to an aspect of the present disclosure is a method for generating a learning model executed by an information processing device. The learning model is configured to output attribute information of a target based on a search query that has been input to search for the target. The method includes obtaining target data. The target data including target datasets. Each of the target datasets includes, as a target data field, identification information of the target and the attribute information of the target. The method further includes defining one or more training tasks and updating some of parameters included in the learning model by giving the target data to the one or more training tasks and executing the one or more training tasks.

A computer-readable storage medium according to an aspect of the present disclosure stores a program for generating a learning model. The learning model is configured to output attribute information of a target based on a search query that has been input to search for the target. The program is configured to cause one or more computers to obtain target data. The target data includes target datasets. Each of the target datasets includes, as a target data field, identification information of the target and the attribute information of the target. The program is further configured to cause the one or more computers to update some of parameters included in the learning model by giving the target data to one or more training tasks and executing the one or more training tasks.

Other features and aspects will be apparent from the following detailed description, the drawings, and the claims.

Throughout the drawings and the detailed description, the same reference numerals refer to the same elements. The drawings may not be to scale, and the relative size, proportions, and depiction of elements in the drawings may be exaggerated for clarity, illustration, and convenience.

This description provides a comprehensive understanding of the methods, apparatuses, and/or systems described. Modifications and equivalents of the methods, apparatuses, and/or systems described are apparent to one of ordinary skill in the art. Sequences of operations are exemplary, and may be changed as apparent to one of ordinary skill in the art, with the exception of operations necessarily occurring in a certain order. Descriptions of functions and constructions that are well known to one of ordinary skill in the art may be omitted.

Exemplary embodiments may have different forms, and are not limited to the examples described. However, the examples described are thorough and complete, and convey the full scope of the disclosure to one of ordinary skill in the art.

In this specification, “at least one of A and B” should be understood to mean “only A, only B, or both A and B.”

Examples of an information processing device for generating a learning model, a method for generating a learning model, and a computer-readable storage medium that stores a program for generating a learning model according to the present disclosure will now be described with reference to the drawings.

Overview of System

As shown in, a systemof the present disclosure includes a search serverand an information processing device. The systemis used to generate a learning model, which will be described later. Each of the search serverand the information processing deviceis an example of a computer. The search servermay provide a website having a search window (e.g., a text box) for searching for products or services. The website may be provided by a web server that is different from the search server.

The website presents information for offering various products or services. Examples of the website include a shopping website where products are sold. Examples of the products or services offered at the website include, but are not limited to, travel plans, accommodations, transportation tickets, event tickets, books, magazines, music, videos, movies, insurance, or securities. The product or service to be searched is hereinafter referred to as a target.

One or more terminals, the search server, and the information processing devicecommunicate with each other via a network. The terminalis, for example, an information processing device such as a smartphone, a personal computer, or a tablet. The networkincludes, for example, the Internet, a wide area network (WAN), a local area network (LAN), a provider terminal, a wireless communication network, a wireless base station, and a dedicated line. All the combinations of the devices shown indo not necessarily have to communicate with one another. The networkmay partially include a local network.

After accessing a website through each terminal, a user performs search for selecting a desired target from targets by inputting a word or phrase (one or more words or phrases) in the search window. A phrase input to the search window is referred to as a search query. The search serverinfers the user's search intent from a search query and outputs a search results screen to the terminal. The search result may be, for example, displayed as a list of targets. The target displayed in the list may include, for example, a link to detailed information of each target. The search result may be a URL of a web page including information of each target. The search results screen is configured to allow the user to select a desired target from the list.

The search serveris a server device including a processor, a memory, and a communication device. The communication deviceenables communication with other devices such as the terminal, the information processing device, and a web server via the network. The memorystores a search program, target data, and search data. Further, the memorystores a learning modelobtained from the information processing device. The search dataincludes a search query that has been input by the user and attribute information of a target selected based on a search result. Details of the search datawill be described later.

The information processing deviceincludes a processor, a memory, and a communication device. The information processing deviceis, for example, a computer such as a server device. The communication deviceenables communication with other devices, for example, the terminaland the search servervia the network. The memorystores a learning programfor machine learning and a generated learning model. The processorof the information processing devicegenerates the learning modelby executing the learning program, which is stored in the memory. The learning modelinfers the user's search intent from an input search query and outputs target attribute information.

The information processing deviceobtains the target dataand the search datafrom the search serveron a regular basis, at a specific time, or in real time, and stores them in the memoryas target dataand search data. The information processing devicemay obtain each of the target dataand the search dataat a different time. The information processing devicemay obtain the target dataand the search datavia a component other than the search server, such as a computer, a server, or a storage. In order to distinguish from the target dataand the search dataupdated at any time in the search server, the data stored in the information processing deviceis referred to as the target dataand the search data.

The target dataand the search dataare used as training data for generating the learning model. All of the target dataand the search datado not have to be used as training data. Instead, some of them selected at random or under a specific condition may be used as training data.

After generating the learning model, the information processing devicemay newly obtain the target dataand the search dataas the target dataand the search data, respectively. Then, the parameters of the learning modelmay be updated by performing additional training using new target dataand new search data. In order to distinguish from the learning modelupdated in this manner, the learning model stored in the search serveris referred to as the learning model. If the learning modelis not updated, the learning modelis substantially equal to the learning model.

The processors,each include an arithmetic unit such as a CPU, a GPU, and a TPU. The processors,are processing circuitry configured to execute various software processes. The processing circuitry may include a dedicated hardware circuit (e.g. ASIC) used to process at least some of the software processing. That is, the software processing simply needs to be executed by processing circuitry that includes at least one of a set of one or more software processing circuits and a set of one or more dedicated hardware circuits.

Each of the memories,is a computer-readable medium. The memories,may each include, for example, a non-transitory storage medium such as a random access memory (RAM), a hard disk drive (HDD), a flash memory, and a read-only memory (ROM). The processors,execute a series of instructions included in the programs stored in the memories,, respectively, upon a given signal or upon satisfaction of a predetermined condition.

Search System

As shown in, the target dataincludes, for example, a catalog tableT (see) of a target. The catalog tableT ofincludes target datasets P1, P2, . . . Pt.illustrates a first target dataset P1 and a second target dataset P2.

The catalog tableT includes, as a target data field, for example, the title of each target and an attribute and an attribute value for the attribute of the target. An attribute and an attribute value for the attribute are hereinafter described as “attribute::attribute value.” The catalog tableT may further include a genre path as a target data field.

As shown in, the title may include, for example, the name, brand name, size, or color of a target, but is not limited thereto. The title may be, for example, a relatively long character string that includes all of the name, brand name, size, and color. Thus, the title may include multiple pieces of information indicating attributes (for example, a name, a brand name, a size, a color, and the like) of a target. In addition, the title may include, for example, an advertising phrase for sales promotion, which is not the attribute of a product, such as “recommended,” “free shipping,” and “10× points.” This allows the target provider to include, in the title, an advertising phrase for prompting the user to make a selection. The title may have a character limit. Since the title includes the name of a target, the title is used as identification information of the target. In addition, the target data field may include identification information (e.g., the name of each target), instead of the title of the target.

The attribute is the category or type of a target. The attribute value indicates a specific content of the attribute. Examples of the “attribute::attribute value” include, but are not limited to, “color::black,” “representative color: gray,” and “brand name::RKTN,” as shown in the first target dataset P1. Multiple attributes can be set for each target. The number of set attributes may differ depending on the target. In addition, different attributes can be set depending on the target. A pair of an attribute and an attribute value is used as attribute information.

The genre path represents the classification of multiple targets according to their types in a hierarchy. The genre path indicates levels L1 to L5 to which each target belongs. The levels L1 to L5 may be set by a website provider. In this example, up to five levels can be set, but the number of layers is not limited thereto. The first level L1 (i.e., the topmost level) is the largest classification. As the levels descend, the targets are classified in more detail.

As shown in, the order of levels can be expressed as the first level L1>the second level L2>the third level L3> . . . . The targets do not necessarily have to be set to the lowermost layer. When the genre path is set, the user can reach a desired target by following the classification in the level of layers.

For instance, in the example of the first target dataset P1, the male running wear of the target “RKTN” is set to levels including the first level L1: sports/outdoors, the second level L2::jogging/marathon, the third level L3::wear, and the fourth level L4::menswear, but is not set to the fifth level L5. In the example of the second target dataset P2 in, the target “smartphone case” is set to the first level L1::smartphone/tablet, the second level L2::smartphone/mobile phone accessories, and the third level L3::case/cover, but is not set to the fourth level L4 or the fifth level L5.

As shown in, the search dataincludes, for example, a log tableT. The log tableT ofincludes search datasets S1, S2, . . . Sq.illustrates a first search dataset S1 and a second search dataset S2.

The search serveraccumulates search logs of each search in the log tableT as search datasets S1, S2, . . . . The log tableT includes, as the search data field, an input search query and the attribute::attribute value of a target selected from the result of search based on the search query. The search dataincluding search datasets accumulated in this manner is stored in the memory. When the number (type) of targets is relatively large, or for a while after a search service is started, the number of target datasets tends to be larger than that of search datasets.

By executing the search program, the processorexecutes a search step of searching the target databased on a search query and an output step of outputting search results. By executing the search program, the processormay further execute a recording step of recording the attribute::attribute value of the target selected by the user as a search log in the memoryafter the output step.

The search results reflect the result of prediction performed by the learning model. For example, when a search query is input, the learning modelis configured to output the attribute::attribute value of a target that is highly related to the input search query as a prediction result that is consistent with the search intent. That is, the target task of each of the learning models,is to output the attribute::attribute value of the target consistent with the search intent of a search query input to search for the target.

The output of the target task of the learning models,may be an attribute value. However, even if the same attribute values is used, the attribute corresponding to its attribute value may differ depending on the type of a target. For example, if the target is aromatic oil, the attribute corresponding to the attribute value “orange” is highly likely to be “aroma.” If the target is clothing, the attribute corresponding to the attribute value “orange” is highly likely to be “color.” Thus, a result that is more consistent with a search intent can be produced by setting the output of the target task to “attribute::attribute value.” When the output of the target task of the learning models,is only an attribute value, it is desirable to be able to infer which attribute (e.g., brand name) the attribute value indicates.

Learning Model

shows a neural networkthat generates the learning model. The neural networkrefers to a learning model in which at least some of the parameters are not finally.

The neural networkincludes, for example, a pre-trained model, adapter modules, and an adapter fusion layer. The pre-trained model, the adapter modules, and the adapter fusion layerare provided, for example, via the Internet.

The pre-trained modelis, for example, Bidirectional Encoder Representations from Transformers (BERT) that is trained to understand the context of an input language.

However, the configuration of the pre-trained modelis not limited thereto. In BERT, unlabeled datasets can be processed. The pre-trained modelincludes parameters θ that are adjusted by training. The training data used in BERT pre-training is, for example, a large general-purpose corpus including unlabeled texts.

BERT includes, for example, a multi-head attention layer, a feedforward layer, and addition-and-normalization layers,,. The addition-and-normalization layers,,are located after the multi-head attention layer, the feedforward layer, and the adapter fusion layer, respectively.

In the multi-head attention layer, feature extraction is performed using an attention structure. For example, each of input words is converted into a vector representation. Then, three parameters, namely, a query (Q), a key (K), and a value (V) are calculated for each word. The query is a query in the attention mechanism, and is different from the query in the above search query. The value is the value of the key. Next, the similarity between the words is calculated from, for example, the inner product of the query and the key. Thus, the anaphoric relationship between the input words is obtained. Then, the similarity is used as a weight to output a value indicating the anaphoric relationship between the query and the value. In the multi-head attention layer, multiple patterns of relationships can be simultaneously learned by combining such attention structures in parallel.

The feedforward layeris a fully connected neural network having a two-layer structure. In the feedforward layer, first, a process is performed to weight the input, add a bias to the input, and then apply the result to an activation function (e.g., rectified linear unit (ReLU)). This output is weighted and a bias is further added. In this process, since individual forward propagation is performed for each word, parallel processing is performed without being affected by words.

Addition in the addition-and-normalization layer means residual connection. For example, the output of the layer, which is prior to the layer, is added to the output of the layer. Normalization means layer normalization. Normalization mitigates gradient vanishing and gradient exploding, thereby allowing for efficient learning.

The adapter modulesare added within the pre-trained modelfor the purpose of fine-tuning. By removing the added adapter modules, the original pre-trained modelis restored. Additionally, additional adapter modulesmay be added later, or the adapter modulesmay be replaced with other adapter modules.

As shown in, each adapter moduleincludes, for example, a feedforward down-projection (FF Down) layerand a feedforward up-projection (FF Up) layer. After the down-projection layer, processing may be performed using an activation function (e.g., ReLU).