Keyword Bids Determined from Sparse Data

2. A method as described in claim 1, further comprising receiving, from the search engine platform, data describing outcomes of submitting the bids for the low-impression keywords.

3. A method as described in claim 2, further comprising updating the first model according to the first algorithm in real-time as the data describing the outcomes is received from the search engine platform.

4. A method as described in claim 3, further comprising generating additional bids for the low-impression keywords using the first model as updated.

5. A method as described in claim 3, wherein the updating includes tuning, by the model trainer, at least one parameter of the first model.

7. A method as described in claim 6, wherein the simulating is performed offline from the search engine platform.

8. The method of claim 1, further comprising determining the training bids based on a reward determined for a training bid as a function of a state associated with the training bid and an action performed relative to a previous training bid to produce the training bid.

9. The method of claim 8, wherein the training further includes determining a policy by mapping the state associated with the training bids to the action to maximize the reward.

10. The method of claim 9, wherein the determining is based on a Q-value approach.

11. The method of claim 8, wherein the action is one of increasing or decreasing the previous training bid to produce the training bid.

12. The method of claim 8, wherein the state is a combination of historical data associated with the training bid and semantic similarity of keywords associated with the training bid.

13. The method of claim 12, further comprising determining the semantic similarity of keywords based on embeddings of the keywords.

14. The method of claim 12, further comprising associating behavior of the search engine simulator in relation to training bids for at least one keyword with training bids for at least one other keyword based on the semantic similarity, the behavior of the search engine simulator being associated with the at least one other keyword using the state.

15. The method of claim 8, further comprising updating the state associated with the training bid based on behavior of the search engine simulator.

16. The method of claim 15, wherein updating the state includes propagating, to the state, an achieved reward from the training bid as adjusted according to the action, the achieved reward corresponding to the behavior of the search engine simulator in response to the training bid as adjusted.

18. A system as described in claim 17, wherein the reinforcement learning based bidder is further configured to submit the bids for the low-impression keywords to the search engine platform.

19. A system as described in claim 18, further comprising a model trainer to update the first machine learning model based on the first algorithm, the model trainer configured to update the first machine learning model, in part, by tuning one or more parameters of the first machine learning model in real-time as data describing outcomes of submitting the bids for the low-impression keywords is received from the search engine platform.