An apparatus and a method are provided for building a spoken language understanding model. Labeled data may be obtained for a target application. A new classification model may be formed for use with the target application by using the labeled data for adaptation of an existing classification model. In some implementations, the existing classification model may be used to determine the most informative examples to label.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method comprising: receiving a first language model for a first domain having a first training set; based on a second language model, which is smaller than the first language model, selectively sampling training data associated with a second domain from a second training set, to yield sampled training data; labeling, via a processor, the sampled training data, to yield second domain labeled data; and concatenating, via the processor, the first language model by replacing a portion of the first training set in the first language model with the second domain labeled data.
2. The method of claim 1 , wherein concatenating the first language model further comprises: determining a distance from the first language model to the second domain labeled data; and modifying the first language model using the distance.
3. The method of claim 2 , wherein the first language model is a speech processing model and wherein the distance comprises a logistic loss function.
4. The method of claim 1 , further comprising: obtaining confidence scores from the first language model and a modified first language model; and engaging in active learning using the confidence scores.
5. The method of claim 1 , wherein modifying the first language model utilizes one of a Boosting algorithm, a Naïve Bayes classifier, a linear model interpolation, and a Bayesian adaptation.
6. The method of claim 5 , wherein probability distributions correspond to an existing language model probability distribution and a modified language model probability distribution.
7. The method of claim 1 , further comprising labeling future utterances using a modified first language model.
8. A system comprising: a processor; and a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising: receiving a first language model for a first domain having a first training set; based on a second language model, which is smaller than the first language model, selectively sampling training data associated with a second domain from a second training set, to yield sampled training data; labeling the sampled training data, to yield second domain labeled data; and concatenating the first language model by replacing a portion of the first training set in the first language model with the second domain labeled data.
9. The system of claim 8 , wherein concatenating the first language model further comprises: determining a distance from the first language model to the second domain labeled data; and modifying the first language model using the distance.
10. The system of claim 9 , wherein the first language model is a speech processing model and wherein the distance comprises a logistic loss function.
11. The system of claim 8 , the computer-readable storage medium having additional instructions stored which result in the operations further comprising: obtaining confidence scores from the first language model and a modified first language model; and engaging in active learning using the confidence scores.
12. The system of claim 8 , wherein modifying the first language model utilizes one of a Boosting algorithm, a Naïve Bayes classifier, a linear model interpolation, and a Bayesian adaptation.
13. The system of claim 12 , wherein probability distributions correspond to an existing language model probability distribution and a modified language model probability distribution.
14. The system of claim 8 , the computer-readable storage medium having additional instructions stored which result in the operations further comprising labeling future utterances using a modified first language model.
15. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising: receiving a first language model for a first domain having a first training set; based on a second language model, which is smaller than the first language model, selectively sampling training data associated with a second domain from a second training set, to yield sampled training data; labeling the sampled training data, to yield second domain labeled data; and concatenating the first language model by replacing a portion of the first training set in the first language model with the second domain labeled data.
16. The computer-readable storage device of claim 15 , wherein concatenating the existing language model further comprises: determining a distance from the first language model to the second domain labeled data; and modifying the first language model using the distance.
17. The computer-readable storage device of claim 16 , wherein the first language model is a speech processing model and wherein the distance comprises a logistic loss function.
18. The computer-readable storage device of claim 15 , the computer-readable storage medium having additional instructions stored which result in the operations further comprising: obtaining confidence scores from the first language model and a modified first language model; and engaging in active learning using the confidence scores.
19. The computer-readable storage device of claim 15 , wherein modifying the first language model utilizes one of a Boosting algorithm, a Naïve Bayes classifier, a linear model interpolation, and a Bayesian adaptation.
20. The computer-readable storage device of claim 19 , wherein probability distributions correspond to an existing language model probability distribution and a modified language model probability distribution.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
August 8, 2011
May 20, 2014
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.