Techniques for using logit values for classifying utterances and messages input to chatbot systems in natural language processing. A method can include a chatbot system receiving an utterance generated by a user interacting with the chatbot system. The chatbot system can input the utterance into a machine-learning model including a set of binary classifiers. Each binary classifier of the set of binary classifiers can be associated with a modified logit function. The method can also include the machine-learning model using the modified logit function to generate a set of distance-based logit values for the utterance. The method can also include the machine-learning model applying an enhanced activation function to the set of distance-based logit values to generate a predicted output. The method can also include the chatbot system classifying, based on the predicted output, the utterance as being associated with the particular class.
Legal claims defining the scope of protection, as filed with the USPTO.
2. The method of claim 1, further comprising responding, by the chatbot system, to the user based on a classification of the utterance as being associated with the particular class.
3. The method of claim 1, further comprising training the machine-learning model by applying an enhanced loss function to the predicted output and an expected output corresponding to the utterance to determine a total loss, wherein the total loss is used to adjust one or more parameters of the machine-learning model, wherein the enhanced loss function includes a set of loss terms for determining the total loss, and wherein the set of loss terms include: (i) a binary cross-entropy loss term; (ii) a mean-square error term; (iii) a margin loss term; and (iv) a threshold loss term.
4. The method of claim 3, wherein each loss term of the set of loss terms is associated with a weight parameter, and wherein the training of the machine-learning model includes adjusting, based on the total loss, the weight parameter of a loss term of the set of loss terms.
5. The method of claim 3, wherein the training of the machine-learning model includes adjusting the learned parameter of the enhanced activation function based on the total loss.
6. The method of claim 3, wherein the margin loss term identifies a minimum confidence margin of 0.1.
7. The method of claim 3, wherein the threshold loss term identifies a minimum threshold confidence of 0.5.
8. The method of claim 1, wherein the distance measured between the probability for the class and the centroid of the distribution associated with the class is one of an Euclidean distance or a cosine distance.
12. The system of claim 11, wherein each loss term of the set of loss terms is associated with a weight parameter, and wherein the training of the machine-learning model includes adjusting, based on the total loss, the weight parameter of a loss term of the set of loss terms.
13. The system of claim 11, wherein the training of the machine-learning model includes adjusting the learned parameter of the enhanced activation function based on the total loss.
14. The system of claim 11, wherein the margin loss term identifies a minimum confidence margin of 0.1.
15. The system of claim 11, wherein the threshold loss term identifies a minimum threshold confidence of 0.5.
16. The system of claim 9, wherein the distance measured between the probability for the class and the centroid of the distribution associated with the class is one of an Euclidean distance or a cosine distance.
20. The computer-program product of claim 19, wherein each loss term of the set of loss terms is associated with a weight parameter, and wherein the training of the machine-learning model includes adjusting, based on the total loss, the weight parameter of a loss term of the set of loss terms.
21. The computer-program product of claim 19, wherein the training of the machine-learning model includes adjusting the learned parameter of the enhanced activation function based on the total loss.
22. The computer-program product of claim 19, wherein the margin loss term identifies a minimum confidence margin of 0.1.
23. The computer-program product of claim 19, wherein the threshold loss term identifies a minimum threshold confidence of 0.5.
24. The computer-program product of claim 17, wherein the distance measured between the probability for the class and the centroid of the distribution associated with the class is one of an Euclidean distance or a cosine distance.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
November 30, 2021
June 25, 2024
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.