Systems and methods for active learning of statistical machine translation systems through dynamic creation and updating of parallel corpus. The systems and methods provided create accurate parallel corpus entries from a test set of sentences, words, phrases, etc. by calculating confidence scores for particular translations. Translations with high confidence scores are added directly to the corpus and the translations with low confidence scores are presented to human translations for corrections.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method for creating or updating parallel corpus in a machine translation system, comprising the steps of: without parallel corpus, translating a test set E from a first collection to a second collection so as to create a set F in the second collection, and translating the set F from the second collection to the first collection so as to create a set E′ in the first collection, wherein differences between E and E′ are determined; computing confidence scores for a translation of each item in the test set E based on a similarity of E and E′; and based on the confidence scores, adding translations to the parallel corpus, wherein the parallel corpus is stored in memory on the machine translation system.
A method for improving machine translation quality by creating or updating a parallel corpus (a collection of translated text pairs). The method translates a test set of sentences from one language to another, and then translates the result back to the original language. Differences between the original and the back-translated text are analyzed. Confidence scores are calculated for each translation based on the similarity between the original and back-translated sentences. Translations with high confidence scores are automatically added to the parallel corpus, which is stored in memory on the machine translation system, to improve future translations.
2. The method of claim 1 , further comprising preparing the test set E to be updated by presenting a Graphical User Interface (GUI) to a user, the GUI displaying at least controls for changing features to compute the confidence scores, the features being at least one of a scoring metric and values used to create two subsets, one comprising the highest confidence scores computed and the other comprising the lowest confidence scores computed.
The method for improving machine translation quality described in Claim 1 further involves a user interface (GUI) that allows users to adjust the criteria for calculating confidence scores. The GUI includes controls for modifying scoring metrics (e.g., how translation quality is measured) and the threshold values used to define "high" and "low" confidence translations. This allows users to fine-tune the active learning process by controlling which translations are automatically added to the parallel corpus or flagged for human review.
3. The method of claim 1 , further comprising the steps of: creating a subset H of the highest confidence scores; adding the translations in the subset H directly to the parallel corpus; creating a subset L of lowest confidence scores; presenting the subset L to human translators for correction; and adding human corrections to the parallel corpus.
The method for improving machine translation quality described in Claim 1 further incorporates human feedback. It involves creating a subset of the translations with the highest confidence scores, which are automatically added to the parallel corpus. Another subset of translations with the lowest confidence scores is presented to human translators for correction. The corrected translations from the human translators are then added to the parallel corpus, improving the overall quality of the machine translation system.
4. The method of claim 3 , wherein the step of presenting the subset L to human translators for correction comprises presenting a Graphical User Interface (GUI) to the translator, the GUI providing at least the items in subset L, a window to make translation corrections and an update button.
The method for improving machine translation quality described in Claim 3 presents the low-confidence translations to human translators through a graphical user interface (GUI). The GUI displays the original text and its machine translation, providing a text box where the translator can correct the translation. An "update" button allows the translator to submit the corrected translation, which is then incorporated into the parallel corpus. This provides a user-friendly way for translators to contribute to the active learning process.
5. The method of claim 1 , further comprising preparing the test set E to be updated by: translating a test set G using an existing parallel corpus; calculating a translation accuracy score for one or more items in the test set G; comparing the translation accuracy score for each item to a desired performance score to determine whether the parallel corpus needs to be updated for that item; and if the translation accuracy score for an item is equal to or greater than a desired performance score, removing that item from the test set G, so as to create the test set E.
The method for improving machine translation quality described in Claim 1 refines the test set used for active learning. Initially, a larger test set is translated using an existing parallel corpus. A translation accuracy score is calculated for each translated item. If an item's accuracy score meets or exceeds a desired performance threshold, it's removed from the test set. The remaining items, forming the test set 'E', are then subjected to the back-translation and confidence scoring process to further update the parallel corpus. This focuses the active learning on areas where the machine translation system struggles most.
6. The method of claim 1 , wherein the confidence scores are computed using at least one of Bilingual Evaluation Understudy or Translation Edit Rate scoring metrics.
In the method for improving machine translation quality described in Claim 1, the confidence scores assigned to translations are calculated using established metrics like Bilingual Evaluation Understudy (BLEU) or Translation Edit Rate (TER). These metrics quantitatively assess the similarity between the original and back-translated sentences, providing a numerical basis for determining the reliability of the machine translation and guiding the selection of translations for inclusion in the parallel corpus.
7. The method of claim 1 , wherein the confidence scores are computed further by using phase posterior probabilities in n-best hypotheses.
In the method for improving machine translation quality described in Claim 1, the confidence scores are computed using phrase posterior probabilities derived from n-best hypotheses generated by the machine translation system. This means the system considers multiple possible translations (the "n-best" hypotheses) and uses the probabilities associated with different phrases within those hypotheses to refine the confidence score for the overall translation, improving accuracy.
8. A computer readable storage medium comprising a computer readable program, wherein the computer readable program when executed on a computer causes the computer to perform the method steps of claim 1 .
A computer-readable storage medium (e.g., a hard drive, SSD, or USB drive) contains a program that, when executed by a computer, performs the method steps for improving machine translation quality by creating or updating a parallel corpus. The method involves translating a test set from one language to another and back, computing confidence scores based on similarity, and adding translations to the parallel corpus, as described in Claim 1.
9. A method for training a machine translation system, comprising the steps of: translating a test set from a first collection to a second collection using an existing parallel corpus stored in memory on the machine translation system; calculating a translation accuracy score for each item in the test set; comparing the translation accuracy score for each item to a desired performance score to determine whether the parallel corpus needs to be updated for that item; if the translation accuracy score for an item is equal to or greater than a desired performance score, removing that item from the test set, so as to create a test set E; using a unidirectional translation corpus, translating the test set E from the first collection to the second collection so as to create a set F in the second collection and translating the set F from the second collection back to the first collection so as to create a set E′ in the first collection, wherein differences between E and E′ are determined; computing confidence scores for a translation of each item in the test set E based on a similarity of E and E′; and adding translations to the parallel corpus based on the confidence scores.
A method for training a machine translation system uses an existing parallel corpus to translate a test set of sentences from one language to another. A translation accuracy score is calculated for each sentence. Sentences that meet or exceed a desired performance score are removed from the test set, creating a refined test set E. This refined test set is then translated back and forth between the languages using a unidirectional translation corpus, and differences are determined. Confidence scores are computed based on the similarity between the original and back-translated sentences. Finally, translations are added to the parallel corpus based on these confidence scores, improving the system's performance.
10. The method of claim 9 , wherein the method steps further comprise: creating a subset H of highest confidence scores; adding the translations in subset H directly to the parallel corpus; creating a subset L of lowest confidence scores; presenting subset L to human translators for correction; and adding human corrections to the parallel corpus.
The method for training a machine translation system described in Claim 9 further refines the parallel corpus using human feedback. Sentences with the highest confidence scores are automatically added to the parallel corpus. Sentences with the lowest confidence scores are presented to human translators for correction. The human-corrected translations are then added to the parallel corpus, further enhancing the system's translation capabilities.
11. The method of claim 10 , wherein the step of presenting the subset L to human translators for correction comprises presenting a Graphical User Interface (GUI) to the translator, the GUI providing at least the items in subset L, a window to make translation corrections, and an update button.
In the method for training a machine translation system described in Claim 10, the low-confidence translations are presented to human translators through a user-friendly graphical interface (GUI). The GUI displays the original sentence, the machine translation, a text box for corrections, and an "update" button to submit the corrected translation to the parallel corpus. This streamlines the human correction process.
12. The method of claim 9 , wherein the scoring metric and threshold values used to compute the confidence scores are defined by the user.
In the method for training a machine translation system described in Claim 9, the scoring metric and threshold values used to compute the confidence scores are user-defined. This means the user has control over which measures of translation quality are most important and how accurate a translation needs to be before it is automatically accepted or flagged for human review.
13. The method of claim 9 , further comprising the step of presenting a Graphical User Interface (GUI) to a user, the GUI displaying at least controls for changing features to compute the confidence scores, the features being at least one of a scoring metric and values used to create two subsets, one subset comprising highest confidence scores computed and the other subset comprising lowest confidence scores computed.
The method for training a machine translation system described in Claim 9 includes a graphical user interface (GUI) that allows users to adjust the criteria for calculating confidence scores. The GUI includes controls for modifying the scoring metrics used to evaluate translations and the threshold values used to define the subsets of high and low confidence translations. This allows the user to customize the active learning process.
14. The method of claim 9 , wherein the translation accuracy score is measured using Bilingual Evaluation Understudy scoring metrics.
In the method for training a machine translation system described in Claim 9, the translation accuracy score, used to determine which sentences to remove from the test set, is measured using Bilingual Evaluation Understudy (BLEU) scoring metrics. This provides a standardized and widely accepted way to evaluate the quality of the machine translation.
15. A computer readable storage medium comprising a computer readable program, wherein the computer readable program when executed on a computer causes the computer to perform the steps of claim 9 .
A computer-readable storage medium (e.g., a hard drive, SSD, or USB drive) contains a program that, when executed by a computer, performs the method steps for training a machine translation system as described in Claim 9. The method includes using an existing parallel corpus to translate a test set, calculating accuracy scores, removing high-scoring items, translating back and forth, computing confidence scores, and adding translations to the parallel corpus.
16. An active learning training system that a user can interact with, comprising: a translation module for translating a test set E from a first collection to a second collection using a unidirectional translation corpus and from the second collection to the first collection using a unidirectional translation corpus so as to create a set E′ in the first collection, wherein differences between E and E′ are determined; a comparison module for computing confidence scores for a translation of each item in the test set E based on a similarity of E and E′ and adding translations to a parallel corpus based on the confidence scores; and memory storage for storing a created or updated parallel corpus.
An active learning system for machine translation training allows a user to interact with and improve a translation model. The system includes a translation module that translates sentences between two languages, then back-translates to detect differences. A comparison module calculates confidence scores for each translation based on similarity between the original and back-translated versions, adding the translations to a parallel corpus based on these scores. The system stores the resulting updated parallel corpus in memory.
17. The system as recited in claim 16 , further comprising a connection port configured for communications with a machine translation device.
The active learning system described in Claim 16 also includes a connection port for communicating with a machine translation device. This allows the system to directly update the parallel corpus used by the machine translation device, improving its translation accuracy in real-time.
18. The system as recited in claim 16 , further comprising: long-term memory storage for a plurality of parallel corpuses; and a translation module capable of performing translations.
The active learning system described in Claim 16 includes long-term memory for storing multiple parallel corpuses and a translation module capable of performing translations between different language pairs. This enables the system to manage and improve translations for multiple language pairs simultaneously.
19. The system as recited in claim 16 , further comprising a user interface that includes a Graphical User Interface displaying at least controls for changing features to compute the confidence scores, the features being at least one of a scoring metric and values used to create subsets of low and high confidence machine translations.
The active learning system described in Claim 16 includes a user interface with a graphical user interface (GUI) that allows the user to control features for computing confidence scores. The GUI includes controls for changing the scoring metric used to evaluate translations and the values used to define the subsets of low and high confidence translations, allowing the user to fine-tune the active learning process.
20. The system as recited in claim 16 , further comprising a user interface that includes a Graphical User Interface (GUI) to a translator, the GUI providing at least items in a subset of low confidence machine translations, a window to make translation corrections, and an update button.
The active learning system described in Claim 16 includes a user interface designed for human translators. The GUI displays a subset of low-confidence machine translations, a window for making corrections, and an update button to submit the corrected translations to the parallel corpus. This facilitates human-in-the-loop correction to improve translation quality.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
August 14, 2012
July 23, 2013
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.