Sentiment Analysis Using Bag-Of-Phrases for Arabic Text Dialects

1. A method for analyzing sentiment in text data, comprising: receiving and storing training data comprising a plurality of words; compiling the training data and removing one or more numbers, control characters, and graphics; identifying one or more same letters within the training data, wherein the same letters comprise letters with a same meaning and with a different format; identifying from the one or more same letters, and one or more same letters with a different format with a different meaning; unifying the one or more letters with a same meaning and a different formats; labeling, by an annotator, one or more phrases from the training data as positive, negative, or neutral; forming a lexicon based on the labeled one or more phrases, wherein the lexicon comprises a plurality of phrases and words labeled by the annotator, the lexicon comprising a set of positive words and phrases and a set of negative words and phrases; receiving a set of target data, the target data comprising target text data; forming a bag-of-phrases from the target data, the bag-of-phrases identifying a plurality of matched words or phrases from the target data, the matched words or phrases comprising words or phrases matching one or more words or phrases from the set of positive words and phrases and/or the set of negative words and phrases from the lexicon; identifying at least one sentiment associated with one or more portions of the target data based on the matched words or phrases.

2. The method for analyzing sentiment in text data of claim 1 , wherein the training data comprises Arabic words.

3. The method for analyzing sentiment in text data of claim 2 , wherein the removing comprises removing one or more of a hamza, dots, a Madod, or a duplicate letter.

4. The method for analyzing sentiment in text data of claim 1 , wherein the labeling is executed by the annotator while labeling the one or more phrases.

5. The method for analyzing sentiment in text data of claim 1 , wherein the annotator is a machine learning algorithm.

6. The method for analyzing sentiment in text data of claim 1 , wherein the annotator is a human user.

7. The method for analyzing sentiment in text data of claim 1 , further comprising identifying one or more of a geographic location, culture, or topic associated with each portion of the data, and wherein the labeling of the training data is based on the identified geographic location, culture, or topic.

8. The method for analyzing sentiment in text data of claim 1 , wherein the sentiment is identified based on a quantity of positive words and phrases from the matched words and phrases or a quantity of negative words and phrases from the matched words and phrases.

9. The method for analyzing sentiment in text data of claim 1 , further comprising forming a document-term matrix comprising one or more phrases from the target data, an identification of a plurality of documents where the one or more phrases are found, and a frequency of the one or more phrases in each of the plurality of documents.