From A to I to Z: Jaid’s Guide to Artificial Intelligence

Ever notice how you can discern a friend or loved one’s emotional state from the tone of their voice or the way they’ve structured their sentences?

Sentiment analysis is the AI equivalent. 

In sentiment analysis, AI is trained using machine learning techniques to identify whether speech or text is positive, negative, or neutral — or, in more advanced models, the degree to which it is positive, negative, or neutral. 

This enables the AI to better understand the meaning of that speech or text, and to formulate a more appropriate and satisfying response. 

Sentiment analysis training usually takes place in four stages. 

First, the text to be analyzed is split into individual words or phrases. 

Sometimes, stop words — words that have a function but not much meaning, like a, an, the, in, and or — are removed. Words may also be reduced to their root forms. For example, a verb in the present continuous (such as running), might be converted into its infinitive (run). Or a plural (people) might be converted into the singular (person).  

Second, the text goes through a process called feature extraction. 

Put simply, this involves sorting the text into categories to make it easier for AI to determine its sentiment. For example, words might be labeled based on their role in a sentence, their theme, or whether they’re place names, people’s names, or other proper nouns.

Third, AI is trained to identify sentiment. This can be done either through supervised or unsupervised learning. 

In supervised learning, the AI is fed text where sentiment has already been identified. Its job is to try its hand at identifying sentiment, then comparing its results to what the trainers have supplied. 

In unsupervised learning, the AI tries to figure out the sentiment of speech or text on its own. 

The fourth and last step involves analyzing the results and using them to optimize AI and improve its performance.

Some facts

Scientists have been interested in sentiment analysis for as long as they’ve been experimenting with AI. But research in the area exploded in the early 2000s with the proliferation of online forums, social media networking sites, and other user-generated online content. 

Since then, researchers have been applying sentiment analysis to a huge range of use cases — from understanding how consumers perceive particular brands to identifying potential health concerns in society at large, and predicting the outcome of elections. 

One of the biggest breakthroughs in sentiment analysis was BERT — a machine learning framework created by Google and open-sourced in 2018. 

BERT can process language bidirectionally — from right to left and left to right simultaneously. This makes it possible for AI to understand unclear and ambiguous words from the surrounding context and assess the writer or speaker’s emotional state more accurately.

While sentiment analysis has advanced by leaps and bounds, it still has some significant limitations. In particular, because AI tends to interpret language literally, it struggles with nuanced sentiments like sarcasm and irony, where the true meaning is often the opposite of what is being said. 

Want to know more?

This research study, in which scientists analyzed 7.2 million tweets to predict the most likely outcome of the 2020 US presidential election, is a fascinating look at the inner workings of sentiment analysis. 

Fancy learning how to train AI to do sentiment analysis? This course on Coursera walks you through the process of building a sentiment analysis model using BERT.

Jaid’s perspective

“Sentiment analysis enables AI to contextualize customer inquiries and, so, become more effective at addressing them successfully. If the AI can tell that a customer is angry, for instance, it can de-escalate. Similarly, if the speaker sounds satisfied, the AI learns that its response is the correct approach to this kind of inquiry, and it can apply this knowledge to address similar inquiries more efficiently in future.“

Discover how Jaid’s AI-powered platform is increasing resolution rates and customer satisfaction via sentiment analysis – contact us today for a free demo!