E is for Entity Recognition

From A to I to Z: Jaid’s Guide to Artificial Intelligence

Entity recognition is the process that enables AI to accurately grasp the meaning of text. 

The AI goes through the text, identifies specific words, and sorts them into predetermined categories: names, titles (for instance, Dr, Captain, or President), places, dates, times, topics, or any other category that is useful to it. 

Once it has recognized each part of the text and, so, made sense of the whole, it’s able to respond appropriately. 

The human brain processes information in a similar way. 

Say you’re looking at a photo. 

Your brain would identify and label the various elements that make it up: the cupboards in your parents’ kitchen, Mum, Dad, you, your brother, your sister, Patches the Jack Russell Terrier, the clothes you’re wearing, and the roast turkey on the serving plate. 

Taken together, these elements make it possible for you to understand what you’re looking at: a family photo taken at Thanksgiving.

Entity recognition has a wide range of practical applications. These include:

• Extracting specific information from unstructured data
• Summarizing text
• Identifying suspicious patterns, such as unusual activity on a credit card
• Helping AI chatbots understand what customers want and formulating the right response
• Translating from one language into another

Some facts:

Entity recognition typically happens in three stages. 

First, a tokenizer breaks the text down into individual words, or ‘tokens’. Tokenizers can break text into sentences, words, or smaller units called ‘subwords’. The latter subdivision is useful for identifying unusual or made-up words. 

Next, a part-of-speech tagger labels each token with its part of speech: noun, pronoun, verb, adjective, adverb, preposition, conjunction, or interjection. 

Finally, a named entity recognizer identifies and classifies named entities — specific words or phrases like people’s names, organizations’ names, product names, locations, and the titles of books, movies, or songs.

Tokenizers, part-of-speech taggers, and named entity recognizers are algorithms or pieces of software. They’re typically either rules-based or machine learning-based. 

Rules-based systems decide how to tokenize, tag, or recognize named entities using a set of predetermined rules. By contrast, machine learning-based systems are trained to predict which token, tag, or category should apply. 

While entity recognition is most commonly used to process written language, it can be used to make sense of any type of information, including code, images, video, and audio. 

Case in point, archaeologists have used entity recognition to interpret hieroglyphics — Ancient Egyptian script. And music streaming services like Spotify use it to recommend artists and songs you might like based on what you’ve listened to in the past.  

Want to know more?

Want to see the entity recognition process in action? Aside from providing a comprehensive, non-technical rundown of how entity recognition works, this article also has a tool you can experiment with. Input your own text, and an AI will extract named entities and tag them.

If you don’t mind getting a bit more technical, this Cornell University paper summarizes the latest thinking around entity recognition and examines the trends shaping where it’ll go next. 

Jaid’s perspective

Entity recognition is one of the cornerstones of AI, because it enables machines to make sense of any kind of data, even if it’s unstructured. From a customer services perspective, entity recognition makes it possible for AI to address complex queries even if they don’t follow a predetermined format. This means consumers can get in touch using their preferred communication channel — whether that’s email, text, or phone — and AI will be able to quickly assess their needs and formulate a satisfying response.

Let us show you how entity recognition makes it possible for Jaid’s AI platform to address your customer queries, allowing your service teams to focus on relationships rather than mundane tasks. Contact us today for a free demo!