G is for GPT

From A to I to Z: Jaid’s Guide to Artificial Intelligence

GPT — Generative Pre-Trained Transformer — is a language model that can interpret text, understand its context, and formulate natural-sounding, human-like responses.

GPT will probably be familiar to you as the last three letters in ChatGPT — the chatbot that has taken the world by storm. But chatbots are only one possible practical application of this language model. Other use cases include:

• Translating text from one language to another
• Summarizing longer texts
• Generating original text in a specific style, based on human input
• Answering customers’ questions

GPT is trained on a vast amount of books, newspapers, journals, website content, and other texts using a technique called unsupervised learning, in which humans provide inputs but no outputs.

The input dataset is unstructured. In other words, it’s not organized in any way. Once the AI receives it, its job is to figure out the patterns and relationships in that dataset. This typically involves one of the following:

• Sorting the data into meaningful categories — what is known as clustering. So, if the model is being trained for a customer services use case, for instance, this could involve sorting new inquiries from follow-ups to an already open inquiry.
• Categorizing the data, which is known as dimensionality reduction. Here, the goal is to remove irrelevant data from the dataset to improve accuracy.

Once GPT has been trained, it can be fine-tuned through exposure to a smaller, task-specific dataset using a technique called transfer learning.

Transfer learning is the AI equivalent of learning from past experience. Either the features or the parameters learned during initial training are adjusted so GPT can perform the new, specific task.

Some facts:

The latest version of GPT — GPT-3 — has 175 billion parameters, making it one of the most advanced language models in the world. The sheer number of parameters means fine-tuning it requires huge amounts of computing power and expertise.

As powerful as GPT-3 is, it still has some significant limitations. In particular:

• It can only generate outputs that are similar to the data it’s been trained on, and can’t make new connections or come up with new ideas
• It’s unable to grasp certain nuances in language — what we humans call “common sense”
• Because its outputs are based on its inputs, they may reflect certain biases in that data, including unacceptable ones like racism

GPT is based on Google’s Transformers architecture, which means it can process data in snippets of varying size, in any order, and, so better grasp context, sentiment, and ambiguity.

Other language models that are based on this technology include T5, BERT, and RoBERTa, a version of BERT trained on a 50% larger dataset using a technique called dynamic masking.

Want to know more?

Despite its largely positive reception, GPT-3 — and ChatGPT in particular — raises important ethical concerns. This article discusses the biggest issue: the risk that disinformation could spread on an unprecedented scale.

This is the paper that introduced the world to transformer architecture. While a lot of it is quite technical, it’s worth reading just to get a sense of what a quantum leap this development was for AI. Transformer architecture, the paper notes, outperformed all its predecessors and could be trained significantly faster.

Jaid’s perspective

With its uncannily human fluency and coherence, GPT-3 is proof positive of AI’s huge potential. Customer service representatives no longer have to spend their workdays addressing run-of-the-mill queries or doing other busywork. Instead, AI can handle these for them in a way that is satisfying to customers, freeing them up to focus on business-critical tasks like ensuring clients receive excellent service. It’s now up to us to harness this potential into a force for good and ensure we tackle the risks and ethical issues that AI raises head on.

Contact us today to learn more about Jaid and how our AI-powered platform streamlines customer service, enabling teams to resolve cases up to 95% faster while delivering exceptional, personalized service.