← Back to Documentation

Augmenting Language Models with Knowledge

Use Operand search results to augment GPT-3 style prompts with additional, context-relevant knowledge. Help your pre-trained language models learn and get better over time, without having to do any additional training.


Large language models (LLMs), like GPT-3, are getting to the point where only a few select companies can afford to reliably train and deploy them.

The latest model from OpenAI is trained on data up until Jun 2021. If you wanted to use this model on per-user data, or data that's been collected since Jun 2021, you'd have to retrain the model on this new data (which is expensive). The best alternative is using semantic search to augment the model with additional, context-relevant knowledge.

Case Study: Leveraging Wikipedia to give GPT-3 Factual, Up-To-Date Knowledge on a Topic

Riff allows anyone to have virtual "office hours" with an expert on a given topic. For example, if you wanted to learn about cellular biology, you could ask a virtual professor of cellular biology questions about the topic.

To accomplish this, Riff uses a combination of OpenAI's GPT-3 and Operand.

When the user asks a question, Riff uses a GPT-3 prompt to choose which Wikipedia article would be most relevant to the question. For example, if the user asks "What is the difference between a virus and a bacterium?", Riff might choose the virus article.

Riff indexes one or more relevant Wikipedia articles with Operand, and then uses search results to pass context relevant knowledge into a conversational GPT-3 prompt. For example, if the user asks "What is the difference between a virus and a bacterium?", Riff might use the following prompt:

Alice is a professor of cellular biology. She's holding office hours.

Relevant information Alice knows:
- Bacteriophages are a common and diverse group of viruses and are the most abundant biological entity in aquatic environments—there are up to ten times more of these viruses in the oceans than there are bacteria, [223] reaching levels of 250,000,000 bacteriophages per millilitre of seawater. [224] These viruses infect specific bacteria by binding to surface receptor molecules and then entering the cell.
- Viruses display a wide diversity of sizes and shapes, called ' morphologies '. In general, viruses are much smaller than bacteria and more than a thousand bacteriophage viruses would fit inside an Escherichia coli bacterium's cell. [47] Many viruses that have been studied are spherical and have a diameter between 20 and 300 nanometres.
- A virus is a submicroscopic infectious agent that replicates only inside the living cells of an organism . [1] Viruses infect all life forms , from animals and plants to microorganisms , including bacteria and archaea.

Student: Hello!
Alice: Hello! How can I help?
Student: What is the difference between a virus and a bacterium?
Alice:

Notice how this prompt is "augmented" with snippets pulled directly from the Wikipedia article. This is a great way to give GPT-3 factual, up-to-date knowledge on a topic. In this case, the prompt is augmented with search results from Operand, after indexing the article.

When executed, Alice responds with:

Viruses are much smaller than bacteria and more than a thousand bacteriophage viruses would fit inside an Escherichia coli bacterium's cell. Viruses also infect all life forms, from animals and plants to microorganisms, including bacteria and archaea.

Case Study: Adding Long-Term Memory to Chatbots

When using GPT-3 to create conversational experiences, it's often important to develop a long-term relationship with the end-user. For example, as a user continues to converse with the chatbot, the chatbot should remember information that the user has told it. This is called "long-term memory".

Long-term memory is a key component of human-like conversational experiences, and it's something that makes chatbots feel more "human".

Circle allows anyone to create virtual AI-powered NPC characters, with any personality and backstory they want. Each Circle chatbot comes equipped with long-term memory, powered by Operand.

For each conversation between a user and a chatbot, Circle creates a corresponding Operand index, which stores the messages sent to and from the chatbot.

When the user sends a message to the chatbot, Circle first does a search to find relevant messages from the conversation history. Circle then uses these messages to augment the GPT-3 prompt, to help the chatbot remember the conversation history.

An example prompt could be:

The following is a conversation with an AI assistant. The assistant is helpful, creative, clever, and very friendly.

Previously, the Human has told the AI that:
- My favorite drink at Starbucks is a Pumpkin Spice Latte.

Human: Hello, who are you?
AI: I am an AI created by OpenAI. How can I help you today?
Human: How basic am I?
AI:

Using the prompt above, the chatbot can respond with:

You are very basic.

Conclusion

Augmenting large-language models with semantic search results is a great way to leverage the latest, large models without having to retrain them on your own data, or even on a per-user basis. We've discussed a few use cases above, but there are many, many more. If you have any questions, or curious about how this technique could help out with your unique use case, reach out!

If you want to start augmenting large-language models today, follow our getting started guide. It's entirely free to get going, no credit card required!