What is a Large Language Model (LLM)?

Imagine you could talk to a computer program, ask it questions, have it write a story, summarize a long document, or even translate languages, and it would understand and respond in a way that sounds remarkably human. That's the essence of a Large Language Model, or LLM.

At its core, an LLM is a type of artificial intelligence (AI) program specifically designed to understand, process, and generate human language (text). Think of it as an incredibly sophisticated pattern-matching machine. It has been trained on enormous amounts of text data, think websites like Wikipedia, extensive collections of books, articles, and other text sources from the internet.

Why "Large"? The term "Large" primarily refers to two aspects:

The immense amount of data it was trained on. This extensive training allows the model to learn grammar, facts, reasoning abilities, different writing styles, and nuances of language.
The sheer number of internal "parameters" it uses to store and process this learned information. We'll explore parameters in detail in the next section, but for now, understand that more parameters generally allow a model to capture more complex patterns and knowledge.

LLMs don't "understand" language in the human sense of consciousness or experience. Instead, they learn statistical relationships between words and concepts. When you give an LLM a prompt (a piece of text input), it predicts the most probable sequence of words to follow, based on the patterns it learned during training. This prediction process allows it to perform tasks like:

Generating Text: Writing emails, code, poems, marketing copy, etc.
Answering Questions: Providing information based on its training data.
Translation: Converting text from one language to another.
Summarization: Condensing long texts into shorter summaries.
Conversation: Engaging in dialogue, like chatbots or virtual assistants.

For example, if you type "The capital of Thailand is", the LLM uses its learned patterns to predict that the most likely next word is "Bangkok". It continues this process word by word to generate coherent and contextually relevant responses.

These models form the foundation for many AI tools you might already interact with, such as advanced chatbots, search engine enhancements, and content creation aids. This chapter, and the course, will help you understand the fundamental connection between how "large" these models are (in terms of parameters) and the computer hardware needed to actually run them.

Was this section helpful?

References

Attention Is All You Need, Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin, 2017 Advances in Neural Information Processing Systems (NeurIPS) DOI: 10.48550/arXiv.1706.03762 - Describes the Transformer architecture, which is fundamental to modern large language models, explaining the mechanism behind their pattern-matching capabilities.
Language Models are Few-Shot Learners, Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, Dario Amodei, 2020 Advances in Neural Information Processing Systems (NeurIPS) DOI: 10.48550/arXiv.2005.14165 - Presents GPT-3, demonstrating how increased scale in parameters and training data enables large language models to perform a wide array of tasks with minimal task-specific data.
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer, Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu, 2019 Journal of Machine Learning Research (JMLR) DOI: 10.48550/arXiv.1910.10683 - Details the Text-to-Text Transfer Transformer (T5) model, offering a comprehensive study of transfer learning techniques and showing how various NLP tasks can be framed as text-to-text problems.
On the Opportunities and Risks of Foundation Models, Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Dilara Bakar, Percy Liang, et al., 2021 arXiv (Stanford Institute for Human-Centered Artificial Intelligence (HAI)) DOI: 10.48550/arXiv.2108.07258 - Introduces the concept of foundation models, of which large language models are a prominent type, discussing their shared capabilities and implications across various applications.