Getting Started with Kerb Toolkit
Chapter 1: Introduction and First Text Generation
Course Overview and Setup
Principles of the Toolkit
Configuring LLM Providers
Executing Your First Generation Call
Handling Streaming Responses
Chapter 2: Advanced Prompting Techniques
Introduction to Prompt Engineering
Creating Dynamic Prompts with the Template Engine
Managing and Versioning Prompts
Implementing Few-Shot Prompting
Extracting Structured Data from LLM Outputs
Parsing JSON and Code Snippets
Chapter 3: Managing Context and Tokens
The Importance of the Context Window
Counting Tokens with the Tokenizer
Strategies for Text Truncation
Managing Token Budgets for Complex Prompts
Chapter 4: Preparing Data for Retrieval
Data Loading Fundamentals
Loading Documents from Different Sources
The Rationale Behind Text Chunking
Applying Chunking Strategies
Text Preprocessing for Better Retrieval
Chapter 5: Embeddings and Semantic Search
Understanding Text Embeddings
Generating Embeddings
Fundamentals of Vector Similarity
Performing Semantic Search
Choosing an Embedding Model
Chapter 6: Building Retrieval-Augmented Generation (RAG) Systems
Anatomy of a RAG System
Creating a Simple Retrieval Pipeline
Implementing Different Search Methods
Improving Relevance with Re-ranking
Managing Retrieved Context for Generation
Chapter 7: Building Conversational Applications with Memory
The Challenge of Stateful Conversations
Implementing Conversation Buffer Memory
Using Summary Memory for Long Conversations
Tracking Entities Across a Conversation
Chapter 8: Developing Autonomous Agents
Introduction to LLM Agents
The ReAct Pattern for Reasoning and Acting
Building a ReAct Agent
Defining and Using Tools
Implementing Plan-and-Execute Agents
Orchestrating Multi-Agent Systems
Chapter 9: Optimizing for Performance and Cost
Identifying Performance Bottlenecks
Implementing LLM Response Caching
Caching Embeddings to Reduce API Calls
Cache Invalidation Strategies
Chapter 10: Ensuring Application Safety and Reliability
Adding Safety Guardrails to Applications
Implementing Content Moderation
Detecting and Masking Personal Information
Introduction to Testing LLM Applications
Using Mocks for Deterministic Tests
Chapter 11: Advanced Capabilities
Processing Image Inputs with Multimodal Models
Benchmarking LLM Outputs with Evaluation Metrics
Preparing Datasets for Fine-Tuning
Improving Relevance with Re-ranking
Was this section helpful?
- The use of MMR for reordering documents and producing summaries, Jaime G. Carbonell and Jade Goldstein, 1998 Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval (ACM) DOI: 10.1145/290941.291025 - Introduces Maximal Marginal Relevance (MMR) for selecting relevant and diverse documents, a method to avoid redundancy in results.
- Introduction to Information Retrieval, Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze, 2008 (Cambridge University Press) - Covers information retrieval fundamentals, including ranking algorithms, precision, and recall, which are core to understanding re-ranking.
- Retrieval-Augmented Generation for Large Language Models: A Survey, Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, Haofen Wang, 2023 arXiv preprint arXiv:2312.10997 DOI: 10.48550/arXiv.2312.10997 - Presents an overview of Retrieval-Augmented Generation (RAG) systems, discussing various stages like retrieval and re-ranking for improved performance.