Query Augmentation: Expansion and Transformation

When users search your Retrieval-Augmented Generation (RAG) system, their queries are often just an initial step, not a perfect way to access the complete information in your knowledge base. Queries can be short, ambiguous, use different vocabulary than your documents, or simply miss the mark in expressing the true information need. This is where query augmentation comes into play. It's a set of techniques designed to refine, expand, or transform the original user query to improve its chances of matching relevant documents in the retrieval phase. By intelligently modifying the query, you can bridge the semantic gap between user intent and your document corpus, leading to more accurate and comprehensive context for the generator.

Effective query augmentation can significantly reduce instances of "I couldn't find anything relevant" and directly contribute to the quality of the final generated answer. Let's explore the primary strategies: query expansion and query transformation.

Query Expansion: Broadening the Search Net

Query expansion aims to enrich the original query by adding new terms or phrases. This helps in retrieving documents that might use synonyms, related terms, or different phrasings for the same idea. The goal is to cast a wider, yet still relevant, net.

Synonym and Acronym Expansion

One of the most straightforward expansion techniques is to identify terms in the query and add their synonyms. For example, if a user queries "how to fix RAG latency," an expanded query might include "how to resolve RAG response time issues" or "troubleshooting RAG performance delay."

Sources for synonyms include:

• General-purpose thesauri: WordNet is a classic example.
• Domain-specific dictionaries: For specialized fields, a custom-built dictionary of synonyms and jargon is invaluable. For instance, in a medical RAG, "myocardial infarction" could be expanded with "heart attack."
• Embedding-based similarity: You can find terms whose embeddings are close to the query terms in vector space. This often uncovers subtle semantic relationships.

Similarly, acronyms and abbreviations are common in queries and documents. "LLM" should ideally match documents mentioning "Large Language Model," and vice-versa. A system will maintain a mapping for these:

• "AWS RAG setup" -> "Amazon Web Services Retrieval-Augmented Generation setup"
• "GDPR compliance" -> "General Data Protection Regulation compliance"

While powerful, naive synonym expansion can sometimes lead to query drift, where the expanded query loses its original focus. Context matters. Adding "bank" as a synonym for "financial institution" is fine, but if the query is about a "river bank," it's problematic. Careful selection or weighting of expanded terms is necessary.

Related Term Expansion

Aside direct synonyms, you can expand queries with terms that are semantically related. This is particularly useful when the user's initial query is broad or uses high-level terms. For instance, a query about "sustainable energy" might be expanded to include "solar power," "wind turbines," "geothermal energy," or "renewable resources."

Methods for finding related terms include:

• Corpus analysis: Analyzing term co-occurrence patterns within your document set can reveal strong relationships.
• Knowledge Graphs: If your RAG system has access to a knowledge graph, you can traverse relationships from query entities to find related entities and attributes. (We cover knowledge graph integration in more detail in "Integrating Knowledge Graphs for Enhanced Retrieval.")
• LLM-based generation: You can prompt an LLM with the original query and ask it to generate a list of related topics or keywords. For example: Original Query: "Best practices for vector database indexing." LLM-Generated Related Terms: "ANN algorithms," "HNSW," "IVFADC," "vector similarity search," "embedding storage."

The following diagram illustrates how different expansion techniques can augment an initial query:

A diagram illustrating how an initial user query can be augmented using synonym, acronym, and related term expansion.

Sub-query Generation

Complex user queries often pack multiple questions or facets into a single sentence. For example, "What are the performance and cost implications of using different re-ranking models in RAG systems for financial document analysis?" This query implicitly asks about performance, cost, re-ranking models, RAG systems, and financial documents.

Decomposing such a query into simpler sub-queries can lead to more targeted retrieval for each aspect. An LLM can be quite effective here:

Original Query: "Compare hybrid search vs. dense retrieval for medical RAG systems regarding accuracy and latency."

LLM-Generated Sub-queries:

1. "Accuracy of hybrid search in medical RAG systems"
2. "Latency of hybrid search in medical RAG systems"
3. "Accuracy of dense retrieval in medical RAG systems"
4. "Latency of dense retrieval in medical RAG systems"
5. "Comparison of hybrid search and dense retrieval for medical RAG"

The results from these sub-queries can then be aggregated or processed to synthesize an answer to the original complex query. This approach can be particularly beneficial when your documents are granular and address specific facets of a topic rather than providing holistic overviews.

Query Transformation: Reshaping for Better Understanding

Query transformation modifies the structure or phrasing of the original query, rather than just adding terms. The aim is often to align the query more closely with the language used in the document corpus or to generate a representation that is more effective for similarity search.

Spelling Correction and Normalization

This is a fundamental preprocessing step. Typos, misspellings, and inconsistent capitalization can easily derail retrieval.

• Spelling Correction: Use algorithms like Levenshtein distance combined with a dictionary, or pre-trained spelling correction models.
• Normalization: Convert text to a consistent case (e.g., lowercase), remove punctuation (or handle it consistently), and standardize variations (e.g., "e-mail" to "email").

While seemingly basic, spelling correction and normalization are critical for production systems where user input is unconstrained.

Query Rewriting and Rephrasing

Sometimes, the user's phrasing, while grammatically correct, might not be optimal for retrieval. An LLM can be employed to rewrite or rephrase the query into alternative forms that might yield better results.

Original Query: "My RAG is too slow, what do I do?" LLM Rephrased Queries:

• "Techniques to reduce latency in RAG systems."
• "Optimizing performance of Retrieval-Augmented Generation pipelines."
• "How to improve RAG system response time?"

These rephrased queries use more formal and descriptive language, which is often closer to the terminology found in technical documentation or research papers that might form your knowledge base. You can then search with the original query and one or more rephrased versions, potentially combining the results.

Document Embeddings (HyDE)

HyDE is a more advanced transformation technique that has shown significant promise. Instead of directly embedding the (often short and keyword-heavy) user query, HyDE uses an LLM to generate a document that answers the query. This generated document, being more verbose and context-rich, is then embedded and its embedding is used to search the vector store.

The rationale is that an ideal answer document is likely to be semantically closer in embedding space to actual relevant documents than the terse original query.

The process is as follows:

1. User Query: e.g., "benefits of multi-vector embeddings for RAG"
2. LLM Generates Document: Prompt an LLM: "Generate a concise paragraph explaining the benefits of using multi-vector embeddings in RAG systems."
   • LLM Output (Document): "Multi-vector embeddings enhance RAG systems by capturing different aspects or granular pieces of information within a single document. This allows for more detailed retrieval, as queries can match specific sub-contexts rather than just the overall document theme, leading to improved relevance and reduced ambiguity in search results, especially for complex or multifaceted documents."
3. Embed Document: Convert this generated paragraph into an embedding vector using your chosen embedding model.
4. Retrieve: Use this embedding to query your vector database of actual document embeddings.

The HyDE process can be visualized as:

The process flow for Document Embeddings (HyDE). An LLM generates an answer to the user's query. This document is then embedded and used for retrieval against the actual document corpus.

HyDE is particularly effective when queries are abstract or when there's a significant vocabulary mismatch between queries and documents. The generated document acts as a "semantic bridge."

Implementation Approaches for Query Augmentation

While query augmentation offers powerful benefits, it's not a silver bullet and requires careful implementation.

• Managing Query Drift: The primary risk with expansion is query drift, where the augmented query deviates too much from the original intent, leading to irrelevant results.
  • Solutions:
    • Weighting: Assign higher importance to original query terms than to expanded terms during retrieval.
    • Selective Expansion: Use domain knowledge or confidence scores to decide which terms to add.
    • Re-ranking: Use a re-ranker (covered in "Advanced Re-ranking Architectures for Relevance") to evaluate results from both original and augmented queries and pick the best ones.
• Latency: Each augmentation step, especially those involving LLM calls (like HyDE or LLM-based rephrasing), adds latency to the retrieval pipeline.
  • Solutions:
    • Efficient LLMs: Use smaller, faster LLMs for augmentation tasks if full power isn't needed.
    • Caching: Cache frequent augmentations or documents.
    • Parallelization: Explore running original query retrieval and augmented query retrieval in parallel if resources allow.
• Complexity: Implementing and maintaining multiple augmentation strategies adds complexity to your RAG system.
  • Solutions: Start with simpler techniques (e.g., synonym expansion from a curated list, basic spell check) and incrementally add more advanced ones. Monitor the impact of each addition.
• Cost: LLM-based augmentation incurs API costs.
  • Solutions: Evaluate the cost-benefit. Sometimes, a simpler, non-LLM augmentation might be sufficient. Use cost-effective models where possible.
• Domain Specificity: Generic augmentation tools might not be optimal. Fine-tuning synonym lists, related terms, or even the LLMs used for augmentation to your specific domain can yield better results.
• Evaluation is Important: Always measure the impact of query augmentation on your retrieval metrics (precision, recall, nDCG, etc.) and end-to-end RAG quality. A/B test different augmentation strategies to find what works best for your specific use case and data.

Query augmentation is a dynamic area, and the choice of techniques will depend on your RAG system's specific requirements, the nature of your data, and the characteristics of user queries. By thoughtfully expanding and transforming queries, you equip your RAG system to better understand user intent and retrieve the most pertinent information, laying a stronger foundation for high-quality generation.