Implementing Hybrid Search

While semantic search excels at finding related results based on meaning, it sometimes struggles with exact keyword matches or specific terms (like product codes, names, or jargon) that might not be well-represented in the embedding space. Conversely, traditional keyword search (often using algorithms like BM25) is excellent at retrieving documents containing specific terms but fails to grasp synonyms, related concepts, or user intent expressed in natural language.

Hybrid search offers a pragmatic solution by combining the strengths of both approaches. It aims to retrieve results that are both semantically relevant and contain important keywords, leading to a more comprehensive and often more accurate search experience.

Why Combine Search Methods?

Consider these scenarios:

1. Precision for Specific Terms: A user searching for a product ID like "XZ-47b" expects results containing exactly that string. Semantic search might return similar products but could miss the exact match if the ID isn't strongly represented in the vector context. Keyword search excels here.
2. Capturing Intent: A user searching for "how to fix a leaky faucet" benefits from semantic search understanding the concept of faucet repair, retrieving guides that use different phrasing ("dripping tap repair," "stop sink leak"). Keyword search alone might miss relevant documents that don't use the exact words "leaky faucet".
3. Boosting Relevance: Sometimes, a document is both semantically similar and contains specific keywords from the query. Hybrid approaches can boost the rank of such documents, considering them highly relevant.

By merging results from both vector similarity search and a keyword-based system, we can mitigate the weaknesses of each individual method.

Strategies for Combining Results

The core challenge in hybrid search is merging two distinct sets of results, each with its own relevance score, into a single, coherent ranked list. Here are common approaches:

1. Score-Based Fusion: This involves retrieving results independently from the semantic search system (vector database) and the keyword search system (e.g., Elasticsearch, OpenSearch using BM25) and then combining their scores.

   • Normalization: Scores from different systems are often on different scales. Vector similarity scores (like cosine similarity) might range from -1 to 1 or 0 to 1, while BM25 scores can vary widely depending on term frequencies and document lengths. Before combining, scores usually need to be normalized to a common range (e.g., 0 to 1). Techniques like min-max scaling or rank-based normalization can be used.
   • Weighted Combination: Once normalized, scores can be combined using a weighted sum. For a document $d$ retrieved by both systems: $$Score_{hybrid}(d) = \alpha \times Score_{semantic}(d) + (1 - \alpha) \times Score_{keyword}(d)$$ Here, $\alpha$ is a weighting factor between 0 and 1. A higher $\alpha$ gives more importance to semantic relevance, while a lower $\alpha$ emphasizes keyword matching. The optimal value for $\alpha$ often requires experimentation and tuning based on the specific dataset and use case. Documents found only by one system might be included lower in the list or assigned a score based only on the system that found them (potentially with a penalty).
   • Reciprocal Rank Fusion (RRF): RRF is a technique that combines ranked lists based on the rank position of each document, rather than relying directly on potentially incomparable scores. For each document $d$, its RRF score is calculated as: $$Score_{RRF}(d) = \sum_{i=1}^{N} \frac{1}{k + rank_i(d)}$$ where $N$ is the number of search systems (typically 2 for hybrid search), $rank_i(d)$ is the rank of document $d$ in the results list from system $i$, and $k$ is a constant (often set to 60, as suggested in the original paper) used to mitigate the impact of high ranks dominating the score. Documents not found by a system are assigned an infinite rank (contributing 0 to the sum). RRF is often preferred as it avoids the complexities of score normalization and weighting.

2. Pre-filtering / Post-filtering: Instead of complex score fusion, simpler strategies involve using one system to filter candidates for the other.

   • Keyword Pre-filtering: Perform a keyword search first to get a candidate set of documents containing essential terms. Then, perform semantic search only within this candidate set. This can be faster but might miss semantically relevant documents that lack the exact keywords.

• Semantic Pre-filtering: Perform a semantic search first to get a set of relevant documents. Then, apply keyword matching or re-ranking based on keyword presence within this set.

Implementation Flow

A typical score-based fusion implementation looks like this:

Flow diagram illustrating a common hybrid search implementation using score-based fusion. The user query is processed in parallel for semantic embedding and keyword extraction. Results from the vector database and keyword index are then combined and re-ranked.

Considerations

• Performance: Executing two searches (vector and keyword) in parallel and then fusing the results adds latency compared to a single search system. Caching and optimization are important.
• Complexity: Implementing and tuning hybrid search is more complex than using a single approach. It requires managing two different indexing and querying systems and developing a strategy.
• Tuning Weights/Parameters: Finding the right balance ($\alpha$ in weighted sums or $k$ in RRF) often requires evaluating search relevance on a representative dataset (as discussed in the next section) and iteratively adjusting the parameters.
• System Integration: Integrating a vector database with an existing keyword search system (like Elasticsearch or Solr) requires careful infrastructure planning. Some modern databases or search platforms offer built-in hybrid search capabilities, simplifying this integration.

Hybrid search represents a powerful technique for leveraging the complementary strengths of semantic understanding and precise term matching, ultimately leading to more effective and user-satisfying search results in many applications.