RelevantSearch.AI
Pattern · Volume 01 · Section E --- Query routing and federation · Updated May 2026

Federated multi-index search

Source: Established pattern in enterprise search; production deployments across Elasticsearch / OpenSearch (cross-cluster search), Solr (distributed search), Coveo (federation), Vertex AI Search, Azure AI Search

Classification — Pattern for combining results from multiple separate indexes into a single result set.

Intent

Search across multiple separate indexes — different content types, different domains, different geographical or organizational boundaries — and combine the results into a coherent unified response, with appropriate ranking across the sources.

Motivating Problem

Many production systems have content in multiple separate indexes for legitimate reasons: different content types (products vs articles vs videos vs support tickets) with different schemas; different domains with different update patterns; different regions or organizations with different governance; legacy indexes that can't be merged. Single-index queries don't work; the user shouldn't have to choose which index to search. Federated search addresses this by routing queries to multiple indexes and combining results, presenting the unified view as if it were a single index.

How It Works

Query distribution. The federation layer takes the incoming query and distributes it to relevant indexes. Distribution can be: broadcast (send to all indexes; combine all results); routed (send only to indexes likely to have relevant content based on query analysis); progressive (start with high-priority indexes, add others if needed). Each strategy trades latency and cost for completeness.

Per-index processing. Each index processes the query using its appropriate retrieval pipeline. Different indexes may use different architectures: the products index uses hybrid retrieval; the articles index uses BM25 plus dense retrieval; the videos index uses metadata-only lexical retrieval. The federation layer doesn't require uniformity across indexes; each can optimize independently.

Result combination. Results from multiple indexes are combined into a single ranked list. The challenge: scores from different indexes aren't directly comparable. Solutions include: per-source result blocks ("Products (5), Articles (3), Videos (2)") avoiding the combination problem entirely; rank-based fusion (RRF) treating each index as a retrieval method; learned ranking models that take per-source scores plus features as input. The choice affects UX as much as quality.

Latency management. Federated search's latency is bounded by the slowest index involved (when results from all indexes are needed for fusion). Production patterns: timeout per index with partial results if slow indexes don't return in time; asynchronous federation where slower indexes contribute results that appear later in the UX; pre-fetching for high-priority indexes. The patterns prevent slow indexes from degrading user experience.

Per-source diversity. Some federated UX shows results grouped by source (the per-source result block pattern). Others show one merged ranked list. The merged pattern is more demanding (requires score comparison across indexes) but produces a more unified experience. The grouped pattern is easier to implement and clearer about provenance, but may be less useful when users don't care which index a result came from.

When to Use It

Enterprise search across multiple content repositories (Confluence + SharePoint + Drive + Jira + Slack + custom systems). Large e-commerce with multiple content types (products + articles + brand pages + community content). Multi-region deployments where content sovereignty requires separate indexes. Multi-tenant SaaS with per-tenant indexes plus shared content.

Alternatives — unified single-index deployment when content can be normalized and combined; the operational complexity of multiple indexes is usually only justified when single-index isn't feasible. Intent-based routing (prior entry) when query heterogeneity matters more than content heterogeneity.

Sources
  • Elasticsearch / OpenSearch cross-cluster search documentation
  • Solr distributed search and cross-collection documentation
  • Coveo federation and source documentation
  • Glean and Hebbia documentation on federated enterprise search architectures

Read in context within Volume 01 →