Reranking

Reranking reorders a set of candidate results — from vector_search, text_search, rrf, or a plain table — using a dedicated reranker model, a chat model, or a decision model. This produces more relevant rankings than the initial retrieval scores alone.

How It Works

1. An initial search (vector, full-text, or hybrid) retrieves candidate documents.
2. The rerank() UDTF sends each candidate's text to a reranker model alongside the query.
3. The reranker scores each document for relevance and returns them in order.

This two-stage pattern (retrieve then rerank) is standard in modern search and RAG pipelines.

Configuration

Reranker Models

Define dedicated reranker models in the rerankers: section of the spicepod. Supported providers:

Provider prefix	Description
cohere:	Cohere Rerank API (e.g., cohere:rerank-v3.5)
voyage:	Voyage Rerank API (e.g., voyage:rerank-2)
jina:	Jina Rerank API (e.g., jina:jina-reranker-v2-base-multilingual)
http:// / https://	Any HTTP endpoint implementing the standard rerank API schema
huggingface: (or hf:)	A cross-encoder reranker downloaded from Hugging Face and run locally, in-process (e.g. huggingface:huggingface.co/BAAI/bge-reranker-base)
file:	A cross-encoder reranker loaded locally from a directory of model artifacts (e.g. file:/models/bge-reranker-base)

rerankers:
  - from: cohere:rerank-v3.5
    name: cohere_rr
    params:
      api_key: ${secrets:COHERE_API_KEY}

  - from: voyage:rerank-2
    name: voyage_rr
    params:
      api_key: ${secrets:VOYAGE_API_KEY}

  - from: jina:jina-reranker-v2-base-multilingual
    name: jina_rr
    params:
      api_key: ${secrets:JINA_API_KEY}

  - from: https://rerank.internal/v1/rerank
    name: byo_rr
    params:
      api_key: ${secrets:INTERNAL_RR_KEY}  # optional

Local Rerankers

The huggingface: and file: prefixes run a cross-encoder reranker in-process instead of calling a hosted API. file: uses local artifacts only. huggingface: downloads model artifacts from the Hugging Face Hub, then runs scoring locally. Both are included in the default spiced build.

rerankers:
  # Downloaded from the Hugging Face Hub. Pin a revision with a trailing `:<rev>`.
  - from: huggingface:huggingface.co/BAAI/bge-reranker-base
    name: local_rr
    params:
      hf_token: ${secrets:HF_TOKEN} # needed for private or gated repositories

  # Loaded from a directory holding config.json, tokenizer.json, and the weights.
  - from: file:/models/gte-reranker-modernbert-base
    name: dir_rr

Both accept the same two tuning parameters:

Parameter	Description	Default
max_seq_length	Maximum sequence length for the (query, document) pair. When unset, the model's own configured maximum applies.	-
truncate	How to handle a pair longer than the maximum sequence length: none rejects the request with an input-validation error, end discards the end of the pair, start discards the start. Matched case-insensitively.	none

huggingface: rerankers additionally accept hf_token (also spelled api_key) for private or gated repositories, and hf_endpoint to download through a mirror or proxy of the Hub. They use the same token, endpoint, cache and offline settings as other Hugging Face models.

LLM-as-Reranker

Any registered chat model can also be used as a reranker without additional configuration. When a model name resolves to a chat model instead of a dedicated reranker, Spice wraps it in an LLM-based reranking adapter. A model with a native decisions API, such as OpenAI gpt-6-luna, reranks as a decision model instead, unless the call sets via => 'chat' or a prompt_template.

models:
  - from: openai:gpt-4o-mini
    name: gpt_mini

SELECT * FROM rerank(
  vector_search(kb, 'onboarding checklist', limit => 40),
  document => 'content',
  model    => 'gpt_mini',       -- resolves to the registered chat model
  strategy => 'pointwise',      -- or 'listwise' (default)
  limit    => 10
);

The strategy parameter controls how the LLM scores documents:

• listwise (default): sends all candidates in a single prompt and asks the LLM to rank them.
• pointwise: sends each candidate individually and asks the LLM to score it.

Decision Models as Rerankers

A decision model, such as TypeSafe Jev, can also be used as a reranker without additional configuration. Spice asks the model whether each document is relevant to the query and uses the returned probability, a value between 0 and 1, as the document's rerank_score.

models:
  - from: typesafe:jev
    name: jev
    params:
      typesafe_api_key: ${secrets:TYPESAFE_API_KEY}

SELECT * FROM rerank(
  vector_search(kb, 'onboarding checklist', limit => 40),
  document => 'content',
  model    => 'jev',            -- resolves to the registered decision model
  strategy => 'listwise',       -- or 'pointwise' (default)
  limit    => 10
);

For decision models, the strategy parameter controls how many requests are sent:

• pointwise (default): sends one request per candidate. Requests run concurrently, subject to the model's own rate limits, and each score is independent of the other candidates.
• listwise: sends all candidates in a single request, with one relevance question per candidate.

The question is fixed, so prompt_template does not apply to decision models. If the model fails for any candidate, or returns an answer that is not a probability between 0 and 1, the whole rerank() query fails.

OpenAI gpt-6-luna and its dated snapshots serve both chat and decisions, and rerank through the Decisions API by default, scored the same way. A chat model without a native decisions API, such as gpt-4o-mini, can also rerank this way with via => 'decisions', through its chat adapter.

Model Resolution

A dedicated reranker named by model is always used as is. Any other model reranks through one of two paths, which the optional via argument selects:

via	prompt_template	Path
Not set	Not set	Decisions when the model has a native decisions API (TypeSafe models, OpenAI gpt-6-luna), otherwise chat. When the model has no handler for that path, the other path is used.
Not set	Set	Chat, with the template.
'chat'	Optional	Chat.
'decisions'	Not set	Decisions. A chat model answers through its chat adapter.
'decisions'	Set	The query fails: the decisions path asks a fixed question.

via and prompt_template apply only to chat and decision models; setting either for a dedicated reranker fails the query. A model that has no chat completions, such as TypeSafe Jev, cannot use via => 'chat' or prompt_template, and the query fails with an error that says to remove them. On the chat path, strategy defaults to listwise; on the decisions path, to pointwise. Dedicated rerankers ignore strategy.

In v2.4.1 and earlier, the model name was looked up in this order: dedicated rerankers, then chat models, then decision models, and via did not exist. OpenAI gpt-6-luna therefore reranked through chat completions with the listwise strategy, and its rerank_score was a relative score from one prompt. It now reranks through the Decisions API with the pointwise strategy, one request per candidate, and its rerank_score is the probability that the document is relevant. Add via => 'chat' to keep the earlier behavior. A prompt_template passed to a decision model with no chat completions was ignored in v2.4.1 and earlier, and now fails the query.

SQL Usage

SELECT * FROM rerank(
  <input>,
  document => '<column>',
  model    => '<reranker_name>',
  limit    => <n>
);

For the complete parameter reference, see Reranking SQL Reference.

Examples

Hybrid Recall + Rerank

SELECT * FROM rerank(
  rrf(
    vector_search(docs, 'delta lake time travel', limit => 50),
    text_search(docs, 'delta lake time travel', limit => 50)
  ),
  document => 'content',
  model    => 'cohere_rr',
  limit    => 10
);

The query is automatically extracted from the nested search UDTFs — no need to specify query explicitly.

Bare-Table Rerank

When reranking a plain table (not a search UDTF), provide the query explicitly:

SELECT * FROM rerank(
  tickets,
  query    => 'auth failures',
  document => 'body',
  model    => 'voyage_rr',
  limit    => 5
);

Custom LLM Prompt

SELECT * FROM rerank(
  vector_search(kb, 'onboarding checklist', limit => 40),
  document        => 'content',
  model           => 'gpt_mini',
  strategy        => 'pointwise',
  prompt_template => 'Rate 0-1: is this useful for a new hire?\nQuery: {query}\nDoc: {document}',
  limit           => 10
);