Jev is a state-of-the-art search reranker

We evaluated Jev as a reranker for search results in turbopuffer. The results may surprise you. Jev isn’t just a good reranker, it performed better than existing state-of-the-art rerankers in our benchmark.

Why Jev is promising as a reranker

The job of a reranker is to intelligently reorder a list of documents so that the most relevant documents appear first. The definition of “relevance” is, of course, subjective and largely baked into the model during training. In a typical search pipeline, reranking comes after the initial retrieval stage, once faster vector or text search has narrowed the candidate set to the 100–1,000 most promising documents.

Jev is a general-purpose decision model, not a specialized reranker model. It takes some state (in our case, the query and document) plus a natural-language question and rubric, then returns a probability distribution over the possible decisions. Unlike a traditional reranker, Jev generalizes across any decision task, not just search.

Thus, Jev can assess relevance based on any question and rubric we provide, allowing us more control over the definition of relevance. This makes it promising as a highly-tunable reranker.

My colleague William (@n0riskn0r3ward) compared Jev to other SOTA reranker models (Voyage rerank-3 and Jina reranker-v3.5), as well as GPT 5.6 Luna (low). He used GPT 5.6 Sol (none) as a relevance baseline. Frontier LLMs like GPT 5.6 Sol are generally too slow and expensive to use as rerankers in production, but they effectively provide an approximate upper bound on relevance quality. You can find the complete results and his thoughts, as well as many other relevance bangers, on his timeline (if you care at all about search relevance, follow William).

The TL;DR: Jev outperformed both rerankers and GPT 5.6 Luna in our benchmark, achieving the highest agreement (~64%) with Sol’s reranking for a 50-document result set. Quite simply, Jev did a better job reranking than SOTA reranking models and a nano-tier LLM.

Jev is also relatively cheap. In our benchmark setup we paid $0.0138 per query. For reference, Voyage rerank-3 was roughly 40% cheaper, while GPT 5.6 Luna (low) was roughly 475% more expensive.

Jev’s Score API as a reranker

There are a few ways we could use Jev’s API to rerank. For example, @aHev tested the Noul API with success, but the method we found most performant used Jev’s Score API. We define ten relevance levels, ranging from “unrelated” (5) to “perfect match” (100). For each document, we instruct Jev to give a probability score for each label, then combine the probabilities into a weighted relevance score. The example below shows how candidates could score against the query “reset password” given 3 relevance labels.

How you can use Jev as a reranker in turbopuffer

We’re now announcing turbopuffer’s native reranking private beta. turbopuffer’s API already supports reranking via reciprocal rank fusion (RRF), but native reranking adds the model-based scoring, including Jev.

If you’d like access to the native reranking private beta, shoot me a DM.