Rerank

Cohere Rerank is a cross-encoder reranking model designed to improve the precision of search results in retrieval-augmented generation (RAG) pipelines and enterprise search systems. It is intended for developers and data teams who have already implemented an initial retrieval stage—using BM25, dense embeddings, or a hybrid approach—and need a second-stage model to reorder candidate documents by true relevance to the query. The primary audience includes teams building knowledge assistants, customer support bots, compliance search tools, and any application where top-K precision directly impacts user trust and downstream LLM output quality. Cohere positions Rerank as a drop-in component that can be deployed via API, on-premises, or in a VPC-isolated environment for compliance-sensitive workloads.

Rerank 4, the latest generation, supports a 32K context window—four times larger than the previous generation—allowing it to score longer documents without truncation. The model accepts a query and a set of candidate passages (typically 50–200 from the first-stage retriever) and outputs relevance scores from 0 to 1 for each passage. Cohere claims that Rerank 4 improves retrieval relevance and reduces latency in enterprise applications compared to its predecessor. The service is priced at $2.00 per 1,000 queries, with no per-token charges for the reranking step itself. For teams that also need embeddings, Cohere Embed 4 costs $0.12 per million input tokens and $0.47 per million output tokens. Cohere also offers flexible deployment: the API endpoint, on-premises containers, or VPC isolation for data residency requirements.

In the reranking market, Cohere Rerank competes directly with Voyage Rerank, Jina Reranker, BGE Reranker, Mixedbread (mxbai-rerank), and ColBERT. Community benchmarks on Reddit and independent evaluations (e.g., ZeroEntropy's 2025 guide) consistently rank Cohere's performance as strong but not universally dominant. Users report that Cohere is 'easy, fast, cheap, and pretty good,' but that LLM-based reranking (e.g., using GPT-4 to reorder candidates) can produce better results in complex reasoning tasks. Cohere's main differentiators are its 32K context window, mature API ecosystem, and enterprise-grade deployment options (on-prem, VPC). However, it is not available natively on Azure or AWS Bedrock, which limits adoption for teams already locked into those clouds.

The honest trade-offs with Cohere Rerank are threefold. First, while the per-query cost ($2.00/1k queries) is low at small scale, it becomes a significant line item at high query volumes—100,000 queries per day would cost $200/day just for reranking. Second, building a complete search pipeline (retrieval + reranking + LLM) requires substantial technical know-how; hidden development costs for integration, caching, and latency optimization are often underestimated. Third, the model is a black box—users cannot fine-tune it on domain-specific relevance judgments, unlike open-source alternatives such as BGE Reranker or ColBERT. For teams that need multilingual support beyond English, Cohere's performance may lag behind specialized multilingual rerankers. Finally, the API dependency means that any outage or latency spike at Cohere directly impacts the user's search quality, which is a risk for mission-critical applications.

Rerank 4 costs $2.00 per 1,000 queries. Embed 4 costs $0.12 per million input tokens and $0.47 per million output tokens. No fixed tiers; pay-as-you-go billing.