Chroma

Reranking is a critical step in making search more useful to users. There’s tons of user and application specific context that is useful to users that is lost if you simply return the results from search alone. Reranking considers signals and data from your application to place results that are more likely to satisfy the user’s search needs. Links - Learning to rank: https://en.wikipedia.org/wiki/Learning_to_rank - XGBoost: https://xgboost.readthedocs.io/en/release_3.2.0/ - LambdaMark: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/MSR-TR-2010-82.pdf - LightGBM: https://github.com/lightgbm-org/LightGBM - Efficient Document Ranking with Learnable Late Interactions: https://arxiv.org/abs/2406.17968 - Passage Re-ranking with BERT: https://arxiv.org/abs/1901.04085 - A Thorough Comparison of Cross-Encoders and LLMs for Reranking SPLADE: https://arxiv.org/abs/2403.10407 - Context-1 Research Report: https://www.trychroma.com/research/context-1 Chapters 0:00 - Intro 2:37 - Example application signals and metrics 6:49 - Aggregating metrics 8:04 - Example algorithmic reranker in code — Chroma builds open-source search infrastructure for AI Fast, serverless, and scalable infrastructure supporting vector, full-text, regex, and metadata search. Built on object storage and trusted by millions of developers. Open-source Apache 2.0. https://trychroma.com/

Chroma offers a variety of lexical search strategies. Full Text Search (FTS), BM25 and SPLADE are the core offerings. Each have their strengths and weaknesses and use cases. Docs on FTS: https://docs.trychroma.com/docs/querying-collections/full-text-search Docs on BM25: https://docs.trychroma.com/integrations/embedding-models/chroma-bm25#chroma-bm25 Docs on SPLADE: https://docs.trychroma.com/integrations/embedding-models/chroma-cloud-splade#chroma-cloud-splade 0:00 - Lexical search overview 0:30 - FTS 0:52 - BM25 1:23 - SPLADE 1:53 - Demo -- Chroma builds open-source data infrastructure for AI Fast, serverless, and scalable infrastructure supporting vector, full-text, regex, and metadata search. Built on object storage and trusted by millions of developers. Open-source Apache 2.0. https://trychroma.com/