Lewis Lien PolanskyCEO & Co-FounderWe've been busy this holiday season, shipping optimizations throughout the stack. I'm thrilled to announce the stable release of our v2 API, among other product updates.
4 min read
Fig. 00 · the v2 API, the new docs site and a 3x drop in search latencyWe've been busy this holiday season, shipping optimizations throughout the stack. I'm thrilled to announce the stable release of our v2 API, among other product updates.
We've been busy this holiday season, shipping optimizations throughout the stack. I'm thrilled to announce the stable release of our v2 API, among other product updates.
TL;DR
We've invested in a beautiful new docs site with an interactive API explorer. You can now supply your API key and test out requests from within the docs!
In addition to the interactive docs, for on-the-fly API development, the Captain Playground has been redesigned for non-technical users to experience the power of high-accuracy RAG within a non-programmatic interface.
Our retrieval precision has been further optimized for healthcare, legal, and financial RAG use cases. We have employed a number of techniques to increase accuracy, namely long-context document embedding and automatic domain detection.
Long-context document embedding entails the embedding of the entire context of a file and supplementing it with the narrow chunk content. This provides both broad and narrow semantic value during similarity lookups.
Automatic domain detection allows Captain to index domain-specific terms within a BM25 index (to supplement dense vector search). This development solves the long-standing issue within RAG (and search more broadly) of domain-specific terms getting "lost" without proper vector correlations.
By combining these two techniques and weighing them, search results provide a better understanding of complex document narratives and niche industry terms like drug names or non-English legal terms.
We have seamlessly migrated away from vector storage, resulting in an over 3x drop in latency. Our experimentation has also shown better recall when object storage search is used as opposed to leveraging vector databases.
This past December we successfully transferred all non-textual knowledge data into object storage. There was no disruption to service uptime.
We remain committed to leveraging state-of-the-art embedding models, but going forward, our embeddings will be stored as objects.
Captain has partnered with the Hacker News team to develop an RAG forum search across the popular aggregator's over 19+ years of posts and comments. More on that in a few weeks.
Our Python and TypeScript SDKs are underway; these will be out later this month.
Happy Shipping!
Lewis Lien Polansky
CEO & Co-Founder · CaptainLewis is co-founder and CEO of Captain, building self-tuning file search for AI agents.
Serving international customers on low latency semantic search.
3 min read
Elasticsearch ships without parsing, OCR, chunking, embeddings, connectors or sync. Teams add four to seven vendors to cover the gaps, then own every seam between them. Captain includes all of it behind one index call.
5 min read
Cloudflare AI Search caps files at 4 MB, chunks at a token count and metadata at five fields, because it runs on Workers and Vectorize. Captain has no file size cap, picks optimal chunk boundaries, and stores whatever metadata you send.
6 min read
Benchmarks, engineering write-ups and the occasional retraction, straight from the source.
