Research reports are the ammunition of investing, but ammunition management is a nightmare. After a few years of fundamental research, tens of thousands of PDFs sit piled up by date; finding what a company looked like at a specific point in time means relying on memory plus full-text-search luck. Asking an LLM directly does not work either — it invents numbers, mixes conventions, and gives conclusions you cannot falsify (I covered this in the InvestPilot post).
InvestRAG is my answer to this problem: turn the scattered ten thousand or so sell-side research reports and meeting minutes into a knowledge base you can query in natural language. Currently 14,122 documents and 428,974 text chunks are indexed, covering reports from August 2025 to the present. When you ask a question, the system first finds the most relevant original passages from the library, then hands them to an LLM to answer based on the source text — numbers have citations, nothing is invented.
The ideal: I ask “what was Pop Mart’s overseas expansion progress in Q1 2026?”, and the system pulls original passages from Deutsche Bank, UBS, Goldman Sachs reports on Pop Mart — with sources — and composes a cited answer.
That is RAG (Retrieval-Augmented Generation): retrieve first, then answer. Understanding the question and composing the language is the LLM’s job; the factual basis is backstopped by the original passages.
What it looks like
A six-layer architecture:
6. Query CLI (scripts/ask.py) / MCP server (plugs into Claude Code)
5. Generate LLM answers based on retrieved sources + cites [1][2]
4. Retrieve Vector search + FTS5 keyword (RRF fusion) + reranker + time decay
3. Store Qdrant local embedded mode (chunks + vectors + metadata)
2. Process Semantic chunking (tables kept whole) + metadata extraction
1. Parse PDF → structured text + tables (pymupdf fast path + MinerU OCR)Tech stack: Python 3.12 + uv / bge-m3 embeddings / bge-reranker-v2-m3 / Qdrant / SQLite FTS5 / qwen3.6:35b-mlx (local LLM) / MinerU (OCR).
A few designs I find interesting
Design 1: Hybrid retrieval — vector + keyword RRF fusion
Pure vector search is not enough for investment research: vectors excel at semantic similarity but miss exact numbers, ticker codes, and proper nouns. So I added SQLite FTS5 keyword search (trigram Chinese substring matching) and fused the results with vector search using RRF (Reciprocal Rank Fusion).
This way both semantic relevance and exact hits are covered. The FTS5 disk-based version also fixed the in-memory rank_bm25 timing out at 6 minutes with 170k chunks.
Design 2: Reranker — cross-encoder separates the scores
After RRF fusion the scores are flat (all around 0.03, low discrimination). I added the bge-reranker-v2-m3 cross-encoder (a model that scores query–passage pairs jointly) to re-score the top 30 from RRF pairwise. After reranking, scores spread from a flat 0.03 to 0.5–0.97, and the top results got a lot more reliable. MPS GPU, ~2-3s/query.
Design 3: Time-decay weighting — the timeliness of research
Research information is highly time-sensitive; a report from six months ago represents a different stage of the company. I added time-decay weighting in retrieval: half-life of 180 days, so documents from six months ago carry half the weight; with a 15% floor, even the oldest documents keep some weight for historical backtesting.
Paired with a mode parameter: “now” (current state, warns about timeliness) and “backtest” (historical, focuses on that period without staleness warnings).
Design 4: Tables kept whole
Over 60% of the information in research reports lives in tables. Standard parsing turns tables into garbled text, or chunking splits them mid-row. My approach: during semantic chunking, each table is kept as a single chunk, not split by fixed token count. The parsing layer uses MinerU (strong table parser) for OCR and pymupdf for normal PDFs (fast path).
Design 5: MCP server — plugging into Claude Code / InvestPilot
InvestRAG registers an MCP server providing a rag_search tool. Claude Code (or the InvestPilot agent harness) can call it to search the report library. Key design: report source text never leaves the local machine — bge-m3 retrieval + LLM summarization both run locally; only the refined {summary, sources} is returned to the cloud agent. Quota-friendly, and avoids sending financial source text through cloud APIs that might trigger risk controls.
Relationship with InvestPilot
These two projects are complementary:
| InvestRAG | InvestPilot | |
|---|---|---|
| Role | Knowledge base / information retrieval | Analysis framework / financial modeling |
| Solves | ”What do the reports say?" | "Is this stock worth it?” |
| Output | Business facts + market consensus (cited) | Financial model + Monte Carlo + payoff |
| LLM role | Understand source + compose answer | Reason + model + audit |
With InvestRAG’s MCP tool plugged into InvestPilot, the agent can query the report library for business facts while doing financial deep-dives — one grabs facts, the other crunches numbers.
Pitfalls I hit
① 66% of reports are garbled. Sell-side reports are 96% image-based PDFs; pymupdf extracts gibberish. OCR is mandatory. I used MinerU (pipeline backend + multiprocess), benchmarked to 4.3s/report (from 533h serial → 29h multiprocess, 10x speedup). Hit a pitfall where missing paddlepaddle caused a 7x slowdown.
② rank_bm25 memory explosion. At 170k+ chunks, the in-memory BM25 took 6 minutes per scan — retrieval timed out. Switched to SQLite FTS5 (disk-based, trigram Chinese), <1s.
③ Embed is the ingestion bottleneck. bge-m3 on long texts (500 chars/chunk), MPS GPU still ~10s/report. Total token count determines time; reducing chunk count doesn’t help. Tried CPU multiprocess (12 cores theoretically beats GPU, but startup/serialization overhead not worth it), ONNX CoreML (pending validation). Currently accepting ~10s/report.
④ Catalog lock repeatedly. ProcessPoolExecutor workers don’t exit after MinerU finishes, holding the SQLite catalog write lock → subsequent ingest all hit database is locked. try/finally + pkill partially fixed; background scenarios still intermittent.
⑤ ollama default num_ctx=262K causes timeout. ollama 0.31.2 auto-sets context window based on VRAM; the openai library default timeout can’t connect. Limiting num_ctx=8192 + timeout=120 fixed it.
Current status
- Indexed 14,122 documents / 428,974 chunks (2025-08 ~ 2026-07)
- Hybrid retrieval (vector + FTS5 + RRF) + reranker + time decay
- Answer generation with qwen3.6:35b-mlx (local, 21GB MLX)
- MCP server integrated with Claude Code
- Pending: ~18,696 older reports to OCR + 21k audio transcripts to ASR
Closing
InvestRAG does not make investment decisions for me. It only finds the original passages and their sources.