codescope

An open-source rebuild of the architecture I designed for my M.Sc. thesis at Siemens Mobility — but recast as a bounded agentic retrieval loop instead of a fixed two-stage chain. SCIP-precise symbol graphs (no name-matching), one Kuzu node type, one CALLS relation, four typed tools the model picks among per turn. Includes a live tool-trace UI so you can watch every decision. Tested against a 20-question hand-written eval on fastapi (6,461 symbols, 12,655 call edges).

Most 'chat with your code' tools do one vector search, dump the top-k chunks into a prompt, and hope. They hallucinate symbols. My thesis system fixed retrieval quality with a knowledge graph + LLM-enriched file summaries — but its pipeline was fixed (vector → graph → synthesize, one shot) and its enrichment phase could take days on large repos. codescope keeps the problem (chat with a repo) but rebuilds every layer: SCIP for IDE-precise resolution, an embedded graph DB instead of Neo4j, and an agent loop that picks among four tools per turn — so the model's reasoning is visible in the UI, turn by turn.

1. 01Index a Python repo with scip-python; parse the protobuf into Symbol nodes + CALLS edges. No LLM at index time.
2. 02Store the graph in Kuzu (embedded) and per-symbol embeddings in LanceDB (also embedded). Zero servers, zero infra.
3. 03Expose four typed tools — find_symbol, callers_of, callees_of, read_source — as the only path the agent has into storage.
4. 04Run a bounded LiteLLM loop (≤ 20 turns) with a system prompt that tells the model to verify with read_source before citing.
5. 05Stream every TraceEvent over WebSocket to a React UI that mounts trace cards in real time as the agent reasons.

The eval was four runs, each addressing one specific failure pattern from the previous. v1→v2 eliminated “confidently wrong” answers via a verify-before-cite prompt rule; v2→v3 recovered honest truncations by raising the turn budget; v3→v4 added an anti-search-loop rule and re-ranked symbol search by caller-count, landing three new wins on central high-fan-in symbols.

• →v4 (gpt-5, MAX=20, anti-loop + re-rank): 13/20 correct on hand-written fastapi questions
• →Iteration arc: 8 → 8 → 10 → 13 across four runs, each change attributable to a specific failure mode
• →Re-rank by caller-count landed three new wins on central, high-fan-in symbols (get_openapi, APIRoute, get_route_handler)
• →Failure mode shifted from 'confidently wrong' (v1) to 'still investigating, ran out of turns' (v2-v4) — the correct failure profile for a real dev tool

• ·13/20 correct on the fastapi eval with gpt-5 (v4 of four iterations)
• ·Zero confidently-wrong answers across all four eval runs
• ·Bounded agent loop, ≤ 20 turns — no fixed pipeline
• ·SCIP-precise CALLS edges, not static name matching
• ·Index time: seconds, not days — no LLM enrichment phase
• ·Embedded Kuzu + LanceDB, no servers