Long-term memory for an AI coding agent
Indexes every past session into Postgres, scores them with a local LLM judge, embeds the
keepers, and injects the relevant ones back into new sessions automatically. Retrieval is
vector search fused with full-text by Reciprocal Rank Fusion. A separate audit layer
extracts testable claims from past answers, checks them against the filesystem, and flags
the false ones on recall — the system catches its own hallucinations.
6,592 sessions650,953 turns
pgvector HNSW · 1024-dimRRF hybrid retrieval
Python · Postgres · Ollama
Autonomous multi-agent coding loop
A local model works a backlog across parallel project lanes overnight while a frontier
orchestrator reviews and promotes the work. Orchestrator and workers are strictly separated —
workers verify but never write. A verdict only counts if it carries an exit code, a quoted log
tail and a file reference, so unproven passes get demoted. Includes atomic task claiming with
stale-claim recovery and three-strike escalation to a human.
2,638 commits8 parallel lanes
9 weeksEvidence-gated verification
Multi-tenant AI layer for a tour company in New Zealand
A production platform where every AI task defaults to a local model and only routes to a
frontier provider when a tenant admin explicitly opts that task in. Before anything can leave
the box, a local NER model running in an isolated subprocess finds and substitutes personal
data, which is rehydrated on the way back. Natural-language reporting compiles to SQL behind a
safety guard, with a stored-query fast path.
53 AI-backed API routesConsent-gated routing
Local PII substitution
Next.js · Postgres · ONNX
Private OpenAI-compatible inference service
A drop-in replacement for OpenAI in a production deployment: the legacy platform points at it
unchanged. Four task-specific skills are routed by the requested model name, with no silent
fallback on an unknown one. The SQL skill is grounded with few-shot examples retrieved by
cosine similarity from an embedded corpus, and degrades gracefully if the vector store is
missing. Zero third-party dependencies.
225 embedded query pairs4 routed skills
Zero dependenciesRuns as a managed service
Model evaluation harnesses
Unattended bakeoffs that gate adoption on the things that actually break an agent loop. A
candidate 35B model was disqualified outright for failing tool-calling before it ever reached
production; survivors were throughput-benchmarked on identical hardware. Results include an
honest limitations section and a documented negative result rather than a buried one.
Tool-call gatingSame-hardware benchmarks
Negative results published
Synthetic training-data pipeline
Generates natural-language→SQL training pairs by wrapping the live production system prompt
and schema around a local 35B model, so every pair is in the exact dialect the real system
expects. Generated SQL is validated against an in-memory database before it’s kept, and prompt
changes are scored as pass-rate against a frozen validation set.
~9,000 validated pairsSQL-validated on generation
Frozen-set scoring