Agentic Chess Coach, Kasra Zamanian

Setup

My chess goes back to age seven, when my father taught me the moves. It was an on-and-off relationship for years, with no real study of the fundamentals and not a single official lesson. When I finally came back to it through Chess.com lessons, I took my rating from 800 to 1800 in about two years of dedicated puzzles and opening study. But I always felt I was missing the fundamentals: annotating my own games as I play, evaluating positions strategically, recalling a game move by move in notation (e4 e5, Nf3, and so on). I had never built any of that.

At the same time, I was increasingly annoyed at Chess.com for steadily raising prices and pushing even basic features, like puzzles, behind the paywall. So one day I decided I would build my own coach. And, being me, I went all in.

I didn't want a chatbot that runs a game through Stockfish. I wanted a full coach: one that assesses all of my games, finds my recurring patterns and flaws, and architects a full year of teaching material. It processes entire chess books and retrieves drills, games, and lessons from them. It advises on the drills I built into the platform as features. It remembers every one of our past interactions. And it is prompted not to give in to my short-term desires.

Why it was hard

Demos are easy. Agentic systems that still hold up after the third edge case are not. From the first commit I treated this as a product I would have to defend in a senior-level review, not a weekend toy. Every part of the agentic workflow had to be done properly, with real evaluation and monitoring, not just a prompt and a hope.

The curriculum is data, not vibes

What happened: A coach that bends to the student's mood is not a coach. The risk with an LLM is that an off-topic or too-advanced question quietly rewrites the lesson plan, or that my own short-term wishes ("skip the endgames, just show me openings") derail a deliberate curriculum.

The fork: Let the model adapt the plan conversationally (flexible, but spineless), or make the curriculum a fixed artifact the model cannot touch.

My call: The curriculum is data. The coach follows a deliberate 12-month plan owned by the orchestrator. Off-topic questions get acknowledged and redirected; they do not mutate the doctrine. The LLM is a participant in the lesson, never its author.

What it led to: A coach with opinions and a backbone, one that keeps me on the path I set in a calm moment instead of the one I want in a lazy one.

Why I skipped LangChain

What happened: The reflex for an agentic system is to reach for LangChain. But LangChain is optimized for LLM-driven control flow, autonomous agents that decide what to do next. My coach is the opposite: a deterministic state machine that I own, with the LLM as a narrowly-scoped subroutine at specific decision points.

The fork: Adopt LangChain (and either bury my state machine inside it or run two competing paradigms in one codebase), or use something lighter and keep orchestration in my own code.

My call: Pydantic AI. It does one thing well, structured LLM output as validated Pydantic models, and leaves orchestration, doctrine, reliability, and observability where the interesting engineering lives: in my code.

What it led to: An architecture I can actually defend in a review, with the state machine and the doctrine front and center instead of hidden behind a framework.

The stack

The architecture is boring on purpose. Boring scales.

Backend: FastAPI + Python, with python-chess for the rule layer.
Engine: Stockfish 17 as a UCI subprocess for evaluation; Lichess-hosted Syzygy tablebases for endgame truth.
LLMs: Claude on Azure AI Foundry (the Microsoft and Anthropic partnership), tiered three ways:
- Haiku 4.5 for tight loops: drill feedback, classifications, short rubric calls.
- Sonnet 4.6 as the workhorse: game annotation, Socratic dialogue, mistake categorization.
- Opus 4.7 surgically only: session planning, monthly assessments, curriculum updates.
LLM framework: Pydantic AI. Model-agnostic, structured-output-first, none of the LangChain bloat.
Voice: Azure OpenAI GPT Realtime over WebSocket (audio in, audio out).
Storage: SQLite, single user, local-first. Repertoires live in PGN trees, not a hand-rolled schema.
Secrets and config: Azure Key Vault via DefaultAzureCredential; YAML for everything non-secret.
Content sources: Lichess puzzle DB (local CSV), Lichess opening explorer (HTTP), and uploaded chess books parsed into a dual-stack RAG (sqlite-vec locally, Azure AI Search as the comparison).
Frontend: Next.js + React + TypeScript, chessground for the board, chess.js for parsing.

Built like an engineer, not a tinkerer

Past the doctrine and grounding above, the rest of the discipline is what makes it hold up:

Reliability is a feature. Idempotent external calls, retries with exponential backoff, per-integration circuit breakers, durable state transitions. A network blip on Azure cannot corrupt a coaching session.
Observability from line one. Every LLM call, tool call, and state transition writes a structured log with session_id and trace_id. "Why did the coach recommend this six weeks ago?" is an answerable question, not a guess.
Measurable quality at every layer. Chess is unusually measurable: outcome metrics (puzzle rating, drill accuracy), rubric-scored content (annotation quality, Socratic depth), and system health (latency, cost per session, refusal rate).
Clean layering. Agent → Domain → State → Integration, dependencies flowing downward only, enforced by an import-linter contract. Swapping a model or a vector store is a one-file change, not a ripple.
Tiered LLMs, because cost is a design constraint. Roughly 85% of calls go to Haiku, 12% to Sonnet, 3% to Opus: the same agentic capability at a fraction of the bill, and every routing decision is logged so the spend is auditable.
Architecture Decision Records. Every non-obvious choice (Foundry over OpenRouter, Pydantic AI over LangChain, SQLite for a single-user system) is written down with its rejected alternatives, so the reasoning can be reconstructed later.

Takeaways

An agentic system is mostly the engineering around the model, not the model itself: doctrine, grounding, reliability, observability, and evaluation are where it lives or dies.
Ground everything verifiable. The LLM teaches; the engine holds the truth.
Match the tool to the control flow. A deterministic state machine you own, plus a lightweight structured-output library, beats a heavyweight agent framework when you are the one deciding what happens next.