Your new scale can inherit the old one's calibration→
Swapping a five-bucket grade for a continuous score usually means recalibrating from scratch. It doesn't have to, if the new curve's knots are the old rubric's boundaries.
Field notes · No. 035
Dev log
Senior DevOps Engineer building in public. Shipped features across Local Fitness, Ghostwriter, and more — the tradeoffs behind them, and what broke.
The Host header can decide which path your auth check sees→
A path-based auth gate is only as trustworthy as the string you hand it. Here is how to build one on the ASGI scope, and how to prove a crafted Host header cannot move it.
Count the grades your rubric never hands out→
A scoring system can pass every unit test it has and still be wrong, because unit tests check the arithmetic and never the answer. Regrading real history and gating on the shape of the letter distribution catches the failure the tests structurally cannot.
Weighting cannot fix a score whose axes measure the same thing→
If two axes of a weighted score are computed from the same underlying input, one of them will punish whatever the other rewards. Here is how to detect that with a sweep, and why the fix is removing the axis rather than lowering its weight.
Your hallucination detector needs a control group→
A grounding check that flags almost everything is the same as one that flags nothing. Here's how to build four synthetic arms with known answers and prove your detector can actually tell them apart.
When the perf gate measures the hardware instead of the code→
A CI benchmark gate compares every PR against a committed baseline. That works until the runner fleet under it drifts. Here's the gate, the drift, and the evidence test that tells a real regression from a hardware mirage.
An agent memory archive needs a search path, not a bigger prompt→
Capping an agent's memory means archiving entries out of the prompt. This release added the other half: full-text recall over the archive with SQLite FTS5, no search service required.
Add memory to an LLM agent without breaking its prompt caches→
My fitness coach now remembers streaks, repeat excuses, and its own past reads. The hard part wasn't storing memories; it was injecting them into prompts that are cache-keyed on their own bytes.
Your model keeps saying the thing you told it not to say→
A rule in the system prompt cannot stop a model from repeating a token that is sitting in its context. Take the token out of the context, carry the meaning in a form the model is free to use, and keep a validated detector as a backstop.
Fit a report to one page by measuring it, not by guessing the CSS→
A generated PDF that overflows to a second page is a layout you never measured. Render it, count the pages, and step down a density ladder until it fits, then drop content only as a last resort and say what you dropped.
The label said running; the pace said otherwise→
Any feature that scores something against 'similar' records has to decide what similar means, and a category label is the tempting wrong answer. Here is how to build a reference cohort that partitions on a measurement instead, and discloses what it threw out.
Grading a workout without failing every easy day→
Any system that grades a user against their own history hits the same wall: the naive rubric punishes the behavior you asked for. Four design decisions fix it, and a real graded report card shows what each one is worth.
One JSON file, two renderers: theming generated PDFs and charts→
If your app generates both PDFs and chart images, the colors live in two places and drift apart. Put them in one token dict, let a local JSON file override any subset of it, and make a broken override impossible to notice.
A watchdog for LLM streams that go quiet instead of failing→
A streaming model call that stops delivering messages but never closes will hang your job with no error at all. Here is how to bound it on the gap between messages, retry it, and log the failure so you diagnose the right thing.
Let the model phrase the judgment, not derive it→
My analysis tools were handing the LLM raw numbers and a legend, then trusting it to classify. Here is how I pulled every judgment into a pure, tested Python module so the model only has to write the sentence.
Benchmarks that count connections, not just milliseconds→
One of my fitness agent's MCP tools was opening nine SQLite connections to answer a single question. The fix was routine; this walkthrough builds the part that outlives it, a pytest-benchmark harness that asserts connection counts, compares against a committed baseline, and fails any PR that regresses either.
Never let one LLM call sink a whole report→
My fitness agent's daily PDF gained a Claude-written coaching line, which put a network call in the middle of a synchronous render. Here's how to pair that call with a deterministic fallback, and how to prove the fallback works before the day you need it.
Giving an MCP agent a way to refresh its own data→
My fitness agent could read the database but never freshen it. Here is how to add a side-effectful sync tool to an MCP server, keep it gap-aware, and keep it out of the loops that should stay read-only.
Why the container synced fine while the host kept hitting 429s→
The same sync code got rate-limited on the host and ran clean in the container, and the difference was an env-var fallback buried in a client library. How to make session caching explicit and pin it with a test so it can't silently regress.
Move the deterministic half of your LLM prompt into code→
When one prompt is doing eleven jobs, most of them have a right answer and don't need a model. Here is how to split an LLM feature into a tested planner, one toolless model call, and an advisory grounding check, then prove the cutover is safe before you flip it.
Wiring up CodeQL and dependency review before a repo goes public→
Before making a repo public, wire in the two security workflows GitHub gives you free: CodeQL for the code you wrote, dependency review for the code you pull in. Setup, verification with real output, and the traps that make both silently useless.
Automating the fix for squash-merge drift on a long-lived branch→
A squash-merged dev-to-main promotion leaves the two branches diverged, so the next promotion PR shows commits that already shipped. Here's a script and workflow that resets the branch automatically, including the branch-protection dance it has to do safely.
A covered line and a tested line are not the same thing→
This release deleted a pile of confirmed-dead code and raised test coverage on the parts that were genuinely thin. The useful part wasn't the percentage, it was catching the tests that were only pretending to check something.
Chaining a release to CI's own success, not a raw push→
How to wire a GitHub release off a green CI run instead of a push, guard it so an already-shipped version can't double-release, and the default-branch trap that catches anyone wiring workflow_run for the first time.
A settings resolver that always falls back to what already worked→
Five coaching preferences were hardcoded into a public repo as if they were universal truths. Turning them into user settings meant building a small resolver: one precedence order across three config layers, where every bad input falls back to the value that already worked instead of raising.
Build a status grader that never gets confused by today→
A training-plan feature graded today's finished workout as still pending, because the code asked the calendar instead of the outcome. Here's how to build a status grader that reads outcomes first, holds judgment only where the day is genuinely unsettled, and hands the UI one verdict it never has to second-guess.
Live MCP server instructions that survive a fresh clone→
An MCP server can hand its client a system-prompt hint at initialize. This build wires that field to a SQLite setting, resolves it fresh on every connect, and fails open when the read misses, so a config change lands without a restart and a fresh clone still starts.
A prompt dial isn't a setting until code reads it too→
My fitness coach's brief had one hardcoded voice. Making it four selectable tones was the easy part; the real work was making sure a numeric 'harshness' dial did something a test could catch, not just something the model might notice.
Why your agent's tool needs to say pending, not missed, when data lags→
My fitness agent's plan tool needed to tell the difference between a workout that failed and one my Garmin sync simply hasn't caught up to yet. Here's how to build a single agent tool that grades real-world state against a data freshness boundary instead of the calendar.
The setting that fixed a slow AI brief was hiding behind one that silently did nothing→
A Claude-generated daily fitness brief went from about four minutes to under 90 seconds, but not from the parallel rewrite I designed. Here's how to gate a fix behind a kill criterion, find out which agent settings are real versus silently inert, and confirm the trade with a blind judge instead of a hunch.
Give your MCP server a write tool, then delete the model from it→
Once an MCP server can write back as well as read, the model has no reason to run inside it anymore. Here's how to build that write tool, and the exact way a 'read-only' guard fails if you enforce it with a keyword check instead of at the connection itself.
Reuse your Claude Agent SDK tool server as a standalone MCP endpoint→
I exposed local-fitness's in-process Claude agent tools to real MCP clients over HTTP and stdio without writing a second copy of them, then closed the three gaps a network-reachable localhost server actually has that a subprocess-launched one doesn't.
Make the agent's only write a draft, and let the database enforce the rest→
I gave an AI agent its first real database write path by making the dangerous action unreachable instead of trusting it to behave: the tool schema never accepts a status field, and a partial unique index backstops the single-active-row rule even if the application code has a bug. Here's how to build the same draft-only boundary for any agent that writes to your own data.
Score your prompt against the types your code already trusts→
I turned a personal Garmin-coaching script into real software: a pytest suite behind a coverage gate, CI wiring, and a scorer that cross-validates the agent's prompt against the typed contract its own output has to satisfy. Here's how to build that check for any prompt-driven agent.
