How I use a BlackOps brain to track my deploys

I use BlackOps to keep the record of my own work, and last night I pointed it at my deploys.

Setting that up was one sentence in a chat. I asked for a brain called Deployment Log and said what it was for. A brain is a set of markdown notes the agent keeps and can read back later, so there was no schema to design, no table to create, no dashboard to configure. The shape of the thing is whatever gets written into it.

Then I had it backfill 34 deploys out of the Vercel build logs, which took about a minute, and it told me something I did not know about my own week.

Two builds died on the same thing

On Thursday a build failed while prerendering the sitemap for my guides section. The error was a database timeout, which sounds like a slow query and is not one. That query runs in 0.171ms over 79 rows. What runs out is the connection pool. The build renders about a thousand pages at once, enough of them hit the database in the same moment that requests queue past the eight second statement timeout, and whichever page draws the short straw takes the entire build down with it.

I measured all of that, wrote it up properly, and shipped a fix 108 minutes later.

Saturday night, a build failed again. Same timeout, same guides section, different page. A guide this time rather than the sitemap.

Nobody noticed for 25 minutes

The pull request checks were green, so I had already moved on to the next task. Checks run against the branch. They say nothing about whether the production build worked after the merge, and Vercel does not retry a build that fails. It sat there until the next merge happened to build clean and quietly replaced it.

The site never went down, for the boring reason that a failed build is not promoted. Production kept serving the previous deployment. What I lost was the ability to deploy, plus 25 minutes of not knowing.

This is what I get now when a build dies. It did not exist on Saturday, so nobody sent me this one. It is generated from that build's own logs, which is why it can tell me where the build stopped and take a guess at why.

The line I care about most is the one saying production is still serving the previous deployment. At 1am that is the difference between a problem and an emergency.

So I wrote a watcher. Checks, then merge, then the production build for that specific commit, then a look at the live page. Any developer reading this could write that in an afternoon, and that is sort of the point. It was the easy half.

Thursday was in there and I had forgotten it

The backfill turned up the Thursday failure, which I had no record of anywhere.

Sitting in a table next to Saturday's, the pair said something neither one said alone. My Thursday fix had gone into a single file, the sitemap route. The queries underneath it, the ones every guide page also calls, I left alone. So the same failure came back two days later through a different door.

On its own, Saturday looked like bad luck. Next to Thursday, it was a fix I had scoped too small.

There is a second thing the table told me and it is less flattering. I had blamed the failures on a layout refactor I shipped that week, reasoning that it pushed more pages into every build. Both halves of that were wrong. Thursday came before the refactor, and the refactor had cut the page count rather than raised it. The builds go 1068, 1058, 1048, 947, 945 down the column. I had been confidently wrong for about an hour, in writing, and the numbers sitting in one place is what corrected me.

What I actually use it for

Four things, and none of them are work I do.

Every deploy writes its own note, and the agent that ran the deploy is the one that writes it. It is already there and already knows the commit, the build time, how many pages rendered and what the error said, so nothing gets retyped later or rounded off from memory.

I get an email with the same report as each deploy finishes. Here is a real one, from the deploy that shipped the fix for all of this.

That is the part that would have saved me the 25 minutes.

I ask it questions in plain language. When did builds start failing. Has this happened before. What was the last commit that shipped clean. Those are the questions I actually had at 1am, and before this I had no way to answer any of them.

And when I learn something later, it goes back into the note. Thursday's entry now says what Saturday taught me about it, so the next person to look, which is usually me in three weeks, gets the conclusion and not just the wreckage.

The Deployment Log also sits next to the other brains in the same account. My client work log, the product brain for this thing, notes on wine I like. Same system, same way of asking. I did not buy a monitoring tool and bolt it onto a notes app and wire a status page between them.

The script is not the valuable half

If you build software you can write the watcher. You probably have. That part is a shell script and an afternoon.

The part you will not do, and I say this as someone who did not do it, is keep the record. Not while the build is red and you are trying to get a deploy out. Nobody writes the incident note at 1am, and if they do they never open it again.

Andrej Karpathy landed on the same shape and wrote it up as an LLM wiki. Raw sources stay put and never get edited, and the model owns a layer of markdown files on top of them. It writes the pages, updates them when new material shows up, and keeps the cross references straight. His reason for handing that layer to a model is the one I just gave you from the other end. People abandon wikis because the upkeep grows faster than what the wiki gives back. A model does not get bored on the fifteenth file.

He is at Anthropic now working on pre-training. I have a deploy log and a bad night. Same conclusion from opposite ends of the problem, which is the part I find convincing. Nobody talked anybody into this.

A brain is that idea with the plumbing already done. Markdown files I own, written and maintained by the agent that did the work, sitting where I can ask them questions.

That is the whole reason I use BlackOps. Not because I cannot write scripts, but because the writing down is the part that quietly does not happen, and it turns out that is the part that was catching my mistakes.

What I would tell you if you asked

I would not oversell the fix. The retry work I shipped reduces the failure. It does not remove it, and one case still fails the build on purpose, because the alternative is quietly publishing a 404 for a guide that exists. A loud failure beats a silent lie.

But the brain has already paid for itself once, in the shape of a fix I made twice because the first one was too narrow and nothing in my setup was in a position to tell me.

Point one at your own work at /start.