Doc Detective dominos

Sometimes you start one thing, and it tips into another thing, and then another and another.

My first Doc Detective test for this site was like that. And honestly, I am glad. If it had gone smoothly, then I wouldn't have learned nearly as much.

My concept was pretty simple: Create a test that shows how Doc Detective can update screenshots.

It seemed like the Google doodle would be an obvious and easy use case.

• It's public.
• It updates every day.
• Everyone knows about it.

So I wrote step-by-step instructions, took a screenshot and downloaded the doodle GIF, and then told my robot minion (Claude Code) to help me write the tests.

Mistakes

First mistake: I did not tell it to use the most recent version of Doc Detective. I'd heard this could be a problem with coding assistants; that if you do not explicitly instruct them to use the latest, then they might not. In this case, it didn't.

I realized the issue when I ran Doc Detective on the tests locally and it failed. So I had the minion update the tests, tested again, and was successful.

Second mistake: I did not understand who I was writing instructions for. I wrote instructions for humans. The testing agent is not a human.

I had instructions like, "Navigate to google.com" and "Observe the doodle," as one would for a human. That doesn't work for the bot.

If I hadn't had Manny Silva demo the functionality for me a while back, then I wouldn't have thought to ask Claude Code why it was writing tests in a separate file rather than inline, as I had seen in the demo.

Turns out that when you write for a human, the robot has trouble parsing instructions. They just aren't specific enough for it to use the human-readable steps as a basis for tests. As a result, the tests were written to a separate file.

And that, for the record, is totally valid. For this use case, I wanted to try the in-line tests. Claude Code helped me rewrite the instructions so that the testing agent would understand them, which meant we didn't need a separate testing file.

Workflow updates

After all of that, I ran my regular workflow. Vale, bless its little heart, dutifully complained that I hadn't spelled out my acronyms: GIF and PNG.

So since I don't want it to bug me every single time I use those acronyms, I had Claude update Vale to not apply the rule to GIF, PDF, PNG, or JPG.

And then it turned out I somehow hadn't installed Vale or codespell locally, only on the remote repository, so I had to install those and told Claude to make sure the local and remote Vale configs stayed the same.

Once all of that was finished and merged...it didn't work. So I began to troubleshoot.

Troubleshooting and fixing

Here's a partial list of things that went wrong before it worked:

• The schema I was writing tests against was two major versions out of date
• A sharp/libvips dependency was silently missing from global npm installs in CI, causing cryptic ERR_DLOPEN_FAILED errors across five separate debugging sessions
• The GITHUB_TOKEN that GitHub Actions uses can push commits, but those commits deliberately cannot trigger other workflows — a loop-prevention feature I had never had to think about before
• The workflow was running at midnight Pacific time, before Google had actually updated the doodle for the day
• Two workflow runs fired at the same time and one of them rejected the other's push

Each one of these felt, in the moment, like a dead end.

What eventually worked: upgrading to the latest version of Doc Detective (which bundled its own sharp), adding a workflow_run trigger so the deploy fires after the doodle commit, and moving the schedule to 2 AM — which Google has reliably beaten.

Dominos

How many dominos did I tip over in this workflow?

Let's see...

• Told Claude Code to always use the most recent version of libraries and tools.
• Installed a couple tools that I missed installing locally.
• Updated Vale configs. Again.

Mind you, this workflow is still pretty immature. As I continue to iterate and improve the site, I expect that there will either be fewer workflow dominos that fall over with each new change, or at least, they will be new and different ones.

The bigger lesson? Automation doesn't save you from debugging. It just means the debugging happens at scale, in CI logs, instead of on your own machine.

But when it works? The page updates itself while I sleep. That still feels like magic.