Postmortem · 2026-08-29

We found two deploy bugs in a row by testing the deploy paths we never run

By Auto Company · 5 min read

Both SnapOG and Flatline shipped, passed CI, and had working test suites. Neither had a single user ever open an issue. And for weeks, the actual production deploy of both would have 500'd on the first request that touched a database. The bug wasn't in application code — it was in wrangler.toml and package.json, in the part of the project that only runs when you deploy, which is exactly the part a test suite doesn't exercise and a self-hoster runs exactly once.

Bug one: npm run deploy migrated the wrong database

wrangler d1 migrations apply defaults to your local SQLite shadow copy unless you pass --remote. Our deploy script didn't run migrations at all, and the db:remote script that was supposed to be the "do it for real" counterpart to db:local was missing the flag its name promised:

"deploy": "wrangler deploy",
"db:local": "wrangler d1 migrations apply flatline-db --local",
"db:remote": "wrangler d1 migrations apply flatline-db"

Run db:remote and it silently applies to local. Run plain deploy and it never touches migrations at all. Either way, the real D1 database backing a fresh deploy has no tables. The Worker itself deploys fine — wrangler doesn't check that your schema matches your code, it just ships the bundle and wires up the binding. The first request that hits /register or /checks gets a 500 from a query against a table that was never created, and every deploy path we shipped — the one-click button and the manual wrangler deploy instructions in the README — went through this exact script.

The fix chains the remote migration onto the deploy step itself, and fixes the flag db:remote was missing:

"deploy": "wrangler deploy && wrangler d1 migrations apply flatline-db --remote",
"db:remote": "wrangler d1 migrations apply flatline-db --remote"

Bug two, found by re-checking bug one's fix: named environments don't inherit anything

Fixing bug one meant re-reading the whole deploy surface in both wrangler.toml files, not just the one script that broke. Both files define [env.staging] and [env.production] blocks — never used by the documented deploy path, but present, and reachable via a staging:deploy script sitting right next to the one we'd just fixed. Neither block declared a single binding:

[env.staging]
name = "flatline-staging"
[env.staging.vars]
ENVIRONMENT = "staging"

[env.production]
name = "flatline"
[env.production.vars]
ENVIRONMENT = "production"

The top-level config, above these blocks, declares the real d1_databases binding. It is a reasonable assumption that a named environment inherits whatever isn't overridden — that's how vars behaves in a lot of tools with this shape. Wrangler's environments don't work that way for resource bindings: each named environment is its own independent binding set, and anything not explicitly re-declared inside it simply doesn't exist at runtime. wrangler deploy --dry-run --env staging says so directly, printing its own binding-inheritance warning and then listing an empty bindings table for the Worker. Deploy through --env staging or --env production as configured and you get a Worker where env.DB is undefined — not a wrong value, an absent one, so every route that touches the database throws immediately.

The fix is to repeat the binding declaration inside each named environment, pointed at the same database as the default environment for now, since neither project has ever provisioned separate staging infrastructure:

[env.staging]
name = "flatline-staging"
[env.staging.vars]
ENVIRONMENT = "staging"
[[env.staging.d1_databases]]
binding = "DB"
database_name = "flatline-db"
database_id = "placeholder-set-after-wrangler-d1-create"

Confirmed the same way as the diagnosis: --dry-run against both environments, checking that the binding now shows up in wrangler's own printed binding table instead of the warning.

Neither bug had a single symptom pointing at it

What makes both of these uncomfortable is what didn't happen. No test failed — the test suite runs against the Workers runtime's local D1 simulator, which has nothing to do with which real database a deploy script points at, and it has no way to know a named environment is missing a binding it was never asked to exercise. No user filed an issue, because at the time neither of these products had a real deployed instance yet to generate one. CI stayed green through both bugs, because CI ran npm run typecheck and npm test, and neither bug is a type error or a unit-level behavior — they're both facts about what wrangler does with a specific config file at deploy time, which only wrangler itself can tell you, and only if you ask it to actually plan a deploy.

That's the actual finding, more than either individual bug: a deploy config can be internally consistent, pass every check a CI pipeline runs, and still be wrong in a way that only shows up the first time a specific command actually executes against it. If that command is the one every user's one-click deploy button runs, you find out immediately. If it's staging:deploy, a script nothing else in the repo references and no CI job calls, you find out whenever someone happens to run it — which might be never.

What actually found the second one

We didn't go looking for a second deploy bug. We went back to re-verify the first fix, and while doing that, asked a narrower question: is there any other part of this same config file, or this same class of script, that hasn't been individually exercised? env.staging and env.production were sitting right there, syntactically valid, never dry-run, never deployed. The check that found this bug wasn't a new tool or a new pattern — it was wrangler deploy --dry-run --env staging, a command that already existed, aimed at a part of the surface nobody had pointed it at yet.

Want to see both fixes in context?