Skip to content
diggn.de
← all posts CI & deployment · · 3 minutes

The bolt that wasn't on the door

Before publishing, the pipeline checks whether the legal pages are still placeholders. The check was correct, had twelve green tests, and never ran.

The Impressum and privacy pages on this site are unfinished on purpose: square brackets instead of text, because a generator or a law firm has to write them and I shouldn't. "Before launch" is exactly the phrase that survives until launch, though, so instead of a note there's a bolt on the door. The pages job checks before it publishes.

What the pipeline said

.gitlab-ci.ymlYAML
script:
  # Refuses to publish while the Impressum and Datenschutz pages are placeholders.
  # "We'll fix it before launch" is exactly what survives to launch, so this is a
  # gate rather than a note. See issue #3.
  - node tools/check-legal.mjs
  - echo "Publishing the build produced in the build stage."

The comment promises what the job is for. The line under it doesn't do that. It runs the file, which is not the same as running the check: tools/check-legal.mjs is a module. It exports functions and constants and calls nothing itself. Its first line being #!/usr/bin/env node makes the mix-up easier, not harder.

Shell
$ node tools/check-legal.mjs
$ echo $?
0

No output, exit 0, pipeline green. Nothing was ever published regardless, because the job is when: manual and nobody clicked it. The bolt was open; the door stayed shut. The entry point is tools/cli-check-legal.mjs, wired up as npm run check:legal. Call that one and it looks rather different: exit 1, and four files carrying 38 placeholders between them.

The twelve green tests

The typo isn't the unsettling part. What's unsettling is everything that was true at the same time.

The check has twelve tests and they all pass. Two of them call runLegalCli itself rather than the inner function: one asserts that it exits non-zero and names the unfinished file, the other the reverse case, that it stays quiet once nothing is left open. A third is called

✔ the real repository still has placeholders, so the deploy is blocked today

and is wrong all the same, at least in its second half. The placeholders were there — that part held. Nothing was blocked.

A test can prove a bolt works and say nothing about whether it's on the door.

What I take from it

The unit under test was fine. What was wrong was the single line that calls it, and no test, type or linter checked that line. Four characters separate check-legal.mjs from cli-check-legal.mjs, and both files exist, so nothing complains.

What would have caught it is cheaper than another test of the function: run the command exactly as the job runs it, in a state where it has to fail. A bolt you've never pushed against a closed door is a guess, like a backup nobody ever restored, which I wrote about in German.

The job calls npm run check:legal now. While fixing it I did the thing that had been missing: ran that same command against the repository with the placeholders still in place. Exit 1, four files, 38 placeholders. I know the bolt holds, because I pushed against it once.

#ci#gitlab#tests

Related