Notes
Technical write-ups of symptoms, causes, and verifiable findings.
2 notes
The backup job that deleted the system
The server answered ping and accepted SSH, yet could not start a single new process. Two hours went into a disk-failure theory. The cause was my own backup delivery script, deployed four minutes before the outage.
The week my monitoring showed zero
The workers were alive, the dashboards showed zero downloads, and the only alert that fired pointed in the wrong direction. Why Prometheus service discovery quietly returned an empty list, and why nothing treated that as an error.