All topics

Tagged "ops"

7 posts under this topic.

The disk that filled itself

Homelab box said 100 percent full, du said half the disk was free. Docker was holding open a log file I had rm'd months ago. lsof +L1 untangles the knot.

debugginglinuxdisk-fulldockerself-hostedops

The webhook that worked in Postman and nowhere else

Two bootstrap paths in one app drifted apart. The HTTP entrypoint loaded the webhook signing secret, the queue worker did not, and a silent-skip guard hid the cause behind a misleading 401.

debuggingwebhooksbootstrapqueue-workersfail-fastops

I blocked Tor exit nodes, then I opened Tor Browser

I deployed a hardened Tor exit node firewall on a SaaS production box, opened Tor Browser to confirm, and the site loaded. The IPv4 fortress was perfect. The IPv6 side door was wide open. This is the script, the punchline, and the rewrite that became TorShield.

toriptablesipsetipv6linuxsecurityops

The node_modules That Wouldn't Die

An internal app deploy kept importing an old Vite plugin export. The lockfile was right, the build was wrong. The culprit was older than the bug.

dockerdeploymentgitci-cdops

The Sentry signup nobody could finish

A colleague signed up on our self-hosted Sentry and never got the email. I had been getting Sentry mail forever, so I assumed he was missing something. He was not. DMARC was silently dropping every message to our Workspace inboxes, and the 'workaround' I shared with him broke too. Here is the bug, the lie my inbox had been telling me, and the shell command that finally got him in.

sentryself-hosteddmarcsmtpops

The sed that didn't stick

A failing nightly backup, a sed hotfix that worked once, and the next morning's cron that failed anyway. Node's require cache had eaten my patch.

dockernodebackupopsopen-source

The day I realised I had never tested a production backup

I had been running test-backup drills for months and felt covered. But I had never pulled a real production snapshot into a lab and restored it. The afternoon I finally did is the story behind this post.

backupdocker-composedatabasesopsopen-source