Spent my morning figuring out why Nginx was dead on a server with many days of uptime.
Senza categoria
24
Post
9
Autori
2
Visualizzazioni
-
Spent my morning figuring out why Nginx was dead on a server with many days of uptime. No reboot, no kernel panic. Just... down. Ubuntu 24.04.
The cause? An automatic unattended-upgrade of libc6. This prompted systemd to work its magic, wisely deciding to restart every running service to apply the patch. Fine.
The problem is, in the exact same minute, the systemd timer for certbot decided it was time to renew certificates.
The result:
- systemd stops Nginx.
- Port 80 becomes free.
- certbot, in standalone mode, immediately grabs it for validation.
- systemd tries to restart Nginx, which fails with "Address already in use".The web server was knocked offline by its own certificate renewal script.
I swear, this is the kind of cascading failure that has never happened to me in years of running *BSD. With a classic cron job, certbot would have failed, logged an error, and tried again the next day. The web server would have remained untouched.
systemd was doing its job, but something failed because of the interactions.
Sometimes, too much automation and too many interconnected parts just create more spectacular ways for things to break.
@stefano How did you even trace this down??
-
@chebra looking carefully at the logs and studying the timing and inteactions
-
@chebra looking carefully at the logs and studying the timing and inteactions
@stefano Oh I need to step up my logging game... a lot...
-
@chebra for me, logs are critical. They saved my life so many times...
Gli ultimi otto messaggi ricevuti dalla Federazione
-
@gureito if the question mark is intended to imply "how do you not know this" I'm not in the habit of letting tech founders live rent free in my head
if the question mark is intended to mean "does this information line up contextually with all the vagueposting" fuck if I know
-
-
@andre123 Io lo uso da tempo, sia con un profilo email (creato apposta per non avere problemi) che un profilo chatmail fornito da loro. Funziona benissimo, peccato che sia quasi totalmente sconosciuto.
-
con #AlessandroOrsini abbiamo dato il via a una rubrica settimanale:
si chiama #SicurezzaInternazionale (o meglio #InsicurezzaNazionale!) sul canale YT di #FattoQuotidiano...
Ecco la prima puntata:
-
@informapirata @politica no, quello no :)
-
@vosje62 @UndisScot
I don't know.
But I'm sure that Scots must never ever be underextimated. Never.
-
@PaoloParti non entro nel merito, perché non sarebbe opportuno, ma mi limito a dire che su questa canzone la penso in modo diverso da te 🙉
Però in fondo chissene! Sono comunque felice per il tuo entusiasmo 🤣🤣🤣
-
@phab @GustavinoBevilacqua se per prendere la mira devono comunque rallentare allora funzionano, no?
