got so mad i wrote 1000 words on how we should build a reliable background job system

[tef@mastodon.social]

got so mad i wrote 1000 words on how we should build a reliable background job system

(hint: it's not using a message broker)

for work, again, as i am cursed to repeat this folk knowledge in each and every company i work at

[tef@mastodon.social]

ended up linking to my own posts under "anti-message broker propaganda" like i'm a real blogger

[tef@mastodon.social]

"here's a technical report i posted to tumblr" every day i am haunted by my earlier choices in life

[ceejbot@toot.cat]

@tef I just built one. It uses postgres. I had you in mind.

[odo@dustbuster.club]

@tef got a link? I’d love to read it.

[artemist@social.mildlyfunctional.gay]

@tef one of the projects i'm working with at work handles jobs by putting them in postgres, and when messages specifically need to be passed they use notify

[artemist@social.mildlyfunctional.gay]

@tef unfortunately, the project is written in perl

[soni@discuss.systems]

@tef I built a similar system at work and your writing on this topic was very insightful!!

[tef@mastodon.social]

@artemist ah, listen notify

you know like it doesn't send them until transaction commit and all sorts of other "haha fuck you" like behaviour

i haven't seen someone build a working system out of it

[tef@mastodon.social]

if you want to know how to build a reliable background job system, i already wrote a very tongue in cheek post about it

https://web.archive.org/web/20250107135013mp_/https://cohost.org/tef/post/1764930-how-not-to-write-a

the tldr? process supervision, operators, sagas, nothing that fancy, nothing that new or special

[eschaton@mastodon.social]

@tef If only common operating systems included such things. The VMS queue system is wonderful, I’ve leveraged it to do things like keep the time close to correct on my SIMH instances.

[wirepair@mastodon.social]

@tef i remember reading https://web.archive.org/web/20250111083106mp_/https://programmingisterrible.com/post/162346490883/how-do-you-cut-a-monolith-in-half and fundamentally changing how i developed software. Thanks for that, was such an enlightening article.