You will not believe the trick-or-treat trick Microsoft just pulled!

[rysiek@mstdn.social]

This might be a good time to consider deploying, contributing to or otherwise supporting @nextcloud:
https://nextcloud.com/

And before anyone says that Nextcloud's UI/UX is lacking: of course it is! Nextcloud has several orders of magnitude less money to throw at UI/UX.

But guess what:
1. this is fixable if they get more resources to work with;
2. every single Nextcloud instance I know of or use (there are many) stayed up and running yesterday.

#Microsoft #Azure #Outage

[rysiek@mstdn.social]

Cascading failure. It cascaded.

> As unhealthy nodes dropped out of the global pool, traffic distribution across healthy nodes became imbalanced, amplifying the impact and causing intermittent availability even for regions that were partially healthy.
https://azure.status.microsoft/en-us/status/history/#incident-history-collapse-YKYN-BWZ

#Microsoft #Azure #Outage

[rysiek@mstdn.social]

> An inadvertent tenant configuration change within Azure Front Door (AFD) triggered a widespread service disruption…

> The trigger was traced to a faulty tenant configuration deployment process. Our protection mechanisms, to validate and block any erroneous deployments, failed due to a software defect which allowed the deployment to bypass safety validations.

Am I reading this right? Global – pardon, "non-regional" – #outage of #Microsoft #Azure was caused by a tenant changing their config? 👀

[rysiek@mstdn.social]

Can't have a snow day without cloud. 🌨️

[rysiek@mstdn.social]

These massive "cloud" outages will continue to happen, inevitably, because they are "normal accidents":
https://en.wikipedia.org/wiki/Normal_Accidents

Any system that:
- is complex,
- is tightly coupled, and
- has catastrophic potential

…can be expected to experience catastrophic failures. AWS, Google Cloud, #Microsoft #Azure check all these boxes.

Fun fact: Internet was specifically designed to *not* be like that. It was designed to be loosely coupled.

But that's not as good for maximizing shareholder value!

[spytfyre@mastodon.scot]

@rysiek software defect
I think it was any tenant change by my reading not one single idiot. They're not pinning this on tenants users but it is written that way so you can misinterpret it that way and blame Microsoft less
Software defect is there to point at that they didn't blame anyone they blamed the software
They're shying away from blaming skynet so it doesn't fire them next

[svelmoe@hachyderm.io]

@rysiek I know I'm only an average 'in the trench' type developer, and my interests have always been code over servers/hosting........

But I cannot express how much I miss the "old days" of normal web-servers and where focus was on delivery of (customer/user) value and optimisations and not just bloating everything with external services sold into a cloak of 'critical infrastructure' and 'pass/sass' cloud stuff.

[mkj@social.mkj.earth]

No matter how you slice it, as I see it, it boils down to: Microsoft's software did insufficient input validation on untrusted data (specifically in this case configuration changes for one customer, it seems), and this led to widespread outages affecting many, many, *many* customers in many different ways.

Security done properly should not rely on "blocking erroneous" stuff, but should verify that everything is good and only *if* that succeeds allow any of it through.

@spytfyre @rysiek