Skip to content

Piero Bosio Social Web Site Personale Logo Fediverso

Social Forum federato con il resto del mondo. Non contano le istanze, contano le persone

This morning, a dev I work with flagged a server that was struggling.

Uncategorized
4 3 14
  • This morning, a dev I work with flagged a server that was struggling. I logged in and, unfortunately, a bunch of "Scrapers" were hammering it, causing an overload. This dev is very sharp and understood the situation. He started extending the caching margins. I prepared a list of countries of interest and, using pf, blocked the rest of the world.
    The load plummeted, and the client doesn't care about their products being visible outside of specific markets.

    Shortly after, a similar situation on another server I manage (subcontracted, to be precise). I didn't know this dev and saw there was no caching in place, so I spoke to him and asked him to implement it. His response, sadly predictable, was that another one of his clients just moved to $CLOUD_PROVIDER and no longer has these problems, simply paying more when these "attacks" become more intense. I suggested he try enabling caching and thinking about their markets of interest. He's a positive person, so he agreed to give it a shot. A short while later, just by adding Varnish and some good VCL, the load dropped from 100% to 5%. Without any blocking.

    He called me a little while ago, incredulous and happy.

    It's always a pleasure to work with people who are willing to experiment, explore, and listen. With people who don't just follow the herd or ad banners, but who think for themselves. I've earned myself a walk outside the office now.

  • This morning, a dev I work with flagged a server that was struggling. I logged in and, unfortunately, a bunch of "Scrapers" were hammering it, causing an overload. This dev is very sharp and understood the situation. He started extending the caching margins. I prepared a list of countries of interest and, using pf, blocked the rest of the world.
    The load plummeted, and the client doesn't care about their products being visible outside of specific markets.

    Shortly after, a similar situation on another server I manage (subcontracted, to be precise). I didn't know this dev and saw there was no caching in place, so I spoke to him and asked him to implement it. His response, sadly predictable, was that another one of his clients just moved to $CLOUD_PROVIDER and no longer has these problems, simply paying more when these "attacks" become more intense. I suggested he try enabling caching and thinking about their markets of interest. He's a positive person, so he agreed to give it a shot. A short while later, just by adding Varnish and some good VCL, the load dropped from 100% to 5%. Without any blocking.

    He called me a little while ago, incredulous and happy.

    It's always a pleasure to work with people who are willing to experiment, explore, and listen. With people who don't just follow the herd or ad banners, but who think for themselves. I've earned myself a walk outside the office now.

    @stefano About paying more. We used to have that classic approach of "just add some resources for now".
    Until a magic point where boss asked "why the hell are we paying twice as much with same amount of users?"
    And then was painful and frustrating process of optimizing and downscaling working system.
    Most cliche IT story ever. :)

  • @stefano About paying more. We used to have that classic approach of "just add some resources for now".
    Until a magic point where boss asked "why the hell are we paying twice as much with same amount of users?"
    And then was painful and frustrating process of optimizing and downscaling working system.
    Most cliche IT story ever. :)

    This post is deleted!
  • This morning, a dev I work with flagged a server that was struggling. I logged in and, unfortunately, a bunch of "Scrapers" were hammering it, causing an overload. This dev is very sharp and understood the situation. He started extending the caching margins. I prepared a list of countries of interest and, using pf, blocked the rest of the world.
    The load plummeted, and the client doesn't care about their products being visible outside of specific markets.

    Shortly after, a similar situation on another server I manage (subcontracted, to be precise). I didn't know this dev and saw there was no caching in place, so I spoke to him and asked him to implement it. His response, sadly predictable, was that another one of his clients just moved to $CLOUD_PROVIDER and no longer has these problems, simply paying more when these "attacks" become more intense. I suggested he try enabling caching and thinking about their markets of interest. He's a positive person, so he agreed to give it a shot. A short while later, just by adding Varnish and some good VCL, the load dropped from 100% to 5%. Without any blocking.

    He called me a little while ago, incredulous and happy.

    It's always a pleasure to work with people who are willing to experiment, explore, and listen. With people who don't just follow the herd or ad banners, but who think for themselves. I've earned myself a walk outside the office now.

    This post is deleted!

Gli ultimi otto messaggi ricevuti dalla Federazione
Post suggeriti
  • 0 Votes
    14 Posts
    48 Views
    @nuintari It was 2006, I think (or 2007 at the latest), and I went with a colleague to a company - the same one that, incidentally, gave us some decommissioned Digital AlphaStations. They showed me a machine (not connected to the network, obviously) that they were still using for payroll. I couldn't quite tell what it was exactly, but it was some kind of Unix system and it had been rebooted for the last time... in 1986. I believe it’s the longest uptime I’ve ever seen.
  • 0 Votes
    1 Posts
    8 Views
    Hot take: pf's built-in connection tracking beats fail2ban/sshguard hands down.One simple ruleset gives you automatic brute-force protection with ZERO userland daemons. No log parsing, no reaction delays, no additional attack surface.table <bruteforce> persistpass in proto tcp to port 22 flags S/SA (max-src-conn 5, max-src-conn-rate 3/30, overload <bruteforce> flush global)Kernel-level enforcement, instant blocking, survives reboots with persist.Why spawn Python processes when your firewall already knows?#bsd #freebsd #runbsd #firewall #pf #sysadmin
  • 0 Votes
    28 Posts
    49 Views
    No matter how you slice it, as I see it, it boils down to: Microsoft's software did insufficient input validation on untrusted data (specifically in this case configuration changes for one customer, it seems), and this led to widespread outages affecting many, many, *many* customers in many different ways.Security done properly should not rely on "blocking erroneous" stuff, but should verify that everything is good and only *if* that succeeds allow any of it through.@spytfyre @rysiek
  • 0 Votes
    2 Posts
    12 Views
    @nicolafioretti @opensource@diggita.com ciao, grazie per il tag, un suggerimento, lascia sempre una riga vuota fra il tag al gruppo @diggita e gli hashtag, così il messaggio risulterà più leggibile, esempio:Perfetto per chi ama il self-hosting e vuole tenere d’occhio i propri servizi con stile e precisione.https://linuxiac.com/uptime-kuma-2-0-arrives-with-mariadb-support-modern-ui-refresh/@opensource@diggita.com #SysAdmin #DevOps #Monitoring #OpenSource #SelfHosted