Skip to content

Piero Bosio Social Web Site Personale Logo Fediverso

Social Forum federato con il resto del mondo. Non contano le istanze, contano le persone

This morning, a dev I work with flagged a server that was struggling.

Uncategorized
4 3 14
  • This morning, a dev I work with flagged a server that was struggling. I logged in and, unfortunately, a bunch of "Scrapers" were hammering it, causing an overload. This dev is very sharp and understood the situation. He started extending the caching margins. I prepared a list of countries of interest and, using pf, blocked the rest of the world.
    The load plummeted, and the client doesn't care about their products being visible outside of specific markets.

    Shortly after, a similar situation on another server I manage (subcontracted, to be precise). I didn't know this dev and saw there was no caching in place, so I spoke to him and asked him to implement it. His response, sadly predictable, was that another one of his clients just moved to $CLOUD_PROVIDER and no longer has these problems, simply paying more when these "attacks" become more intense. I suggested he try enabling caching and thinking about their markets of interest. He's a positive person, so he agreed to give it a shot. A short while later, just by adding Varnish and some good VCL, the load dropped from 100% to 5%. Without any blocking.

    He called me a little while ago, incredulous and happy.

    It's always a pleasure to work with people who are willing to experiment, explore, and listen. With people who don't just follow the herd or ad banners, but who think for themselves. I've earned myself a walk outside the office now.

  • This morning, a dev I work with flagged a server that was struggling. I logged in and, unfortunately, a bunch of "Scrapers" were hammering it, causing an overload. This dev is very sharp and understood the situation. He started extending the caching margins. I prepared a list of countries of interest and, using pf, blocked the rest of the world.
    The load plummeted, and the client doesn't care about their products being visible outside of specific markets.

    Shortly after, a similar situation on another server I manage (subcontracted, to be precise). I didn't know this dev and saw there was no caching in place, so I spoke to him and asked him to implement it. His response, sadly predictable, was that another one of his clients just moved to $CLOUD_PROVIDER and no longer has these problems, simply paying more when these "attacks" become more intense. I suggested he try enabling caching and thinking about their markets of interest. He's a positive person, so he agreed to give it a shot. A short while later, just by adding Varnish and some good VCL, the load dropped from 100% to 5%. Without any blocking.

    He called me a little while ago, incredulous and happy.

    It's always a pleasure to work with people who are willing to experiment, explore, and listen. With people who don't just follow the herd or ad banners, but who think for themselves. I've earned myself a walk outside the office now.

    @stefano About paying more. We used to have that classic approach of "just add some resources for now".
    Until a magic point where boss asked "why the hell are we paying twice as much with same amount of users?"
    And then was painful and frustrating process of optimizing and downscaling working system.
    Most cliche IT story ever. :)

  • @stefano About paying more. We used to have that classic approach of "just add some resources for now".
    Until a magic point where boss asked "why the hell are we paying twice as much with same amount of users?"
    And then was painful and frustrating process of optimizing and downscaling working system.
    Most cliche IT story ever. :)

    This post is deleted!
  • This morning, a dev I work with flagged a server that was struggling. I logged in and, unfortunately, a bunch of "Scrapers" were hammering it, causing an overload. This dev is very sharp and understood the situation. He started extending the caching margins. I prepared a list of countries of interest and, using pf, blocked the rest of the world.
    The load plummeted, and the client doesn't care about their products being visible outside of specific markets.

    Shortly after, a similar situation on another server I manage (subcontracted, to be precise). I didn't know this dev and saw there was no caching in place, so I spoke to him and asked him to implement it. His response, sadly predictable, was that another one of his clients just moved to $CLOUD_PROVIDER and no longer has these problems, simply paying more when these "attacks" become more intense. I suggested he try enabling caching and thinking about their markets of interest. He's a positive person, so he agreed to give it a shot. A short while later, just by adding Varnish and some good VCL, the load dropped from 100% to 5%. Without any blocking.

    He called me a little while ago, incredulous and happy.

    It's always a pleasure to work with people who are willing to experiment, explore, and listen. With people who don't just follow the herd or ad banners, but who think for themselves. I've earned myself a walk outside the office now.

    This post is deleted!

Gli ultimi otto messaggi ricevuti dalla Federazione
Post suggeriti
  • 0 Votes
    5 Posts
    24 Views
    @rysiek this also means that no cirque de soleil trapeze act of mcp servers, llms checking other llms output, ralph wiggum model, openclaw bots or other 'stuff on top' fixes the core issue. im actively working on a con talk on this topic, so your good press today is getting a hat tip in my slides :D
  • 0 Votes
    1 Posts
    6 Views
    Today, I lived through one of those (not only IT) horror stories. It's not over yet, but itโ€™s definitely going to end up in a blog post. It's one of those situations where the 'horror' has already happened or is still going to happen, but this time, I'm the one handling the rescue. I couldn't prevent the disaster itself, but I managed to save the data. What-a-day!#IT #SysAdmin #HorrorStories #SaveTheData #OwnYourData
  • 0 Votes
    1 Posts
    7 Views
    Goodbye Linux & Podman, hello FreeBSD & Jails! Just migrated my blog (https://blog.hofstede.it) to a fully native BSD stack (where my Gemini Capsule was already living).Stack (using Bastille VNET Jails):- Caddy (Ingress, TLS, Reverse-Proxy)- Nginx Jail (Internal. Static file serving)- PFThe Cool Part: A Zero-Trust CI/CD pipeline. My Forgejo runner deploys via restricted rrsync into an air-gapped "transporter" jail, which nullfs mounts the web root. Security: Source-IP restricted, no interactive shells, no PTY.The simplicity of files-on-disk beats container abstraction every time.#BastilleBSD #SelfHosted #SysAdmin #IPv6 #ZFS #FreeBSD #RunBSD
  • 0 Votes
    1 Posts
    13 Views
    Hi folks! My employer is currently looking to hire a fractional CTO at about 6 hours per month, and this person would ideally provide high-level direction and guidance regarding technical systems administration, cybersecurity, and digital infrastructure for our team.Ideal candidates would have substantial career experience and live in Canada. Values-alignment (eco-stuff) is also very important.Please DM for more info!#GetFediHired #sysadmin #cybersecurity #ClimateTech #climatechange