Skip to content
0
  • Home
  • Piero Bosio
  • Blog
  • World
  • Fediverso
  • News
  • Categories
  • Old Web Site
  • Recent
  • Popular
  • Tags
  • Users
  • Home
  • Piero Bosio
  • Blog
  • World
  • Fediverso
  • News
  • Categories
  • Old Web Site
  • Recent
  • Popular
  • Tags
  • Users
Skins
  • Light
  • Brite
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse

Piero Bosio Social Web Site Personale Logo Fediverso

Social Forum federato con il resto del mondo. Non contano le istanze, contano le persone
nothacking@infosec.exchangeundefined

nothacking

@nothacking@infosec.exchange
About
Posts
1
Topics
0
Shares
0
Groups
0
Followers
0
Following
0

View Original

Posts

Recent Best Controversial

  • Web design in the early 2000s: Every 100ms of latency on page load costs visitors.
    nothacking@infosec.exchangeundefined nothacking@infosec.exchange

    @alexskunz @david_chisnall

    The thing is, you don't a CAPTCHA. Just three if statements on the server will do it:

    1. If the user agent is chrome, but it didn't send a "Sec-Ch-Ua" header: Send garbage.
    2. If the user agent is a known scraper ("GPTBot", etc): Send garbage.
    3. If the URL is one we generated: Send garbage.
    4. Otherwise, serve the page.

    The trick is that instead of blocking them, serve them randomly generated garbage pages.

    Each of these pages includes links that will always return garbage. Once these get into the bot's crawler queue, they will be identifiable regardless of how well they hide themselves.

    I use this on my site: after a few months, it's 100% effective. Every single scraper request is being blocked. At this point, I could ratelimit the generated URLs, but I enjoy sending them unhinged junk. (... and it's actually cheaper then serving static files!)

    This won't do anything about vuln scanners and other non-crawler bots, but those are easy enough to filter out anyway. (URL starts with /wp/?)

    Uncategorized
  • 1 / 1
  • Login

  • Login or register to search.
  • First post
    Last post