Skip to content

Piero Bosio Social Web Site Personale Logo Fediverso

Social Forum federato con il resto del mondo. Non contano le istanze, contano le persone

Feature Requests

You have a cool idea about NodeBB? Post it here.


This is a forum category containing topical discussion. You can start new discussions by mentioning this category.

1 Topics 3 Posts
  • Any protection against AI Crawlers and AI learning bots?

    3
    3 Posts
    2 Views
    As a solution, the link you suggested me kinda helped, but not really... My alternative solution for this was to implement Anubis anti-crawler, since I don't trust Cloudflare that much... My Forum is not on Fediverse yet and I didn't share with anyone and still I was receiving high numbers of clicks and inconsistent unique visitors... I told robots.txt to block every bot since well I'm still building the Forum and still I received one bot in there... Hopefully with Anubis the problem can be at least minimized, but for anyone having a Forum... I suggest search for how to implement an anti-bot yourself since otherwise your bandwidth is going to cry for help. I implemented Anubis just today successfully, but I'll be monitoring the traffic of the site either way...

Gli ultimi otto messaggi ricevuti dalla Federazione
  • As a solution, the link you suggested me kinda helped, but not really... My alternative solution for this was to implement Anubis anti-crawler, since I don't trust Cloudflare that much... My Forum is not on Fediverse yet and I didn't share with anyone and still I was receiving high numbers of clicks and inconsistent unique visitors... I told robots.txt to block every bot since well I'm still building the Forum and still I received one bot in there...

    Hopefully with Anubis the problem can be at least minimized, but for anyone having a Forum... I suggest search for how to implement an anti-bot yourself since otherwise your bandwidth is going to cry for help. I implemented Anubis just today successfully, but I'll be monitoring the traffic of the site either way...

    read more

  • Shaknunic here's the catch-22... If you want to stop AI crawlers, you can stop the good ones using robots.txt

    https://github.com/ai-robots-txt/ai.robots.txt

    If you want to stop the bad ones (which you probably do)... you can't

    They masquerade as legitimate users, and they have more funding behind them than you have time. Every time you try to block them someone on the other end will adapt.

    Unfortunately right now the only reliable solution is a hard challenge from CloudFlare for nearly all requests.

    read more

  • The forum has the robot.txt which is a basic protection against search bots, but AI bots are still a threat, so maybe someone can make a plugin or devs can integrated that into the next versions of the Forum? NodeBB joining Fediverse is excellent and I'm using NodeBB Forums for the exactly reason, I understand that project is like in pre-alpha but suggesting new things to be implemented is also a good thing.

    read more
Post suggeriti
  • 0 Votes
    3 Posts
    2 Views
    As a solution, the link you suggested me kinda helped, but not really... My alternative solution for this was to implement Anubis anti-crawler, since I don't trust Cloudflare that much... My Forum is not on Fediverse yet and I didn't share with anyone and still I was receiving high numbers of clicks and inconsistent unique visitors... I told robots.txt to block every bot since well I'm still building the Forum and still I received one bot in there... Hopefully with Anubis the problem can be at least minimized, but for anyone having a Forum... I suggest search for how to implement an anti-bot yourself since otherwise your bandwidth is going to cry for help. I implemented Anubis just today successfully, but I'll be monitoring the traffic of the site either way...