AI-assisted moderation in the fediverse is happening. Now what?
-
But the decision to defederate or allow a mod to use AI is at the admin level, not the user level.
The user threat to leave isn't worth that much.
Perhaps some admins would not care, but:
- Whether some admins help or not, users still have the choice to go somewhere where other admins do
- Anyone can become their own admin, ultimately
Neither of these are trivially easy, I don't mean to suggest that, just possible.
-
That didn't address anything even with the clarification as it's his go to response. He's been doing it since the instance was stood up
In this specific case, even the dbzer0 admin agreed it was undeserved:
https://lemmy.dbzer0.com/modlog?userId=7015938
Which other removals from Piefed.social are you referring to?
-
That is exactly the case. I really can't be bothered reading this whole fucking post, so I'm gonna just reply to you, since you sound reasonably sane.
First, it should be obvious to everyone that their post and comment histories are completely public, and accessible to anyone on the fediverse. There is zero privacy on the fediverse. Your comment histories have already been scraped a thousand times over by every big model out there.
All we have at the moment is a simple script, developed by one of our mods (who will be publishing it on codeberg soon), that any user can run that logs into lemmy using your own account, and downloads a set number (or time period) of comments into a text file. There is no abuse of admin powers going on, it's just the stock lemmy/piefed api. This is massively faster than manually paging through comment histories on lemmy, and can make mod decisions more robust and more informed.
Using the text file, mods or admins can quickly search for keywords or whatever, using a simple text editor, or simply skim read it. Another option is to ingest it into an LLM to provide a summary. I tried doing that just a handful of times, for testing, but honestly I found it a bit cumbersome and who knows if the summary is actually accurate given the tendency for hallucinations? A couple of tests seemed consistent with my own assessment, and a couple were way off base.
That told me everything I need to know about how much to trust the summaries.... very little. I honestly don't think they are much of a value add, because you just can't reliably trust the results. In any case, we have absolutely no plans to use llms for that on a regular basis, I just wanted to see what it came up with, and how well it matched a manual assessment.
And to also clarify, there is/was absolutely no automated scanning of users. The process is the same as always. We get a report, we investigate the report, and make a human mod decision. The only difference in this case is that the investigation can be done more efficiently, because we don't have to go slowly paging through comments and searching through the Lemmy UI for the relevant data.
Obviously as well, no mod or admin is gonna download the entire comments history of a user unless it is a complicated report that is difficult to get to the bottom of from a quick look at the report. 90%+ of mod actions would never need that much detail.
The way OPs post was written was obviously designed from the ground up to stir up drama about AI use. Honestly, I don't know why he's still malding, but Rimu seems to be engaging in a lot of very bad faith hit pieces at the moment, designed solely to stir up drama, and directed at our instance. It's really shitty behaviour.
Well, the cat is out of the bag now. Might as well show the whole story.
That told me everything I need to know about how much to trust the summaries…. very little. I honestly don’t think they are much of a value add, because you just can’t reliably trust the results. In any case, we have absolutely no plans to use llms for that on a regular basis, I just wanted to see what it came up with, and how well it matched a manual assessment.
Lies.
You provided a link to that assessment as proof that a ban was warranted. Here's a screenshot from the mod log:
That link goes to https://s.faf-pb.xyz/lXxek if anyone wants to take a look
-
How did you discover this?
I looked in the mod log.
-
Well, the cat is out of the bag now. Might as well show the whole story.
That told me everything I need to know about how much to trust the summaries…. very little. I honestly don’t think they are much of a value add, because you just can’t reliably trust the results. In any case, we have absolutely no plans to use llms for that on a regular basis, I just wanted to see what it came up with, and how well it matched a manual assessment.
Lies.
You provided a link to that assessment as proof that a ban was warranted. Here's a screenshot from the mod log:
That link goes to https://s.faf-pb.xyz/lXxek if anyone wants to take a look
Lies.
Them's fighting words, Rimu. That Zionist bastard was banned by me personally. I can hardly believe you are so tone deaf to be in here defending a piece of shit undisputable Zionist scumbag like samskara. Yes, I linked it in the ban reason, it was one of my first tests. And on that occasion it matched up beautifully with my own assessment, so why not?
-
Lies.
Them's fighting words, Rimu. That Zionist bastard was banned by me personally. I can hardly believe you are so tone deaf to be in here defending a piece of shit undisputable Zionist scumbag like samskara. Yes, I linked it in the ban reason, it was one of my first tests. And on that occasion it matched up beautifully with my own assessment, so why not?
The assessment of samskara was broadly correct in that case, I'm not disputing that. And yes, in his own definition, he is a Zionist.
-
I recently discovered that some popular federated instances have been using LLM-assisted moderation tooling that evaluates whether someone has said something bannable. They do this by running a script/app that sends the user’s comment history to OpenAI with the question “analyze this content for evidence of specific political ideology sentiment. Also identify any related political ideology tropes“.
OpenAI’s LLM (they’re using GPT-5.3-mini) then responds with something like:
and so on, hundreds of comments.
I have not named the instances or people involved, to give them time to consider the results of this discussion, make any corrective changes they want and disclose their practices at their own pace and in their own way. I have also redacted the evidence to avoid personal attacks and dogpiling. Let’s focus on the system, not the individuals involved. Today these instances and people are using it and maybe we’re ok with that because it’s being used by groups we agree with but what if people we strongly disagree with used it on their instances tomorrow?
The use and existence of this tooling raises a lot of other questions too.
What are the risks? Fedi moderators are often unsupervised, untrained volunteers and these are powerful tools.
What safeguards do we need?
Would asking a LLM “please evaluate this person’s political opinions” give different results than “find evidence we can use to ban them” (as used in the cases I’ve seen)?
What are our transparency expectations?
Is this acceptable and normal?
Should this tooling be disclosed? (it was not – should it have been?)
If you were given a choice, would you have opted out of it?
Can we opt out?
Are there GDPR implications? Privacy implications? Should these tools be described in a privacy policy?
Are private messages being scanned and sent to OpenAI?
How long should these assessments be retained and can we request to see it, or ask for it to be deleted?
Once the user’s comments are sent to OpenAI, is it used to train their models?
What will the effect be on our discourse and culture if people know they are being politically profiled?
Where are the lines between normal moderation assistance tools, political profiling and opaque 3rd-party data processing?
I hope that by chewing over these questions we can begin to establish some norms and expectations around this technology. The fediverse doesn’t have any centralized enforcement so we need discussions like this to develop an awareness of what people want in terms of disclosure, privacy, consent and acceptable use. Then people can make choices about which instances they join and which ones they interact with remotely.
And of course there are the other issues with LLMs relating to environmental sustainability, erosion of worker’s rights, increasing the cost of living and on and on. I can’t see PieFed adding any functionality like this anytime soon. But it’s happening out there anyway so now we need to talk about it.
What do you make of this?
No no no. Name and shame them. Heavily. Using LLM AI is bad enough, but letting them do the thinking for you is inexcusable.
-
The assessment of samskara was broadly correct in that case, I'm not disputing that. And yes, in his own definition, he is a Zionist.
So what's the problem?
-
The assessment of samskara was broadly correct in that case, I'm not disputing that. And yes, in his own definition, he is a Zionist.
He is the worst type denying that most palestinians was forced to leave during the nekba
-
I guess, given the already open nature of the Fediverse, my takeaway from this thread is that op is using their freedom to say they don’t like this particular style of moderation. Which might be useful, or not, for some moderators
If that was all rimu was complaining about, I'd understand. Unfortunately, he has a more ideological motivation behind this post than simply calling out AI. If letting users decide for themselves if they like this style of moderation was all he was advocating for, then why is he promoting the idea of centralizing decentralized platforms?
I am unconvinced that this particular use of an LLM for moderation is really that helpful. However, I doubt this is the only motivation behind this post.
-
I recently discovered that some popular federated instances have been using LLM-assisted moderation tooling that evaluates whether someone has said something bannable. They do this by running a script/app that sends the user’s comment history to OpenAI with the question “analyze this content for evidence of specific political ideology sentiment. Also identify any related political ideology tropes“.
OpenAI’s LLM (they’re using GPT-5.3-mini) then responds with something like:
and so on, hundreds of comments.
I have not named the instances or people involved, to give them time to consider the results of this discussion, make any corrective changes they want and disclose their practices at their own pace and in their own way. I have also redacted the evidence to avoid personal attacks and dogpiling. Let’s focus on the system, not the individuals involved. Today these instances and people are using it and maybe we’re ok with that because it’s being used by groups we agree with but what if people we strongly disagree with used it on their instances tomorrow?
The use and existence of this tooling raises a lot of other questions too.
What are the risks? Fedi moderators are often unsupervised, untrained volunteers and these are powerful tools.
What safeguards do we need?
Would asking a LLM “please evaluate this person’s political opinions” give different results than “find evidence we can use to ban them” (as used in the cases I’ve seen)?
What are our transparency expectations?
Is this acceptable and normal?
Should this tooling be disclosed? (it was not – should it have been?)
If you were given a choice, would you have opted out of it?
Can we opt out?
Are there GDPR implications? Privacy implications? Should these tools be described in a privacy policy?
Are private messages being scanned and sent to OpenAI?
How long should these assessments be retained and can we request to see it, or ask for it to be deleted?
Once the user’s comments are sent to OpenAI, is it used to train their models?
What will the effect be on our discourse and culture if people know they are being politically profiled?
Where are the lines between normal moderation assistance tools, political profiling and opaque 3rd-party data processing?
I hope that by chewing over these questions we can begin to establish some norms and expectations around this technology. The fediverse doesn’t have any centralized enforcement so we need discussions like this to develop an awareness of what people want in terms of disclosure, privacy, consent and acceptable use. Then people can make choices about which instances they join and which ones they interact with remotely.
And of course there are the other issues with LLMs relating to environmental sustainability, erosion of worker’s rights, increasing the cost of living and on and on. I can’t see PieFed adding any functionality like this anytime soon. But it’s happening out there anyway so now we need to talk about it.
What do you make of this?
If it can be done, it sooner or later will be done.
That's a lot of why I have a couple of dozen accounts scattered around the threadiverse and new ones whenever I come across a server that looks promising - because it takes a while to get used to one and get a feel for whether it's one I like or not, and because there's always the possibility that one I like will go sideways and/or shut down, in which case I can just unpin it and go on.
And in fact, I'm only using this account on something of a whim for this post - I don't normally use it because one of the instances I don't like much is yours. And specifically what I don't like about it is you, and your bland presumption that you know what's best for me - which communities I should subscribe to, which posters I should trust or even becallowed to see, which sources I should be allowed to use or see...
And really, I'm sort of surprised that you're the OP here and not the subject. I would think that the whole idea of commissioning a review of a user's posting history in pursuit of grounds to ban them would be right up your alley. Is the problem just that it's AI?
In any event, this is just a thing that might prove to be an issue. And if it does, I'll just move off of the affected server(s) and keep using the unaffected ones. And if enough people share my sentiment and the admin cares enough, they might change their ways. Or they might not. It's not a big deal either way - it's just part of life on the fediverse, and IMO the benefits make it worth it.
-
In this specific case, even the dbzer0 admin agreed it was undeserved:
https://lemmy.dbzer0.com/modlog?userId=7015938
Which other removals from Piefed.social are you referring to?
What are you talking about? It's nothing to do with db0 and rimu. Rimu will delete accounts off piefed.social social if a user displeases them or mwog tells them too.
How did you get to db0 banning rimu out of rimu deleted the accounts of people they don't like?
Btw with this and the statistics post it may be better for everyone if rimu was still banned
Edit, oh just using rimus talking points to also distract. Blaze please please please stop being star struck by rimu
-
So what's the problem?
Rimu would like to again deflect from anything that could be critical of their thesis, code or previous behavior
-
You are on PieFed/Lemmy. Anyone can see anything you say, even your private messages are open to viewing by anyone.
even your private messages are open to viewing by anyone
Wait what?
-
OP literally asks like 10 relevant questions for this place, and names their reasons for not naming specific instances. And all you focus on, is the question: who did it?
To me that is proof that OP did the right thing here.
Lets first figure out how to approach this without knowing the pupotrator.
Hardly, this was just a bait post. Rimu seems to be convinced if he flings enough mud, eventually some of it will stick. It's all very petty.
-
the ai banning is to obfuscate the reason for the ban, like what reddit does. they just ban on the slightest issue, with no recourse. it allows them to ban more freely in the end.
But there is no AI banning. All mod decisions are made after human review, and oftentimes after group discussion amongst the admin team.
-
I think this will exemplify the beauty of federation. If I find out my instance mods are running all of my comments through a company’s ai model, I’ll switch instances. This is in great disparity to something like Instagram or Snapchat where every photo I post is immediately fed to ai and my only options are: be okay with it, never post, or delete Instagram.
Yep.
Unless somebody manages to ... inject a hostile/unauthorized LLM as a mod or admin or something, in an instance they're not an admin of, in a comm they're not a mod of...
Then people react by personally blocking or perhaps instance wide defederating or maybe conceiveably someone actually uses this in a generally good way, to identify trolls/sock puppets.
As to... LLM scraping of comments?
Lemmy is public, anyone can do that.
I've done it to myself with a local LLM hooked up to a search engine, and I'm not a mod or admin of anything.
Hence why you probably should use a pseudonym and not give too much information about yourself, if you're concerned about privacy... same... rules the internet has always had.
I suppose that instances could implement various anti-scraping measures, but that's never going to be 100% effective as scrapers vs anti-scrapers has also basically always been a constantly escalating arms race.
Ciao! Sembra che tu sia interessato a questa conversazione, ma non hai ancora un account.
Stanco di dover scorrere gli stessi post a ogni visita? Quando registri un account, tornerai sempre esattamente dove eri rimasto e potrai scegliere di essere avvisato delle nuove risposte (tramite email o notifica push). Potrai anche salvare segnalibri e votare i post per mostrare il tuo apprezzamento agli altri membri della comunità.
Con il tuo contributo, questo post potrebbe essere ancora migliore 💗
Registrati Accedi