AI-assisted moderation in the fediverse is happening. Now what?
-
Sure, but we all know how power mods be.
Obviously, but unless the modlog is being spammed with many entries, and I receive notifications for each one individually (which is actually happening right now from communities on dbzer0, and unlike Lemmy, PieFed actually sends notifications for such events, but I will set that sub-topic aside for s moment), then in theory I do not care if I am (preemptively?) banned from let's say c/conservative@newreddit.com or c/extremetankiedeathsquad@thereallemmy.fuckallwesterners (to be clear, these are hypothetical made-up names!🤪), if I never wanted to post or vote on their content anyway? They are even doing me a favor if it prevents it from showing in my feed (which it would on Lemmy iirc, but on PieFed it would not).
-
I recently discovered that some popular federated instances have been using LLM-assisted moderation tooling that evaluates whether someone has said something bannable. They do this by running a script/app that sends the user’s comment history to OpenAI with the question “analyze this content for evidence of specific political ideology sentiment. Also identify any related political ideology tropes“.
OpenAI’s LLM (they’re using GPT-5.3-mini) then responds with something like:
and so on, hundreds of comments.
I have not named the instances or people involved, to give them time to consider the results of this discussion, make any corrective changes they want and disclose their practices at their own pace and in their own way. I have also redacted the evidence to avoid personal attacks and dogpiling. Let’s focus on the system, not the individuals involved. Today these instances and people are using it and maybe we’re ok with that because it’s being used by groups we agree with but what if people we strongly disagree with used it on their instances tomorrow?
The use and existence of this tooling raises a lot of other questions too.
What are the risks? Fedi moderators are often unsupervised, untrained volunteers and these are powerful tools.
What safeguards do we need?
Would asking a LLM “please evaluate this person’s political opinions” give different results than “find evidence we can use to ban them” (as used in the cases I’ve seen)?
What are our transparency expectations?
Is this acceptable and normal?
Should this tooling be disclosed? (it was not – should it have been?)
If you were given a choice, would you have opted out of it?
Can we opt out?
Are there GDPR implications? Privacy implications? Should these tools be described in a privacy policy?
Are private messages being scanned and sent to OpenAI?
How long should these assessments be retained and can we request to see it, or ask for it to be deleted?
Once the user’s comments are sent to OpenAI, is it used to train their models?
What will the effect be on our discourse and culture if people know they are being politically profiled?
Where are the lines between normal moderation assistance tools, political profiling and opaque 3rd-party data processing?
I hope that by chewing over these questions we can begin to establish some norms and expectations around this technology. The fediverse doesn’t have any centralized enforcement so we need discussions like this to develop an awareness of what people want in terms of disclosure, privacy, consent and acceptable use. Then people can make choices about which instances they join and which ones they interact with remotely.
And of course there are the other issues with LLMs relating to environmental sustainability, erosion of worker’s rights, increasing the cost of living and on and on. I can’t see PieFed adding any functionality like this anytime soon. But it’s happening out there anyway so now we need to talk about it.
What do you make of this?
They do this by running a script/app that sends the user’s comment history to OpenAI with the question “analyze this content for evidence of specific political ideology sentiment.
To me the problem is what they are looking for not how they are doing it. Thought experiment: in what way would it be qualitatively different if they hired a team of people in Upper Elbonia to do the same thing?
-
So its hard for me to get into these things without harping on my personal philosophy. Which is that I think ideally this should mirror the way we would interact in person. So moderating or running a community is like running or being part of the core group that runs a club. Would you want to throw that to a robot? Basically I don't feel people should create or run or moderate communities unless they enjoy it. So the idea of ai moderation is to me pointless. Of course at this point I notice you are talking instances. Boy that is different. This is more like talking about running the institution that allows spaces for clubs to meet. It kinda feels understandable then. Honestly people complain about being banned but I kinda feel like anyplace that bans me is kinda doing me a favor. Like I would like the option to just mark it permanent. Its less things I got to block. Its the same reason I would like blocking to be symetric. Saves me some work (ok and the creepy I turn them invisible so I don't see them but they can watch me). I really would like to be able to block an instance seperately for communities or users. Ok as usually im digressing quite a bit but I guess in the end run I kinda see why at the instance level it might be used but I would be concerned it would start being used at the community level. It would be nice to know its happening at either level and have the ability to block them if a user is not wild about the concept.
For some of us, power carries with it a certain level of responsibility. 🕸️
For others, it is just a fun tool to wield over the fates of others. 🐒 And AI lets them do that with less effort. 💤
-
The use of AI for moderation isn't the choice of users, but moderators and admins.
I disagree: ultimately it can be, if users choose instances that defederate from those who allow their kids to use AI tools. (Autocorrect changed "mods" to "kids", but I think I will leave it that way bc it's funnier 😜)
-
whats funny is I don't have much negative dbzer0 experience until you two guys start making this about that.
That's an odd takeaway, for sure.
-
In Fediverse moderation tools, are there consensus forming mechanisms to ensure that even if 20% of the volunteer moderators are malicious, none of their wrongful moderation suggestions leak through to the stream of final moderation actions? If not, I'd be reluctant to add moderators.
You're risking bad actors either way, its just a question if you have enough good actors to counterbalance the bad.
Unsure what level of power a mod has to mess up a server or access for everyone, but seems like the worst case scenario is you just strip their mod tools and ban... So why not screen a bit and add some people instead of handing data to AI, which is what some of us came here to escape.
-
im just saying when there is a particular thing and someone starts pulling in something unrelated as a conspiracy it leaves a bad taste. Its a bit like folks suddenly saying something about lemmy.ml being so and so in an unrelated type post that gets me to do my posting in their communities but in a reverse kinda way. rimus last post about a trend he saw I think it had some interesting perspectives and few if any where that the instances were ban happy. Similarly this one has some good conversations going.
unrelated
Well, it is noting that there's now at least an appearance of a relationship going on. Especially so, if you're already clued into the 'ban-happy' statistics magic thread.
Similarly this one has some good conversations going.
Hopefully those continue unimpeded and this can remain contextual or tangental.
-
I don't doubt it even for a second.
I'm lowkey kind of fascinated this morning with what feels like a moment of real panic among western liberal-democratic institutions (projecting a little from my morning news and coffee). That an anarchist instance is getting this much targeted harassment feels like a microscopic extension of that (if I allow myself to be so bold)
As far as I can tell, dbzer0 isnt even being explicitly called out here, but it has an undeniable bdzer0 flavor to it. If it doesnt come out that this was one of our mods at this point, I'd almost be disappointed.
I’m lowkey kind of fascinated this morning with what feels like a moment of real panic among western liberal-democratic institutions (projecting a little from my morning news and coffee). That an anarchist instance is getting this much targeted harassment feels like a microscopic extension of that (if I allow myself to be so bold)
As far as I can tell, dbzer0 isnt even being explicitly called out here, but it has an undeniable bdzer0 flavor to it. If it doesnt come out that this was one of our mods at this point, I’d almost be disappointed
These type of posts are why people (from all walks of life) view western leftism as more of an aesthetic, performative thing.
-
unrelated
Well, it is noting that there's now at least an appearance of a relationship going on. Especially so, if you're already clued into the 'ban-happy' statistics magic thread.
Similarly this one has some good conversations going.
Hopefully those continue unimpeded and this can remain contextual or tangental.
this one did not name any instances. paranoia is the only reason to link them or knowldge that the two things are indeed connected. Each is fine to me though. Bringing up data and slicing and dicing it a bit is fine and the convo brought up issues with the way it was massaged. Similarly this is about a particular thing. Niether indicated an opinion or want for some sort of action against instances. so yeah. unrelated. but yeah I like good and vibrant convo which is why I have enjoyed both posts. one thing im not wild about this is some comments suggest its an admin thing and should be discussed in some admin community but im not wild about the cigar filled room type thing.
-
I understand that some form of automation is necessary - we saw large instances closing because they couldn't find mods. My main objection in this scenario would be is that I didn't consent to train OpenAI models. I think the users should know if their instance uses external services like that.
I also suspect that there might be cheaper and more ethical solutions. Although it's hard to talk about this without seeing the actual results.
That was not why lemm.ee died. That one was lack of admins, presumably due to their choice to become a "free speech haven" where all voices were allowed to be heard. But then having given in that far to incel culture, nonetheless still trying to pull back to stop short of becoming another 4chan. The burden on those admins was INTENSE, and they simply gave up.
Btw, PieFed already offers a number of tools that helps reduce the burden on human mods to make decisions easier.
::: spoiler some examples
One example is an indicator that reports the likelihood that a user's content is generated by AI.Many other tools reduce even the need for moderation in the first place, e.g. iirc if someone receives 10x more downvotes than upvotes, a visual icon is added next to their username to indicate that they are known as a contentious user. Note this is not a filter, just a label. As as end-user reads through comments, upon seeing this they are warned that having a deeper discussion with such a person is unlikely to be considered pleasant, by the majority of others who have done so in the past.
Another feature example is keyword filtering - if someone wants to remove all content containing the words "Trump" or "Musk", then rather than downvote or report it to mods for its removal, they can have it removed at the level of themselves, thereby substantially reducing the burden of mods for such things as e.g. having to keep politics out of unrelated (comics?) communities.
Still another pair of examples is the ability to automatically collapse or even hide comments that fall below a given vote threshold - personally I have these options off, but if someone were to want that for themselves, then the tool is available to them, again independently of any need for moderator intervention. And the user likewise controls what that threshold is, rather than the mod having to make a single call for everyone in the entire community.
:::PieFed's democratization of moderation features are breathtakingly awesome to behold - Reddit and even Lemmy (+ Mbin, nodeBB, etc.) have nothing that even comes close! And note that article was even written two years ago (but I will stop myself short of making my comment a billet-doux on PieFed, instead just saying that it has long had this automation already).
-
Defederate, no question.
Are you gonna tell us which instance is doing this?
OP already answered this: not at the present time.
-
I recently discovered that some popular federated instances have been using LLM-assisted moderation tooling that evaluates whether someone has said something bannable. They do this by running a script/app that sends the user’s comment history to OpenAI with the question “analyze this content for evidence of specific political ideology sentiment. Also identify any related political ideology tropes“.
OpenAI’s LLM (they’re using GPT-5.3-mini) then responds with something like:
and so on, hundreds of comments.
I have not named the instances or people involved, to give them time to consider the results of this discussion, make any corrective changes they want and disclose their practices at their own pace and in their own way. I have also redacted the evidence to avoid personal attacks and dogpiling. Let’s focus on the system, not the individuals involved. Today these instances and people are using it and maybe we’re ok with that because it’s being used by groups we agree with but what if people we strongly disagree with used it on their instances tomorrow?
The use and existence of this tooling raises a lot of other questions too.
What are the risks? Fedi moderators are often unsupervised, untrained volunteers and these are powerful tools.
What safeguards do we need?
Would asking a LLM “please evaluate this person’s political opinions” give different results than “find evidence we can use to ban them” (as used in the cases I’ve seen)?
What are our transparency expectations?
Is this acceptable and normal?
Should this tooling be disclosed? (it was not – should it have been?)
If you were given a choice, would you have opted out of it?
Can we opt out?
Are there GDPR implications? Privacy implications? Should these tools be described in a privacy policy?
Are private messages being scanned and sent to OpenAI?
How long should these assessments be retained and can we request to see it, or ask for it to be deleted?
Once the user’s comments are sent to OpenAI, is it used to train their models?
What will the effect be on our discourse and culture if people know they are being politically profiled?
Where are the lines between normal moderation assistance tools, political profiling and opaque 3rd-party data processing?
I hope that by chewing over these questions we can begin to establish some norms and expectations around this technology. The fediverse doesn’t have any centralized enforcement so we need discussions like this to develop an awareness of what people want in terms of disclosure, privacy, consent and acceptable use. Then people can make choices about which instances they join and which ones they interact with remotely.
And of course there are the other issues with LLMs relating to environmental sustainability, erosion of worker’s rights, increasing the cost of living and on and on. I can’t see PieFed adding any functionality like this anytime soon. But it’s happening out there anyway so now we need to talk about it.
What do you make of this?
sounds like a great opportunity to inject attack through a comment.
-
Yes, but then you are willingly accepting the risk of posting in a fully public forum anyway. What I'm saying is, you could, if you wanted, not have personally identifiable information on your account.
What I'm saying is that's actually very hard unless you run a super sterile account on purpose. Even just your writing style is a pretty good fingerprint. Your IP. Any pictures you've posted. It's a rough world out there for privacy
-
I’m lowkey kind of fascinated this morning with what feels like a moment of real panic among western liberal-democratic institutions (projecting a little from my morning news and coffee). That an anarchist instance is getting this much targeted harassment feels like a microscopic extension of that (if I allow myself to be so bold)
As far as I can tell, dbzer0 isnt even being explicitly called out here, but it has an undeniable bdzer0 flavor to it. If it doesnt come out that this was one of our mods at this point, I’d almost be disappointed
These type of posts are why people (from all walks of life) view western leftism as more of an aesthetic, performative thing.
Right - which is why it was quite interesting watching Fox news have a 20 minute power struggle over the sudden popularity of May Day in the US and the rise of ""extreme socialist sentiment""
All online political discourse is performance - feel free to speculate how well it is representative of IRL leftist spaces in the west.
-
I recently discovered that some popular federated instances have been using LLM-assisted moderation tooling that evaluates whether someone has said something bannable. They do this by running a script/app that sends the user’s comment history to OpenAI with the question “analyze this content for evidence of specific political ideology sentiment. Also identify any related political ideology tropes“.
OpenAI’s LLM (they’re using GPT-5.3-mini) then responds with something like:
and so on, hundreds of comments.
I have not named the instances or people involved, to give them time to consider the results of this discussion, make any corrective changes they want and disclose their practices at their own pace and in their own way. I have also redacted the evidence to avoid personal attacks and dogpiling. Let’s focus on the system, not the individuals involved. Today these instances and people are using it and maybe we’re ok with that because it’s being used by groups we agree with but what if people we strongly disagree with used it on their instances tomorrow?
The use and existence of this tooling raises a lot of other questions too.
What are the risks? Fedi moderators are often unsupervised, untrained volunteers and these are powerful tools.
What safeguards do we need?
Would asking a LLM “please evaluate this person’s political opinions” give different results than “find evidence we can use to ban them” (as used in the cases I’ve seen)?
What are our transparency expectations?
Is this acceptable and normal?
Should this tooling be disclosed? (it was not – should it have been?)
If you were given a choice, would you have opted out of it?
Can we opt out?
Are there GDPR implications? Privacy implications? Should these tools be described in a privacy policy?
Are private messages being scanned and sent to OpenAI?
How long should these assessments be retained and can we request to see it, or ask for it to be deleted?
Once the user’s comments are sent to OpenAI, is it used to train their models?
What will the effect be on our discourse and culture if people know they are being politically profiled?
Where are the lines between normal moderation assistance tools, political profiling and opaque 3rd-party data processing?
I hope that by chewing over these questions we can begin to establish some norms and expectations around this technology. The fediverse doesn’t have any centralized enforcement so we need discussions like this to develop an awareness of what people want in terms of disclosure, privacy, consent and acceptable use. Then people can make choices about which instances they join and which ones they interact with remotely.
And of course there are the other issues with LLMs relating to environmental sustainability, erosion of worker’s rights, increasing the cost of living and on and on. I can’t see PieFed adding any functionality like this anytime soon. But it’s happening out there anyway so now we need to talk about it.
What do you make of this?
this is flat out not ok, does not matter who is doing it. our instance ls should defederate all which do this.
I would opt out that's no question, but I don't believe it's possible. GDPR does not matter here, as nothing can be proven unless the perpetrators give up themselves
-
I disagree: ultimately it can be, if users choose instances that defederate from those who allow their kids to use AI tools. (Autocorrect changed "mods" to "kids", but I think I will leave it that way bc it's funnier 😜)
But the decision to defederate or allow a mod to use AI is at the admin level, not the user level.
The user threat to leave isn't worth that much.
-
OP literally asks like 10 relevant questions for this place, and names their reasons for not naming specific instances. And all you focus on, is the question: who did it?
To me that is proof that OP did the right thing here.
Lets first figure out how to approach this without knowing the pupotrator.
The posts/comments on the fediverse are already public. The privacy questions are better answered here by another commenter:
My issue is that OP is not providing any solid proof. They are just giving 'wink wink's about some 'popular' instances doing it. When asked whether they have proof, OP says they has proof of some mods doing it. Mods don't handle instances, admins do. They haven't yet provided any concrete proof, yet creating an impression that some instances are banning en masse using LLMs based on "political ideology".
-
Yeah you can do that but now you're on my do-not-trust list. And probably a few other people's lists.
I appreciate you being open about your opinions because now I can make an more informed choice about interacting with you and the instance you run.
Don't you think everyone deserves the information they need to choose which instances they want to interact with, according to whatever criteria is important to them? Even if your criteria are different?
It's absolutely wild that the obvious needed to be pointed out at all, and that the reaction to it was 'you just made my list, buddy'.
-
Right - which is why it was quite interesting watching Fox news have a 20 minute power struggle over the sudden popularity of May Day in the US and the rise of ""extreme socialist sentiment""
All online political discourse is performance - feel free to speculate how well it is representative of IRL leftist spaces in the west.
From my time in the US, what you describe seems like a pretty run of the mill local rhetorical strategy; a tried and true American polemic if you will.
Sure, but even with something like online political discourse you can do better than getting obsessed over a bastardized provincial political term (liberal which means something completely different outside of the US) and providing cover for promoters of russian and Chinese genocidal imperialism and propaganda. We are not talking about a deep discussion on philosophy and political economy, just the absolute basics.
Can't speak for your definition of leftist spaces, but many (not all of course) self-proclaimed western leftists IRL have a very performative approach to imperialism (among other things).
-
From my time in the US, what you describe seems like a pretty run of the mill local rhetorical strategy; a tried and true American polemic if you will.
Sure, but even with something like online political discourse you can do better than getting obsessed over a bastardized provincial political term (liberal which means something completely different outside of the US) and providing cover for promoters of russian and Chinese genocidal imperialism and propaganda. We are not talking about a deep discussion on philosophy and political economy, just the absolute basics.
Can't speak for your definition of leftist spaces, but many (not all of course) self-proclaimed western leftists IRL have a very performative approach to imperialism (among other things).
Sure, but even with something like online political discourse you can do better than getting obsessed over a bastardized provincial political term (liberal which means something completely different outside of the US)
Sorry, I'm having a hard time following the train of thought - are you referring to my use of 'liberal-democratic institutions'?
I don't want to word-vomit on you unnecessarily if you're pointing to something else or speaking broadly about leftist discourse on lemmy (i've seen plenty of debates like the one you're describing)
and providing cover for promoters of russian and Chinese genocidal imperialism and propaganda
Ok well now i'm even more confused - where is this jab coming from?
Maybe you're taking issue with my categorizing db0 as a leftist space and are speaking broadly about the perspectives about china and russia from that instance? What are we talking about here?
Ciao! Sembra che tu sia interessato a questa conversazione, ma non hai ancora un account.
Stanco di dover scorrere gli stessi post a ogni visita? Quando registri un account, tornerai sempre esattamente dove eri rimasto e potrai scegliere di essere avvisato delle nuove risposte (tramite email o notifica push). Potrai anche salvare segnalibri e votare i post per mostrare il tuo apprezzamento agli altri membri della comunità.
Con il tuo contributo, questo post potrebbe essere ancora migliore 💗
Registrati Accedi