does anyone know who's behind "open.news"?

[ansuz@social.cryptography.dog]

does anyone know who's behind "open.news"?

https://open.news/

> Open.News is the command center for the decentralized newsverse.

Looks like they're ingesting people's fediverse feeds into LLMs and feeding slop to people. I only noticed because it was mostly visiting non-existent or malformed URLs.

> We index live conversations across RSS, Bluesky, and Mastodon so you never miss the story behind the story. FeedBrainer's conversational AI transforms the firehose into a calm, contextual briefing tailored to you.

#slop #scrapers

[ansuz@social.cryptography.dog]

it says "powered by Feedbrainer.ai", and while I didn't find any matches for that I did find https://feedbrain.ai/

I can't tell if open.news is a subsidiary of feedbrain, or just someone depending on their API. If it is them, they're based in Dubai:

https://feedbrain.ai/terms

[ansuz@social.cryptography.dog]

sysadmins should be able to grep for an "open.news" user-agent.

They're not generating enough traffic to cause any problems, but at this point I have zero patience left for LLM companies.

[ansuz@social.cryptography.dog]

looking deeper into my logs it seems like their first attempts to scrape my sites coincided with one of my threads that got picked up by an unusual number of "trending-bots".

my best guess at the moment is that this service is using these bots as a starting point for their scraping campaigns, so I might just start blocking them

[ansuz@social.cryptography.dog]

Update on this: open.news seems to be operated by the same people as "readily.news"

They have a "sign up" button, but if you click through that (do it in a container tab if you want to check) they ask you for your mastodon account's id.

Once you enter your account's id, you should be redirected to your instance's sign-in page.

I haven't gone further than that, but I guess this service is gaining full access to people's accounts in this way, then using those accounts to scrape the network so that their AI can provide a daily digest of what happened on fedi

[ansuz@social.cryptography.dog]

I'm kind of tempted to make a throwaway account on mastodon.social or something like that so that I can see how the rest of this works.

I did it with pixelfed.social, and the page loads as it normally would, but with an extra hidden iframe. I don't see that behaviour with mastodon.social.

Maybe this only works with particular fedi instances that lack some security feature?

[ansuz@social.cryptography.dog]

Okay, so it's just the standard OAuth workflow, but I was otherwise right, and their app basically gets full access to your account just as a mobile app for mastodon might 😬

[ansuz@social.cryptography.dog]

hey @Mastodon,

are you aware that https://readily.news/ is leveraging full access to users' mastodon accounts to scrape followers-only fediverse posts?

[ansuz@social.cryptography.dog]

I found a mastodon.social account with a link to readily.news in their bio:

https://mastodon.social/@librenews

I can't tell if he's affiliated with the project/company or if they've injected their link into his bio after he'd given them access to his account. Both seem plausible.

[ansuz@social.cryptography.dog]

Oh, and it seems readily.news was discussed at FediForum 2023:

https://fediforum.org/2023-03/session/4-c/

Do I know anyone who attended that who remembers any relevant details?

Matt's name appears there too, so it's really looking like it might be his project, but who knows 🤷

[ansuz@social.cryptography.dog]

I'm working on a (hopefully brief) write-up of everything I know about this latest fediverse scraper

[ansuz@social.cryptography.dog]

As usual, this post turned out somewhat longer than I'd originally intended.

This is pretty much everything I know "readily.news", the latest non-consensual service attempting to scrape the Fediverse for all it is worth:

https://cryptography.dog/blog/what-little-i-know-about-readily-news/

I've done my part. You'll need to supply your own torches and pitchforks.

[ansuz@social.cryptography.dog]

It's been a few days since I posted about https://readily.news AKA "open.news", a service which:

1. asks for complete access to your Mastodon/fedi account

2. ingests whatever your account can see via your account and summarizes it using LLMs (seemingly from OpenAI?)

3. sends you a daily, personalized newsletter

It's a particularly bad kind of scraper because it basically hijacks existing community infra to do the scraping for it.

Because accounts' host instances are the actors gathering up all the content there's no way for remote servers to detect which of their followers' accounts have been compromised, nor to block their posts from ending up in the hands of the upstream LLM providers.

We'll probably need admins of affected instances to run a database query to detect and revoke permissions granted to this service via OAuth to limit its access.

I asked the guy who
the guy who appears to be behind it (https://mastodon.social/@librenews
) if he could confirm his affiliation, but he doesn't actually seem to be very active on Mastodon (preferring Bluesky) and so he still hasn't responded.

I'm actually a little surprised at how little reaction there's been to this based on how quickly other scrapers were run off the network, but I get that people are busy.

If you want more details, the specifics of my investigation are in this post:

https://cryptography.dog/blog/what-little-i-know-about-readily-news/

...and I'd appreciate if others could corroborate my findings.

#infosec #fediscrapers #scrapers #LLMs #AI

[homebrewandhacking@mastodon.ie]

@ansuz

Thanks for the heads up. I've put the block in.

Had you considered adding # fediblock to your post? Seems like a moderator scale problem.

@davey_cakes

[ansuz@social.cryptography.dog]

In case anybody who is more deeply familiar with Mastodon's database internals feels like helping to shut this service down:

I think it would be great to have a command instance admins could run to identify which (if any) of the accounts they host have handed over account access to Readily.news.

It achieves access through the OAuth confirmation dialog shown in the attached screenshot

[ansuz@social.cryptography.dog]

Alternatively, if anybody else feels they can toot about this in a way that gets more traction, please do!

Maybe I ought to change the headline of my article to "Techbro builds Cambridge Analytica for the Fediverse" or something more inflammatory like that?