Returning objects in a collection vs. IDs
Technical Discussion
27
Posts
6
Posters
124
Views
-
In another topic (#Smithereen 0.11 is out!), grishka@mastodon.social says this:
> I have this convention where I return IDs for remote objects (obviously, how else would you authenticate them if not by fetching from the origin servers) but complete objects that are local, to save resources. IIRC I saw Mastodon do it for replies so just copied this behavior
It was in response to a discussion about conversation contexts — a resolvable collection of items that represents a conversation, thread, reply tree, etc.
Most implementors return a collection of object IDs, some return a collection of activities.
What about a mix of object IDs and full objects? That's interesting — it's the first I've heard of it.
My gut reaction is that it's messy... mixing types: strings for some items in the collection and complete objects for others.
... but I think at the end of the day, there are three things we need to acknowledge:
- Most implementors end up converting ids to complete objects anyway by fetching from the remote resource (unless it is not needed)
- Sending additional bytes over the wire is fairly inconsequential (unless I suppose you're sending many Mbits...)
- Including the entire object saves on additional calls if the receiving party does not already know about the object.
- You can trust the objects owned by the server you're requesting data from. (this is fe34)
I wonder if an amendment to f228 is needed silverpill@mitra.social to capture this use-case?
-
@julian How should it be amended? Just add a sentence saying that some collection items might be full objects instead of IDs?
You can trust the objects owned by the server you're requesting data from. (this is fe34)
It's complicated. Those objects are embedded within another object (a collection), and FEP-fe34 currently doesn't recommend trusting embedded objects, except for a few special cases:
An object is the object of a Create activity, and it has the same origin and the same owner as the activity.
An embedded object is identified as a fragment of the wrapping object.
An embedded object is anonymous (doesn't have an ID).This is because ActivityPub Clients can publish arbitrary activities, and a malicious actor may embed an object that is attributed to another actor on the same domain. The server hosting the malicious actor can't realistically verify every embedded object, so if a remote server trusts same-origin embedded objects unconditionally, the malicious actor might succeed in impersonating another actor.
But maybe collection items should be added to the list of exceptions, because collections are almost always server-managed? I need to think about this.
-
@julian How should it be amended? Just add a sentence saying that some collection items might be full objects instead of IDs?
You can trust the objects owned by the server you're requesting data from. (this is fe34)
It's complicated. Those objects are embedded within another object (a collection), and FEP-fe34 currently doesn't recommend trusting embedded objects, except for a few special cases:
An object is the object of a Create activity, and it has the same origin and the same owner as the activity.
An embedded object is identified as a fragment of the wrapping object.
An embedded object is anonymous (doesn't have an ID).This is because ActivityPub Clients can publish arbitrary activities, and a malicious actor may embed an object that is attributed to another actor on the same domain. The server hosting the malicious actor can't realistically verify every embedded object, so if a remote server trusts same-origin embedded objects unconditionally, the malicious actor might succeed in impersonating another actor.
But maybe collection items should be added to the list of exceptions, because collections are almost always server-managed? I need to think about this.
@silverpill @julian @technical-discussion
> The server hosting the malicious actor can't realistically verify every embedded objectHow so? It's a responsibility of that server to authenticate its actors, and restrict them from impersonating other actors on the same server, C2S or not. I don't quite understand the threat model here. In mine, clients are always assumed malicious.
-
@silverpill @julian @technical-discussion
> The server hosting the malicious actor can't realistically verify every embedded objectHow so? It's a responsibility of that server to authenticate its actors, and restrict them from impersonating other actors on the same server, C2S or not. I don't quite understand the threat model here. In mine, clients are always assumed malicious.
grishka@mastodon.social silverpill@mitra.social yes I had the same question.. I assumed this would only occur in a C2S context but even then the server is an intermediary and can verify.
-
grishka@mastodon.social silverpill@mitra.social yes I had the same question.. I assumed this would only occur in a C2S context but even then the server is an intermediary and can verify.
Here's en example:
{ "type": "Create", "actor": "https://social.example/alice" "object": { "type": "Note", "attributedTo": "https://social.example/alice", "attachment": { "type": "Note", "attributedTo": "https://social.example/bob" } } }It contains an embedded
Notethat is attributed to another actor. There are many possible ways to embed an object, and malicious embedding could be difficult to detect for the origin server. -
> messy... mixing types
if mixing types is a concern you can serve ids in object form: {"id": "foo"}
-
Here's en example:
{ "type": "Create", "actor": "https://social.example/alice" "object": { "type": "Note", "attributedTo": "https://social.example/alice", "attachment": { "type": "Note", "attributedTo": "https://social.example/bob" } } }It contains an embedded
Notethat is attributed to another actor. There are many possible ways to embed an object, and malicious embedding could be difficult to detect for the origin server.@silverpill @julian @technical-discussion ideally you would traverse everything and replace anything you don't understand with "id" references.
But anyway, I feel like we're getting too carried away about a very niche aspect of the whole thing. Almost like on that SocialHub forum.
In my own AP extensions I always just act like C2S doesn't exist because I've never seen it used in practice, and it's wildly impractical to use anyway.
-
Here's en example:
{ "type": "Create", "actor": "https://social.example/alice" "object": { "type": "Note", "attributedTo": "https://social.example/alice", "attachment": { "type": "Note", "attributedTo": "https://social.example/bob" } } }It contains an embedded
Notethat is attributed to another actor. There are many possible ways to embed an object, and malicious embedding could be difficult to detect for the origin server.@silverpill @julian @technical-discussion @grishka this wouldn't be "malicious", it would be "unverified". anyone can say anything about anything. it'd be like if i said herman melville wrote moby dick, and you didn't trust me until you verified with herman melville himself that he wrote moby dick. it's just a statement that you can either accept or reject. maybe you don't trust alice to make claims about bob, or maybe you trust social.example to make claims on social.example
-
@silverpill @julian @technical-discussion ideally you would traverse everything and replace anything you don't understand with "id" references.
But anyway, I feel like we're getting too carried away about a very niche aspect of the whole thing. Almost like on that SocialHub forum.
In my own AP extensions I always just act like C2S doesn't exist because I've never seen it used in practice, and it's wildly impractical to use anyway.
@grishka @silverpill @julian @technical-discussion what does exist is publishing an AS2 resource on a web server, which you can do with any "api" you want -- ftp, rsync, ssh, anything that can get a file in a folder served by nginx with the appropriate headers. "c2s" is literally just an endpoint that you can POST some json to and it will publish that json and POST it to others. you can replace it with curl.
-
@grishka @silverpill @julian @technical-discussion what does exist is publishing an AS2 resource on a web server, which you can do with any "api" you want -- ftp, rsync, ssh, anything that can get a file in a folder served by nginx with the appropriate headers. "c2s" is literally just an endpoint that you can POST some json to and it will publish that json and POST it to others. you can replace it with curl.
@trwnh @silverpill @julian @technical-discussion if I remember the spec correctly, that endpoint does have to do at least *some* interpretation of the json — for example, the spec explicitly says that Block activities that come into the inbox "MUST NOT" be exposed in the client's view of the inbox, but instead interpreted and acted upon by the server itself
-
@trwnh @silverpill @julian @technical-discussion if I remember the spec correctly, that endpoint does have to do at least *some* interpretation of the json — for example, the spec explicitly says that Block activities that come into the inbox "MUST NOT" be exposed in the client's view of the inbox, but instead interpreted and acted upon by the server itself
@grishka @silverpill @julian @technical-discussion i think you're referring to how Block should not be sent to an inbox if posted to an outbox? but this is just a rule for outboxes. the user/agent can manually send whatever notifications they want. the user/agent can also follow those rules and just store the relevant json on any properly configured web server.
-
@grishka @silverpill @julian @technical-discussion i think you're referring to how Block should not be sent to an inbox if posted to an outbox? but this is just a rule for outboxes. the user/agent can manually send whatever notifications they want. the user/agent can also follow those rules and just store the relevant json on any properly configured web server.
@grishka @silverpill @julian @technical-discussion point being, you can completely upend a lot of fedi applications' security models with just nginx and curl and subdirectories.
-
@grishka @silverpill @julian @technical-discussion point being, you can completely upend a lot of fedi applications' security models with just nginx and curl and subdirectories.
@trwnh @silverpill @julian @technical-discussion I still wouldn't call that "upending". The actor objects — that reside on that server — contain public keys, which the entire security model of the entire fediverse relies on. So it follows, then, that since a server can serve any arbitrary public key for any of its actors, and the rest of the fediverse will unquestionably trust it, it can also be trusted with any other objects on the same domain.
-
@trwnh @silverpill @julian @technical-discussion I still wouldn't call that "upending". The actor objects — that reside on that server — contain public keys, which the entire security model of the entire fediverse relies on. So it follows, then, that since a server can serve any arbitrary public key for any of its actors, and the rest of the fediverse will unquestionably trust it, it can also be trusted with any other objects on the same domain.
@grishka @silverpill @julian @technical-discussion the key can be on a different keyserver
-
@grishka @silverpill @julian @technical-discussion the key can be on a different keyserver
@trwnh @silverpill @julian @technical-discussion it can in theory but no software currently in existence supports that
-
@trwnh @silverpill @julian @technical-discussion it can in theory but no software currently in existence supports that
@trwnh @silverpill @julian @technical-discussion but even if, the actor object is still the ultimate source of truth as it's the one which contains the key ID
-
@trwnh @silverpill @julian @technical-discussion but even if, the actor object is still the ultimate source of truth as it's the one which contains the key ID
@grishka @silverpill @julian @technical-discussion true but we are talking about mistakes. like taking the origin of the keyId on the http sig instead of following the link to the owner/controller. such assumptions might be true for monoliths, but not everything is a monolith
-
Web Annotations has a neat solution here which is prefers representation, which allows the requestor to ask for minimal or full representation (i.e., just the IDs or the full representations)
You'd still likely want to gate that on authorization
-
@silverpill @julian @technical-discussion ideally you would traverse everything and replace anything you don't understand with "id" references.
But anyway, I feel like we're getting too carried away about a very niche aspect of the whole thing. Almost like on that SocialHub forum.
In my own AP extensions I always just act like C2S doesn't exist because I've never seen it used in practice, and it's wildly impractical to use anyway.
@grishka I am developing a client application where this is a real concern.
But I agree that in general, originating servers are responsible for verification of client data. This part of FEP-fe34 will likely be revised in the future. -
@silverpill @julian @technical-discussion ideally you would traverse everything and replace anything you don't understand with "id" references.
But anyway, I feel like we're getting too carried away about a very niche aspect of the whole thing. Almost like on that SocialHub forum.
In my own AP extensions I always just act like C2S doesn't exist because I've never seen it used in practice, and it's wildly impractical to use anyway.
> and it's wildly impractical to use anyway.
@grishka shush, you! 🤫
Gli ultimi otto messaggi ricevuti dalla Federazione
-
@caohuak Thank you!
-
@silverpill I created an issue.
https://activitypub.software/TransFem-org/Sharkey/-/issues/1212
-
The minutes from this month's ForumWG minutes were just posted.
Ironically, I need to make a separate post to reference it in this group, because we have not invented federated cross-posting yet.
But soon!
-
Hi Nutomic, thanks for joining the forum.
While I have you, I wanted to ask you about the removal of audience from Lemmy. I believe the rationale for it was that it was not used outside of Lemmy, but both Piefed (rimu@piefed.social) and NodeBB use it.
I still think the explicit use of the audience property is more helpful than not.
I discovered another use case for it, although it is a bit of an edge case. Last night a user from lemmy.ca posted to activitypub@community.nodebb.org. Here, there are no followers to that group actor, so it did not federate here. However, one of the relays this site subscribes to did Announce the topic, and when it arrived here, NodeBB could not accurately determine whether the topic/post _actually belonged to activitypub@community.nodebb.org.
Specifically, it is because sometimes users (on Mastodon or other sites) mention group actors but don't intend for the post to be made there. It is not a strong enough signal.
But audience being explicitly set does provide that strong signal. I've updated NodeBB's code to trust the group actor if located in audience.
-
@carloshr as always, my best ideas are the ones that seem so obvious to me that i figure someone else surely has already had them
-
@jonny what a great idea! 🤯
-
what if a torrent contains self-authenticating signed activitypub objects???? what if we could jump back and forth between federation and p2p?
-
@jonny oh shi
