Expanding collections on delivery

[julian@activitypub.space]

@silverpill@mitra.social (and others) reference this line from the spec re: delivery:

> If a recipient is a Collection or OrderedCollection, then the server MUST dereference the collection (with the user's credentials) and discover inboxes for each item in the collection.

— https://www.w3.org/TR/activitypub/#delivery

Was there a specific use case/story that corresponded with this directive?

The only commonly addressed collection I can think of is a followers collection, and:

1. It's not common to address somebody else's followers collection.
2. Even if it were, 7.1.2 specifically refers to inbox forwarding to manage delivery.

So am I missing something, are there other user collections that are often addressed (and expected to be expanded), or should we remove this from the spec?

cc @evan @trwnh@mastodon.social

[trwnh@mastodon.social]

@julian wrote:

> Was there a specific use case/story

have you ever used diaspora* "aspects", google+ "circles", facebook "privacy lists", instagram "close friends", or twitter "circles"? this spec language is intended to allow for this kind of mailing-list style usage in to/cc/audience/bto/bcc.

> The only commonly addressed collection [...] is a followers collection

this is a limitation with mastodon et al who only care about followers and assume followers are the only collection mattering.

[trwnh@mastodon.social]

@julian

> should we remove

if this is removed, then you lose a very valuable ability to address things like "the audience of a conversation" or "the members of a group". i would strongly recommend NOT removing it.

current behavior in mastodon is to upgrade "direct" posts to "limited" when a recipient is detected that doesn't dereference to either a user or a user's followers. they get presented in the mastodon api as "followers" though (for compat), so further interactions lose this nuance.

[trwnh@mastodon.social]

@julian

> 7.1.2

wrt inbox forwarding, this only helps when addressing collections of *someone else*, where the contents are private. for your own collections, unless you plan to deliver all such activities to yourself with the expectation that you will forward them (why didn't the outbox do it for you?^1), it doesn't help you.

^1: if the outbox doesn't have your credentials, then it can't do this. in this case, you or your client is responsible for deliveries, and the outbox only publishes.

[trwnh@mastodon.social]

@julian

i would be fine with removing this collection expansion behavior from outbox delivery if it was decided that outbox delivery itself is problematic and should be removed -- probably in favor of the client being responsible for sending notifications, where the client can apply whatever logic it wants.

this is kinda what mastodon does right now as a monolith -- it is both the activitypub client (submitting to its internal outbox) and also the http agent for linked data notifications.

[trwnh@mastodon.social]

@julian there's probably a bunch of open issues on the https://github.com/w3c/activitypub/issues tracker regarding the problems with outbox delivery. those problems might be addressable all together, but it might instead make more sense to conceive of a sort of "LDN proxy" which handles deliveries instead (and holds your keys as an HTTP agent sending signed messages)

[julian@activitypub.space]

@trwnh@mastodon.social so collection expansion is mainly for when I am sending an activity to collections that I control?

Then I'm wondering why this needs to be explicitly spelled out and required because it seems to be inferred already from a UX perspective.

[trwnh@mastodon.social]

@julian i don't think it's "inferred", and leaving ambiguous cases up to inference in specification is typically called "unspecified behavior" ;)

say you are an outbox and you get an activity to: some id. you deref the id and get some info. what do you do?

- in all cases, if it has an `inbox`, you send an LDN to that id if you can.
- in case it's an as:Collection, you iterate over its items in theory and repeat step 1 recursively. (this is also problematic because it can be both paged+unpaged)

[trwnh@mastodon.social]

@julian now remove the requirement. what do you do instead?

- if it has ldp:inbox, send an LDN

...and that's it. at no point were you ever told or required to do anything else, so your followers/audience/members/etc will never get the activity even if addressed, because the collection was never expanded.

[julian@activitypub.space]

@trwnh@mastodon.social said in Expanding collections on delivery:
> say you are an outbox and you get an activity to: some id. you deref the id and get some info. what do you do?

Simple. My outboxes send a "not supported" HTTP tag 🤣

But I'm being facetious.

From a C2S standpoint I suppose that makes sense. Thanks.

[trwnh@mastodon.social]

@julian well, sure, with a monolithic implementation, the client and the outbox and the delivery agent are all the same app. but they don't have to be. the model is that the client submits to the outbox, and the outbox could also talk to a separate delivery agent internally. it's all opaque from outside the outbox. your internal "outbox" is the code that serializes activities and sends them to the delivery workers.

[silverpill@mitra.social]

@julian Yes, I think in practice expansion should be performed only for local collections.

the server MUST dereference the collection (with the user's credentials) is confusing, because it sounds like we're talking about remote collections here.

@trwnh

[trwnh@mastodon.social]

@silverpill @julian @technical-discussion

a "local collection" might still have access control on it.

(the interface being assumed throughout the AP spec is HTTP, or at least HTTP semantics; "with the user's credentials" in this case means using an Authorization header that lets the outbox access the collection. it's only confusing if you have a monolith with no boundaries between the outbox and anything else; in that case it'd be "lookup the collection in your db/store/etc")

[trwnh@mastodon.social]

@silverpill @julian @technical-discussion

example: alice and bob on site.example each have followers collections, but alice can't see bob's followers. if alice addresses bob's followers collection, then alice's outbox can't deliver to bob's followers. alice must address bob, and bob can choose to forward to bob's followers (inbox forwarding)

if site.example has a collection of "local users" that alice can see, then alice can address it and alice's outbox can deliver to items

[trwnh@mastodon.social]

@silverpill @technical-discussion it's part of the outbox delivery algorithm, which bridges between c2s and s2s. the intention is that the outbox publishes activities via c2s, but then optionally delivers based on addressing properties via s2s

(this ends up having other issues in practice due to the lack of an envelope, but at least the intent of "relevant activities should trigger notifications for relevant entities" makes sense, per 6.1 clients "look at" some relevant props)