#ActivityPub developers only please: how many items should be in a full collection page?

[evan@cosocial.ca]

#ActivityPub developers only please: how many items should be in a full collection page?

#EvanPoll #poll

[evan@cosocial.ca]

If you implement ActivityPub on the server side, feel free to reply with your page size policy.

[julian@activitypub.space]

@evan@cosocial.ca I don't support polls on NodeBB yet so can't vote, but...

Default perPage is 20*. This is configurable in the admin control panel.

* except for user outboxes. That uses a cursor, not pages.

[smallcircles@social.coop]

@julian @evan

There is no option "it depends.. ". Shouldn't this depend on what you intend to use the collection for? A design choice depending on your use case and domain model.

Or is the missing context to the poll's question is "..in a typical fediverse microblogging environment"?

[smallcircles@social.coop]

@julian @evan

Well.. atm all options have 25% of the vote 😅

[evan@cosocial.ca]

@smallcircles @julian

https://evanp.me/pollfaq#itdepends

[deadwisdom@fosstodon.org]

@evan Totally depends on the collection, shouldn't it?

[naturzukunft2026@mastodon.social]

@evan configurable 😀

[evan@cosocial.ca]

@squinky dude

[evan@cosocial.ca]

@naturzukunft2026 great. What's the default? And what *should* the value be? It's an opinion poll; have an opinion.

[naturzukunft2026@mastodon.social]

@evan Haha, it depends....
in changinggraph.org it is currently 20

[macgirvin@activitypub.space]

@evan@cosocial.ca As an opinion that is likely to be very unpopular... the page size ideally should be set by the client. Only they know their resource makeup and ability to process the returned information.

Otherwise, we tend to run page size between 60 and 100 depending on the content. And if it's less than 100 entries (and especially when returning an id-list rather than a list of activities), we'll usually just send them out without paging.

The desire is to balance resource usage to get the highest rate of information transfer, and those are the only levers we have available, and they (currently) aren't settable by clients, so it seems the best we can do is default to "large chunks".

I admit that I'm not fond of the page size of 12 that I found in Eugen's followers list of over half a million entries (some years ago). That's a lot of network requests and makes their clients work a lot harder than they need to.

[julian@activitypub.space]

@macgirvin I'm okay with that. I just wasn't really aware of any perPage style value that could be passed to servers.

[silverpill@mitra.social]

@julian @macgirvin https://codeberg.org/silverpill/feps/src/branch/main/9f9f/fep-9f9f.md#pagination

[macgirvin@activitypub.space]

@silverpill@mitra.social -- Thanks. Don't know how I missed that, but I'll start using it right away. Cheers.

[django@social.coop]

@macgirvin @general @silverpill @julian that is a very useful thing for clients, thanks for that FEP!

[macgirvin@activitypub.space]

@silverpill@mitra.social @evan@cosocial.ca -- Now I see why I missed it. These additional properties aren't actually defined in the AS spec but only the FEP(?) immediately after implying that the pagination properties were all specified in the AS spec. I get it now, but maybe a wee bit of word-smithing in the FEP could make this a bit clearer.

I guess the question remains how to determine if a site offers cursor-based pagination or recognises 'maxItems' per FEP-9f9f. Guess you need to just try it, maybe with an odd number like 19, and see if what you get back is "consistent" with your request, to see if you should continue trying to use client defined pagination for this site. (This might be another good use case for the server 'capability' mechanism.)

[evan@cosocial.ca]

So, here's the trade-off: adding embedded objects can reduce the number of extra HTTP requests required to render the page of objects. For example, if showing a `followers` collection, adding each actor's name, avatar, and so on can be a real savings.

However, it puts a lot of costs on the server -- looking up cached or local data about each object.

Long story short: adding embedded objects is a pressure towards having smaller page sizes.

[evan@cosocial.ca]

If you're not showing embedded objects, then filling up a collection page is usually just a couple of database queries. And adding more items to the page has very little extra cost.

The bigger your pages are, the fewer requests a client has to make to get all the data.

So, I think if you're not doing embedded objects, the pressure is towards bigger pages.

[evan@cosocial.ca]

There are a couple of other confounding factors.

Adding embedded objects makes supporting HTTP Caching harder. The `ETag` header isn't too hard, but `Last-Modified` is difficult. You need to check not only what the collection page modification date is, but also each of the embedded objects (and take the max date!). It's a pain and most folks don't even implement it.