Flow control

[evan@activitypub.space]

I've been thinking lately about flow control. That's a feature of some networks where a receiver can tell a sender to slow down its sending rate to match the receiver's processing rate.

In TCP flow control, the receiving host returns a receiving buffer size in its acknowledgement segment, so the sending host know how much data it can send without overflowing the buffer.

I wonder if there are ways that a receiving ActivityPub protocol server could tell the sending server to slow down? Maybe we could reuse some of the RateLimit headers.

Another option would be a special header that says how big your incoming activity queue is. "I have a very long processing queue right now, please keep stuff in your outgoing queue for a while."

[evan@activitypub.space]

Obviously the other option is to keep sending until the receiving server fails, and then do exponential backoff until the remote server can process your activity again.

But I'm looking for something more adaptive that adjusts the sending rate before the receiver fails.

[fentiger@mastodon.social]

@evan https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/429 ?

[evan@activitypub.space]

fentiger@mastodon.social yes, but by that point you've failed. What I want is to have more adaptive sending, so you don't get to the 429 code.

[fentiger@mastodon.social]

@evan No, you haven't. You just need to back off and retry, just as you do for an ACK timeout in TCP.

[julian@activitypub.space]

fentiger@mastodon.social yes I was also thinking of the venerable 429.

evan the thing is, both approaches rely on the receiving end to support it. I would make the argument that a 429 would have nominal support from some implementations, but then again I wouldn't actually bet money on it.

Either way you're looking at an FEP to establish flow control signalling or 429 handling. I'd then make the argument that putting together guidance for 429 (+ exponential back off) is easier than specifying flow control logic.

[jamesmarshall@sfba.social]

@evan does AP have a way to send control messages, just meant for the receiving server?

[rimu@piefed.social]

Currently whenever a POST is sent to an inbox the response body is ignored. The http status code is checked for errors but that's it. Perhaps some more complex information about the state of the receiving ingress queue could be conveyed in the body.

[evan@activitypub.space]

fentiger@mastodon.social Flow control is the feature in TCP where the receiving host, in its ACK, can send back information about its remaining buffer space. The sending host can slow down its sending rate to prevent overflowing that buffer. If this works correctly, no error is ever sent.

[silvermoon82@wandering.shop]

@evan
HTTP 420/CHILL OUT?

Actually I do see HTTP 429 TOO MANY REQUESTS, which seems exactly what we want. I don't know what support is like though, I've not seen explicit support for it in the few bits of AP Federation code I've taken apart.