FEP-4f05: Soft Deletion
-
Hi!
Sorry for being late to the party.
I understand the need to be able to undo deletions, this is something we face at Mastodon for the edge case of appealing moderation decisions (currently, most moderation decisions can be reversed upon appeal, but not post deletion).
I have some concern with the FEP as it stands regarding performances, and ensuring consistency wrt. chronology of events, caching and possible out-of-order activities.
Indeed, performance-wise, the FEP asks recipients of a \
Delete\to fetch the object that has just been deleted. This means that for a post that has, over its lifetime, reached a thousand different servers, in addition to ideally reaching all of those servers again (either directly or through inbox forwarding), the authoring server must now handle all of these servers fetching the now-deleted post all at once. I fear this is an especially bad instance of the thundering herd issue.As for ensuring consistency wrt. chronology of events, we face a lot of challenges:
- depending on their architecture, servers may emit outgoing activities (or process incoming ones) out-of-order (for instance, Mastodon queues jobs into work queues, but if there are multiple workers, a later job can finish before an earlier job does)
- due to network failures, servers may fail to deliver an activity on time and retry later
- due to caching (e.g. Mastodon offers short-time caching on reverse proxies, but does not invalidate the reverse-proxy cache when the resource is changed), fetched data might actually be older than just-delivered data
The ActivityPub primer makes note of this but offers no solutions besides “The receiving server, if it receives an activity that refers to an unknown activity, should store that activity for later processing.” While this is relatively easy to do when an object cannot be brought back once it’s deleted, this breaks done if you can undo the \
Delete\, and I have seen no solution offered for that in the current FEP.Using \
published\in activities and \published\/\updated\or similar in objects might help with that, but I’m afraid this might not be enough because of the seconds resolution of \xsd:Datetime\(and it would require extra care that the lifecycle of an object is indeed serialized with a monotonic time).Regarding the performance issue, and avoiding the thundering herd problem, one could simply embed the object itself (so, a
Deletewith an expandedTombstoneinobject) into the activity. You could additionally sign it (LD Signature) or attach a proof (Object Integrity Proofs) if necessary.As for sub-second resolution of
updated/published... isxsd:Datetimerequired? I've honestly just been sending ISO Strings, which include millisecond-level accuracy. -
@claire@social.sitedethib.com I re-read the text of the FEP and noted the following:
> When a Delete activity is encountered, the referenced object MAY be either the full object or a reference to one.
>
> If object is a reference, the server MUST request the object (via its id) from the origin server directly.Emphasis is mine. In situations where you choose to embed the full object in the activity, then you are not bound by the
MUSTto refetch the object.Now, when talking about hard deletes, you cannot literally embed a non-existent object, so a re-fetch would be necessary, although I am hoping that 404 handlers are a great deal faster.
I like
published. I can add that in to the FEP if it makes it easier to handle situations where multiple Deletes and Updates are encountered out-of-rder due to network congestion, parallel processing, etc. -
\
julian2:xsd:dateTime\is required as per https://www.w3.org/TR/activitystreams-vocabulary/#dfn-published but i skimmed over the definition too fast, it definitely allows fractional seconds!Emphasis is mine. In situations where you choose to embed the full object in the activity, then you are not bound by the
MUSTto refetch the object.It appears I must have read too fast once again, and was confused by the “Unexpected responses” section.
julian2:Now, when talking about hard deletes, you cannot literally embed a non-existent object, so a re-fetch would be necessary, although I am hoping that 404 handlers are a great deal faster.
That can still be an issue, negative hits are still expensive and in general you may not want to cache them (to avoid an attacker targeting something that does not exist yet).
-
julian2:
If object is a reference, the server MUST request the object (via its id) from the origin server directly.
i think this requirement can be removed, as the behavior on receiving a Delete is up to the receiver and not the sender. that's also where the issue lies -- receivers assuming Delete is a permanent removal. any or all of the following behaviors on receiving a Delete are "valid" in some sense:
- do nothing to the object, just store the activity
- expunge object from HTTP cache
- expunge object from AS2/RDF dataset
- edit the object to say it is "deleted"
- convert object to a Tombstone
- prevent reuse of the object.id
- fetch the object using HTTP GET and handle caching/refetching using HTTP cache control headers
having a reference doesn't imply needing to fetch it if you already have information about it. if you don't already have information about it then you can also choose not to fetch on Delete activities. the point of having an id is that you can choose whether or not to obtain additional information! that's what linked data is founded on -- the linking. every link is in effect a boundary between two records of information.
if the goal is to prevent receivers from completely purging an object, then you can't really do this. if the goal is to stop receivers from preventing reuse of the id, then recommend that they SHOULD NOT do this.
more generally i would ask you to consider two different senses of "deletion":
- Delete / Undo Delete
- Update(object.formerType=object.type, object.type=Tombstone) / Update(object.type=object.formerType)
a Tombstone is still an Object and can have all the properties of Object btw, so it's valid to have this:
type: TombstoneformerType: Notecontent: "[deleted]"attributedTo:or this:
type: TombstoneformerType: Notecontent: "the text is still there but the account was deleted"attributedTo: type: Tombstone formerType: Personor this:
type: TombstoneformerType: Notecontent: "the text is still there but the account was deleted"attributedTo: # GET someone HTTP/1.1# HTTP/1.1 404 Not Found -
julian2:
If object is a reference, the server MUST request the object (via its id) from the origin server directly.
i think this requirement can be removed, as the behavior on receiving a Delete is up to the receiver and not the sender. that's also where the issue lies -- receivers assuming Delete is a permanent removal. any or all of the following behaviors on receiving a Delete are "valid" in some sense:
- do nothing to the object, just store the activity
- expunge object from HTTP cache
- expunge object from AS2/RDF dataset
- edit the object to say it is "deleted"
- convert object to a Tombstone
- prevent reuse of the object.id
- fetch the object using HTTP GET and handle caching/refetching using HTTP cache control headers
having a reference doesn't imply needing to fetch it if you already have information about it. if you don't already have information about it then you can also choose not to fetch on Delete activities. the point of having an id is that you can choose whether or not to obtain additional information! that's what linked data is founded on -- the linking. every link is in effect a boundary between two records of information.
if the goal is to prevent receivers from completely purging an object, then you can't really do this. if the goal is to stop receivers from preventing reuse of the id, then recommend that they SHOULD NOT do this.
more generally i would ask you to consider two different senses of "deletion":
- Delete / Undo Delete
- Update(object.formerType=object.type, object.type=Tombstone) / Update(object.type=object.formerType)
a Tombstone is still an Object and can have all the properties of Object btw, so it's valid to have this:
type: TombstoneformerType: Notecontent: "[deleted]"attributedTo:or this:
type: TombstoneformerType: Notecontent: "the text is still there but the account was deleted"attributedTo: type: Tombstone formerType: Personor this:
type: TombstoneformerType: Notecontent: "the text is still there but the account was deleted"attributedTo: # GET someone HTTP/1.1# HTTP/1.1 404 Not FoundOkay, I am perfectly fine to relax the requirement from a MUST to a SHOULD.
Does that resolve the thundering herd concern acceptably?
Other solutions would entail:
Setting explicitnullasobject(yes @trwnh@mastodon.social this is yet another example of a place where null makes sense!) if the object is hard deleted.- Sending an
ETagheader with the Delete activity. When re-requesting, send that same value inIf-Modified-Sinceand the receiver can opt to terminate execution early with an HTTP 304.
-
@julian how does null have anything to do with this? Delete null doesn't make sense
-
@julian how does null have anything to do with this? Delete null doesn't make sense
@trwnh@mastodon.social hm, you're right. I should stop thinking about FEPs after business hours.
-
@julian unrelated but i am also wondering why the mention changed from my socialhub account to my mastodon account 🤔
-
@julian unrelated but i am also wondering why the mention changed from my socialhub account to my mastodon account 🤔
@trwnh@mastodon.social no particular reason, except I think mentions to SocialHun accounts don't work?
-
@julian :weary:
i can't keep doing replies in less than 500 characters lmao
-
@julian :weary:
i can't keep doing replies in less than 500 characters lmao
@trwnh@mastodon.social I forked this thread out into a new thread (so I don't think it'll keep showing up on SocialHub, but who the heck knows when federation is concerned lol)
As for 500 chars, perhaps it's high time you switched to an instance with looser character limits...
Except your content wouldn't migrate over wonk wonk
Feed RSS
Gli ultimi otto messaggi ricevuti dalla Federazione
-
@trwnh@mastodon.social I forked this thread out into a new thread (so I don't think it'll keep showing up on SocialHub, but who the heck knows when federation is concerned lol)
As for 500 chars, perhaps it's high time you switched to an instance with looser character limits...
Except your content wouldn't migrate over wonk wonk
-
@julian :weary:
i can't keep doing replies in less than 500 characters lmao
-
@trwnh@mastodon.social no particular reason, except I think mentions to SocialHun accounts don't work?
-
@julian unrelated but i am also wondering why the mention changed from my socialhub account to my mastodon account 🤔
-
@trwnh@mastodon.social hm, you're right. I should stop thinking about FEPs after business hours.
-
@julian how does null have anything to do with this? Delete null doesn't make sense
-
Okay, I am perfectly fine to relax the requirement from a MUST to a SHOULD.
Does that resolve the thundering herd concern acceptably?
Other solutions would entail:
Setting explicit null as object (yes @trwnh@mastodon.social this is yet another example of a place where null makes sense!) if the object is hard deleted. Sending an ETag header with the Delete activity. When re-requesting, send that same value in If-Modified-Since and the receiver can opt to terminate execution early with an HTTP 304.
-
julian2:
If object is a reference, the server MUST request the object (via its id) from the origin server directly.
i think this requirement can be removed, as the behavior on receiving a Delete is up to the receiver and not the sender. that's also where the issue lies -- receivers assuming Delete is a permanent removal. any or all of the following behaviors on receiving a Delete are "valid" in some sense:
do nothing to the object, just store the activityexpunge object from HTTP cacheexpunge object from AS2/RDF datasetedit the object to say it is "deleted"convert object to a Tombstoneprevent reuse of the object.idfetch the object using HTTP GET and handle caching/refetching using HTTP cache control headershaving a reference doesn't imply needing to fetch it if you already have information about it. if you don't already have information about it then you can also choose not to fetch on Delete activities. the point of having an id is that you can choose whether or not to obtain additional information! that's what linked data is founded on -- the linking. every link is in effect a boundary between two records of information.
if the goal is to prevent receivers from completely purging an object, then you can't really do this. if the goal is to stop receivers from preventing reuse of the id, then recommend that they SHOULD NOT do this.
more generally i would ask you to consider two different senses of "deletion":
Delete / Undo DeleteUpdate(object.formerType=object.type, object.type=Tombstone) / Update(object.type=object.formerType)a Tombstone is still an Object and can have all the properties of Object btw, so it's valid to have this:
type: TombstoneformerType: Notecontent: "[deleted]"attributedTo:or this:
type: TombstoneformerType: Notecontent: "the text is still there but the account was deleted"attributedTo: type: Tombstone formerType: Personor this:
type: TombstoneformerType: Notecontent: "the text is still there but the account was deleted"attributedTo: # GET someone HTTP/1.1# HTTP/1.1 404 Not Found