Skip to content

Piero Bosio Social Web Site Personale Logo Fediverso

Social Forum federato con il resto del mondo. Non contano le istanze, contano le persone

Alright it's late and i need to go to bed, but here's a draft FEP to do full account migration with posts and whatever other kinda objects you want to bring with you.

  • Of course, not trying to say otherwise. Just that there is a preconception that jonny@neuromatch.social had to post there as part of the FEP process, which even I was under the assumption of.

    We chatted about that and you disabused me of that notion :smirk:

  • @kopper i was looking for an instance metadata item that was just some list of tokens that was "the list of things that the instance supports," and i could have sworn it existed, but i couldn't find it when i looked. most fedi apps (and even most LD apps) don't actually dereference the URIs in a context and treat them as tokens anyway, but yes failure to resolve terms is a big problem and am not a fan of DNS-based linked data.

    jonny@neuromatch.social kopper@not-brain.d.on-t.work Perhaps Capability Discovery is what you might be looking for? There is some implementor support for this one.

  • Alright it's late and i need to go to bed, but here's a draft FEP to do full account migration with posts and whatever other kinda objects you want to bring with you. It's a trivial expansion of existing ActivityPub/streams systems and supports gradual migration as it's implemented and after an account migration. It should be possible to migrate pretty much everything this way, both private and public objects.

    criticism, feedback, revisions, etc. welcome - i don't think this is a "final version" and there are certainly things i overlooked.

    https://codeberg.org/fediverse/fep/src/commit/e6f7b7ce32aa6f84dcfa7bfdc10fd65119d75984/fep/1580/fep-1580.md

    https://codeberg.org/fediverse/fep/pulls/692

    @jonny you're amazing 😍

  • Alright it's late and i need to go to bed, but here's a draft FEP to do full account migration with posts and whatever other kinda objects you want to bring with you. It's a trivial expansion of existing ActivityPub/streams systems and supports gradual migration as it's implemented and after an account migration. It should be possible to migrate pretty much everything this way, both private and public objects.

    criticism, feedback, revisions, etc. welcome - i don't think this is a "final version" and there are certainly things i overlooked.

    https://codeberg.org/fediverse/fep/src/commit/e6f7b7ce32aa6f84dcfa7bfdc10fd65119d75984/fep/1580/fep-1580.md

    https://codeberg.org/fediverse/fep/pulls/692

    This looks fine to me, great to see many edge cases considered.

  • i'm almost disappointed bc it's so simple and obvious. there are a lot of caveats and requirements there because i was trying to make it clear enough to be implementable, but really it's as simple as doing this: https://neuromatch.social/@jonny/115338330944599542

    @jonny I'm seeing notes here for the technical challenge of performing a migration of objects between two willing hosts given a user in good standing with sufficient credentials. I think it's great that you have written that down.

    But I think the reason we don't have this yet is because that's only 20% of the problem. The other 80% of the challenge is contained in that first sentence of mine.

    • How do we signal this intention to migrate to both hosts?
    • How do we validate that both hosts are willing?
    • How do the hosts validate credentials and good standing of both the migrating user and each other?
    • What obligations does each party have to propagate and maintain this effective URL redirect?
    • What controls does each party have in order to manage this transition process?
    • How do we manage moderation of the incoming content?
    • How will this impact current user agreements on most instances?

    Any specification here needs to address admins and moderators as key players.

  • Alright it's late and i need to go to bed, but here's a draft FEP to do full account migration with posts and whatever other kinda objects you want to bring with you. It's a trivial expansion of existing ActivityPub/streams systems and supports gradual migration as it's implemented and after an account migration. It should be possible to migrate pretty much everything this way, both private and public objects.

    criticism, feedback, revisions, etc. welcome - i don't think this is a "final version" and there are certainly things i overlooked.

    https://codeberg.org/fediverse/fep/src/commit/e6f7b7ce32aa6f84dcfa7bfdc10fd65119d75984/fep/1580/fep-1580.md

    https://codeberg.org/fediverse/fep/pulls/692

    @jonny Could I possibly encourage you to add a brief 1-2 sentences-or-so "Social Considerations" section -- or, well, the way you organized your headings a "Social" heading in your "Discussion" section -- next to "Privacy" and "Security".

    I think in this case the basic Social motivation and implication for this specifically is reasonably obvious to everyone in the discussion right now, but because that obviousness won't be as available to people looking back at this during a more future implementation and won't be as available to people reading this looking for interactions with their own drafting FEP at a later point, it's worth including in the text itself.

    If you don't want to or if this isn't the sort of feedback you're looking for no worries and sorry for imposing.
    (Alternatively if you'd be up for it but have many other things atm, and if you trust me enough, I could try to grab time from dodging between the holidays at the moment and do a brief one for you, if you want.)

  • @jonny I'm seeing notes here for the technical challenge of performing a migration of objects between two willing hosts given a user in good standing with sufficient credentials. I think it's great that you have written that down.

    But I think the reason we don't have this yet is because that's only 20% of the problem. The other 80% of the challenge is contained in that first sentence of mine.

    • How do we signal this intention to migrate to both hosts?
    • How do we validate that both hosts are willing?
    • How do the hosts validate credentials and good standing of both the migrating user and each other?
    • What obligations does each party have to propagate and maintain this effective URL redirect?
    • What controls does each party have in order to manage this transition process?
    • How do we manage moderation of the incoming content?
    • How will this impact current user agreements on most instances?

    Any specification here needs to address admins and moderators as key players.

    @gatesvp
    All these questions are addressed in the FEP except moderation, and I'm talking with masto devs about what would be good there

  • @jonny Could I possibly encourage you to add a brief 1-2 sentences-or-so "Social Considerations" section -- or, well, the way you organized your headings a "Social" heading in your "Discussion" section -- next to "Privacy" and "Security".

    I think in this case the basic Social motivation and implication for this specifically is reasonably obvious to everyone in the discussion right now, but because that obviousness won't be as available to people looking back at this during a more future implementation and won't be as available to people reading this looking for interactions with their own drafting FEP at a later point, it's worth including in the text itself.

    If you don't want to or if this isn't the sort of feedback you're looking for no worries and sorry for imposing.
    (Alternatively if you'd be up for it but have many other things atm, and if you trust me enough, I could try to grab time from dodging between the holidays at the moment and do a brief one for you, if you want.)

    @gaditb
    I briefly addressed this in the intro but I agree its worth expanding on

  • @gaditb
    I briefly addressed this in the intro but I agree its worth expanding on

    @jonny (My low-key agenda is, I kinda think every FEP should have a section about it. It's like, part of the discussion and the years of prior context people are discussing from,,)

  • @jonny (My low-key agenda is, I kinda think every FEP should have a section about it. It's like, part of the discussion and the years of prior context people are discussing from,,)

    @jonny Okay actually more directly-specufic-to-content thought. Although one that I don't know the right answer to:

    > Constitutively private objects like bookmarks, blocks, and mutes SHOULD be ingested during the ingestion routine for the purposes of creating a seamless migration from the perspective of the migrating Actor, but MUST NOT be included in the migration collection.

    For Blocks in particular -- In cases where, as is allowed by SHOULD, a migration (either source or destination) implements migrating posts but /not/ Blocks,
    that silently opens up old posts to now no longer be blocked from interaction, which feels particular significant currently (where we don't have shared blocklists) since it means primarily accounts that have been specifically blocked on an individual basis -- likely (? this is an assumption) for recurringly-applicable reasons.

  • @jonny Okay actually more directly-specufic-to-content thought. Although one that I don't know the right answer to:

    > Constitutively private objects like bookmarks, blocks, and mutes SHOULD be ingested during the ingestion routine for the purposes of creating a seamless migration from the perspective of the migrating Actor, but MUST NOT be included in the migration collection.

    For Blocks in particular -- In cases where, as is allowed by SHOULD, a migration (either source or destination) implements migrating posts but /not/ Blocks,
    that silently opens up old posts to now no longer be blocked from interaction, which feels particular significant currently (where we don't have shared blocklists) since it means primarily accounts that have been specifically blocked on an individual basis -- likely (? this is an assumption) for recurringly-applicable reasons.

    @gaditb
    Really good point, I have some ideas for language here, and there does need to be a bit more specificity about how to handle visibility for e.g. instance software that may not have blocks importing from software that does - basically that visibility must be at least as constrained as the source object

  • @gaditb
    Really good point, I have some ideas for language here, and there does need to be a bit more specificity about how to handle visibility for e.g. instance software that may not have blocks importing from software that does - basically that visibility must be at least as constrained as the source object

    @jonny Or at minimum alerted to the user, possible?

    But does that even. make sense? Most of the stuff here is, because of FEPs' linage from RFCs, spoken in the language of protocols and behavior/format specifications -- is it coherent to define a part of the specifications as the... I guess, social protocol, interface, etc., of a compliant piece of software towards its users?

    (On the one hand those are literally the original pre-computer definitions of "protocol" and "compliant", but on the other hand that is CLEARLY misusing the terms there.)

    And even if it is coherent, is it possible to actually do it without becoming a "the Passkey people yelling at and threatening the Keepass devs"?

  • @jonny Or at minimum alerted to the user, possible?

    But does that even. make sense? Most of the stuff here is, because of FEPs' linage from RFCs, spoken in the language of protocols and behavior/format specifications -- is it coherent to define a part of the specifications as the... I guess, social protocol, interface, etc., of a compliant piece of software towards its users?

    (On the one hand those are literally the original pre-computer definitions of "protocol" and "compliant", but on the other hand that is CLEARLY misusing the terms there.)

    And even if it is coherent, is it possible to actually do it without becoming a "the Passkey people yelling at and threatening the Keepass devs"?

    @gaditb
    I think both are needed and have their place. And in this case I am actually not sure if there is an opposing party to accidentally be perceived as yelling at - as far as I can tell people pretty universally agree that you should retain control of the things you said and did while moving around and not always lose everything (maybe there is some disagreement about what to move, but the FEP is purposely designed to leave that up to the implementation and ideally the actor)

  • @gatesvp
    All these questions are addressed in the FEP except moderation, and I'm talking with masto devs about what would be good there

    @jonny

    All these questions are addressed in the FEP except moderation

    You and I seem to be reading different docs here.

    I'm looking at FEP-73cd which looks like the best summary of the various complex cases and it still has several Required use cases that don't have an FEP specification. (table at the bottom)

    I'm looking at FEP-1580 and it seems to be operating under the base assumption that "everyone is OK with this". It doesn't use the word "admin" or "administrator" even once. It never addresses "mod" or "moderator" as one of the players in this process.

    The words "agreement" or "contract" appear zero times in the specification.

    I'm talking with masto devs about what would be good there

    That's a reasonable step, but again, I don't think that's the key problem. None of this matters without Masto Admins and Masto Mods also on board.

    Every failure case in these specs falls on Admins and Mods to resolve, shouldn't they be first consulted?

  • @jonny

    All these questions are addressed in the FEP except moderation

    You and I seem to be reading different docs here.

    I'm looking at FEP-73cd which looks like the best summary of the various complex cases and it still has several Required use cases that don't have an FEP specification. (table at the bottom)

    I'm looking at FEP-1580 and it seems to be operating under the base assumption that "everyone is OK with this". It doesn't use the word "admin" or "administrator" even once. It never addresses "mod" or "moderator" as one of the players in this process.

    The words "agreement" or "contract" appear zero times in the specification.

    I'm talking with masto devs about what would be good there

    That's a reasonable step, but again, I don't think that's the key problem. None of this matters without Masto Admins and Masto Mods also on board.

    Every failure case in these specs falls on Admins and Mods to resolve, shouldn't they be first consulted?

    @jonny

    This FEP is written to minimize the responsibility of the source instance,

    You have this line right there in the spec, and I just don't understand this assumption.

    By minimizing the responsibility of the Source instance, you're dumping all of the work on the Target and 3rd Party instances. But they're not the primary actors here.

    The key parts of this chain are the Actor and the Source. They trust each other.

    • They have an established User Agreement in place
    • The Source has an established history of Actor behaviour
    • The Actor has a high enough trust in the Source that they have published enough that it justifies migration

    When Actor signs up for an account with Target, that new Target User Agreement doesn't assume that Actor is going to bring 200k old posts with them.

    If I'm Target, I don't even want this. My default answer here is "no, you cannot do this without talking to me first". We don't have that relationship yet.

    This frankly sounds like a giant spam vector.

  • @jonny

    This FEP is written to minimize the responsibility of the source instance,

    You have this line right there in the spec, and I just don't understand this assumption.

    By minimizing the responsibility of the Source instance, you're dumping all of the work on the Target and 3rd Party instances. But they're not the primary actors here.

    The key parts of this chain are the Actor and the Source. They trust each other.

    • They have an established User Agreement in place
    • The Source has an established history of Actor behaviour
    • The Actor has a high enough trust in the Source that they have published enough that it justifies migration

    When Actor signs up for an account with Target, that new Target User Agreement doesn't assume that Actor is going to bring 200k old posts with them.

    If I'm Target, I don't even want this. My default answer here is "no, you cannot do this without talking to me first". We don't have that relationship yet.

    This frankly sounds like a giant spam vector.

    @jonny

    Inherent in your specification is the assumption that Target's default stance is simply to accept all incoming transfer requests as legitimate.

    This is a very Actor-centric view: "It's my content, I can bring it wherever I want, this should be as seamless as possible". But that's an oversimplification of the Publisher (Source) / Actor relationship that's actually in place.

    And I don't think that's a fair assumption on behalf of Target. In fact, I don't even think it's a safe assumption for the network as a whole, because it's a giant spam vector. None of this is should be automatic, Target needs an active sign-off on content transfers.

    I think this is relevant, because an active sign-off from both Source and Target actually changes parts of these specifications. They don't have to drip transfer, they can coordinate bulk operations, they can negotiate size limits, etc.

  • @jonny

    All these questions are addressed in the FEP except moderation

    You and I seem to be reading different docs here.

    I'm looking at FEP-73cd which looks like the best summary of the various complex cases and it still has several Required use cases that don't have an FEP specification. (table at the bottom)

    I'm looking at FEP-1580 and it seems to be operating under the base assumption that "everyone is OK with this". It doesn't use the word "admin" or "administrator" even once. It never addresses "mod" or "moderator" as one of the players in this process.

    The words "agreement" or "contract" appear zero times in the specification.

    I'm talking with masto devs about what would be good there

    That's a reasonable step, but again, I don't think that's the key problem. None of this matters without Masto Admins and Masto Mods also on board.

    Every failure case in these specs falls on Admins and Mods to resolve, shouldn't they be first consulted?

    @gatesvp

    It doesn't use the word "admin" or "administrator" even once. It never addresses "mod" or "moderator" as one of the players in this process.

    This is why I said "except moderation" and then said "I'm working on it"

    this is the least helpful feedback I've gotten, because you are indeed failing to read the document while also assuming that I haven't thought about the most basic parts of the problem.

  • @jonny

    This FEP is written to minimize the responsibility of the source instance,

    You have this line right there in the spec, and I just don't understand this assumption.

    By minimizing the responsibility of the Source instance, you're dumping all of the work on the Target and 3rd Party instances. But they're not the primary actors here.

    The key parts of this chain are the Actor and the Source. They trust each other.

    • They have an established User Agreement in place
    • The Source has an established history of Actor behaviour
    • The Actor has a high enough trust in the Source that they have published enough that it justifies migration

    When Actor signs up for an account with Target, that new Target User Agreement doesn't assume that Actor is going to bring 200k old posts with them.

    If I'm Target, I don't even want this. My default answer here is "no, you cannot do this without talking to me first". We don't have that relationship yet.

    This frankly sounds like a giant spam vector.

    @gatesvp
    I dont even know where to start with this because its just based on fully mis-understanding the document

  • @jonny

    Inherent in your specification is the assumption that Target's default stance is simply to accept all incoming transfer requests as legitimate.

    This is a very Actor-centric view: "It's my content, I can bring it wherever I want, this should be as seamless as possible". But that's an oversimplification of the Publisher (Source) / Actor relationship that's actually in place.

    And I don't think that's a fair assumption on behalf of Target. In fact, I don't even think it's a safe assumption for the network as a whole, because it's a giant spam vector. None of this is should be automatic, Target needs an active sign-off on content transfers.

    I think this is relevant, because an active sign-off from both Source and Target actually changes parts of these specifications. They don't have to drip transfer, they can coordinate bulk operations, they can negotiate size limits, etc.

    @gatesvp
    Again, see how in my initial response I said "except moderation" and "I'm working on it"

    The entire move process already requires an active sign-off from the source and target actors, and this FEP provides a means of proving that. It also directly addresses the possibility of bulk transfers and does as much as is feasible, and there is already a discussion on how it could be made more efficient.

  • @gatesvp
    Again, see how in my initial response I said "except moderation" and "I'm working on it"

    The entire move process already requires an active sign-off from the source and target actors, and this FEP provides a means of proving that. It also directly addresses the possibility of bulk transfers and does as much as is feasible, and there is already a discussion on how it could be made more efficient.

    @jonny

    The entire move process already requires an active sign-off from the source and target actors,

    But I'm not talking about the Source and Target Actors, I'm talking about the Source and Target Administrators. That's a different human.

    Again, I just read all of these specs for the first time this morning, it's very possible I missed something here. You seem pretty confident that you have addressed Administrator concerns. And I'm happy to retract all of my comments and provide different and more useful feedback if you can even just clip a portion of the text that I missed with respect to the Administrators and help me get up to speed.


Gli ultimi otto messaggi ricevuti dalla Federazione
Post suggeriti
  • 0 Votes
    8 Posts
    50 Views
    👋 @hongminhee@hollo.social maybe @admin@spark.box464.social can provide an account? @mayel@sunbeam.city ​@_elena@mastodon.social
  • 0 Votes
    1 Posts
    8 Views
    Fedify 1.10.0: Observability foundations for the future debug dashboard Fedify is a #TypeScript framework for building #ActivityPub servers that participate in the #fediverse. It reduces the complexity and boilerplate typically required for ActivityPub implementation while providing comprehensive federation capabilities. We're excited to announce #Fedify 1.10.0, a focused release that lays critical groundwork for future debugging and observability features. Released on December 24, 2025, this version introduces infrastructure improvements that will enable the upcoming debug dashboard while maintaining full backward compatibility with existing Fedify applications. This release represents a transitional step toward Fedify 2.0.0, introducing optional capabilities that will become standard in the next major version. The changes focus on enabling richer observability through OpenTelemetry enhancements and adding prefix scanning capabilities to the key–value store interface. Enhanced OpenTelemetry instrumentation Fedify 1.10.0 significantly expands OpenTelemetry instrumentation with span events that capture detailed ActivityPub data. These enhancements enable richer observability and debugging capabilities without relying solely on span attributes, which are limited to primitive values. The new span events provide complete activity payloads and verification status, making it possible to build comprehensive debugging tools that show the full context of federation operations: activitypub.activity.received event on activitypub.inbox span — records the full activity JSON, verification status (activity verified, HTTP signatures verified, Linked Data signatures verified), and actor information activitypub.activity.sent event on activitypub.send_activity span — records the full activity JSON and target inbox URL activitypub.object.fetched event on activitypub.lookup_object span — records the fetched object's type and complete JSON-LD representation Additionally, Fedify now instruments previously uncovered operations: activitypub.fetch_document span for document loader operations, tracking URL fetching, HTTP redirects, and final document URLs activitypub.verify_key_ownership span for cryptographic key ownership verification, recording actor ID, key ID, verification result, and the verification method used These instrumentation improvements emerged from work on issue #234 (Real-time ActivityPub debug dashboard). Rather than introducing a custom observer interface as originally proposed in #323, we leveraged Fedify's existing OpenTelemetry infrastructure to capture rich federation data through span events. This approach provides a standards-based foundation that's composable with existing observability tools like Jaeger, Zipkin, and Grafana Tempo. Distributed trace storage with FedifySpanExporter Building on the enhanced instrumentation, Fedify 1.10.0 introduces FedifySpanExporter, a new OpenTelemetry SpanExporter that persists ActivityPub activity traces to a KvStore. This enables distributed tracing support across multiple nodes in a Fedify deployment, which is essential for building debug dashboards that can show complete request flows across web servers and background workers. The new @fedify/fedify/otel module provides the following types and interfaces: import { MemoryKvStore } from "@fedify/fedify"; import { FedifySpanExporter } from "@fedify/fedify/otel"; import { BasicTracerProvider, SimpleSpanProcessor, } from "@opentelemetry/sdk-trace-base"; const kv = new MemoryKvStore(); const exporter = new FedifySpanExporter(kv, { ttl: Temporal.Duration.from({ hours: 1 }), }); const provider = new BasicTracerProvider(); provider.addSpanProcessor(new SimpleSpanProcessor(exporter)); The stored traces can be queried for display in debugging interfaces: // Get all activities for a specific trace const activities = await exporter.getActivitiesByTraceId(traceId); // Get recent traces with summary information const recentTraces = await exporter.getRecentTraces({ limit: 100 }); The exporter supports two storage strategies depending on the KvStore capabilities. When the list() method is available (preferred), it stores individual records with keys like [prefix, traceId, spanId]. When only cas() is available, it uses compare-and-swap operations to append records to arrays stored per trace. This infrastructure provides the foundation for implementing a comprehensive debug dashboard as a custom SpanExporter, as outlined in the updated implementation plan for issue #234. Optional list() method for KvStore interface Fedify 1.10.0 adds an optional list() method to the KvStore interface for enumerating entries by key prefix. This method enables efficient prefix scanning, which is useful for implementing features like distributed trace storage, cache invalidation by prefix, and listing related entries. interface KvStore { // ... existing methods list?(prefix?: KvKey): AsyncIterable<KvStoreListEntry>; } When the prefix parameter is omitted or empty, list() returns all entries in the store. This is useful for debugging and administrative purposes. All official KvStore implementations have been updated to support this method: MemoryKvStore — filters in-memory keys by prefix SqliteKvStore — uses LIKE query with JSON key pattern PostgresKvStore — uses array slice comparison RedisKvStore — uses SCAN with pattern matching and key deserialization DenoKvStore — delegates to Deno KV's built-in list() API WorkersKvStore — uses Cloudflare Workers KV list() with JSON key prefix pattern While list() is currently optional to give existing custom KvStore implementations time to add support, it will become a required method in Fedify 2.0.0 (tracked in issue #499). This migration path allows implementers to gradually adopt the new capability throughout the 1.x release cycle. The addition of list() support was implemented in pull request #500, which also included the setup of proper testing infrastructure for WorkersKvStore using Vitest with @cloudflare/vitest-pool-workers. NestJS 11 and Express 5 support Thanks to a contribution from Cho Hasang (@crohasang@hackers.pub), the @fedify/nestjs package now supports NestJS 11 environments that use Express 5. The peer dependency range for Express has been widened to ^4.0.0 || ^5.0.0, eliminating peer dependency conflicts in modern NestJS projects while maintaining backward compatibility with Express 4. This change, implemented in pull request #493, keeps the workspace catalog pinned to Express 4 for internal development and test stability while allowing Express 5 in consuming applications. What's next Fedify 1.10.0 serves as a stepping stone toward the upcoming 2.0.0 release. The optional list() method introduced in this version will become required in 2.0.0, simplifying the interface contract and allowing Fedify internals to rely on prefix scanning being universally available. The enhanced #OpenTelemetry instrumentation and FedifySpanExporter provide the foundation for implementing the debug dashboard proposed in issue #234. The next steps include building the web dashboard UI with real-time activity lists, filtering, and JSON inspection capabilities—all as a separate package that leverages the standards-based observability infrastructure introduced in this release. Depending on the development timeline and feature priorities, there may be additional 1.x releases before the 2.0.0 migration. For developers building custom KvStore implementations, now is the time to add list() support to prepare for the eventual 2.0.0 upgrade. The implementation patterns used in the official backends provide clear guidance for various storage strategies. Acknowledgments Special thanks to Cho Hasang (@crohasang@hackers.pub) for the NestJS 11 compatibility improvements, and to all community members who provided feedback and testing for the new observability features. For the complete list of changes, bug fixes, and improvements, please refer to the CHANGES.md file in the repository. #fedidev #release
  • 0 Votes
    2 Posts
    12 Views
    @reiver it's going to be great!
  • 0 Votes
    2 Posts
    17 Views
    reiver@mastodon.social so, are we doing this?