from cm0002@lemmy.world to fediverse@lemmy.world on 09 Sep 16:14
https://lemmy.world/post/35676484
This past June, I put together a write-up about two major approaches to backfilling conversations. The ability to properly backfill conversations means we will be able to make major inroads toward solving the feeling that the fediverse is quiet.
I, alongside several other members of the SWICG Forums and Threaded Discussions Task Force (ForumWG) have been working toward building implementor support for Conversational Contexts — the ability to explicitly classify a set of objects as belonging to a conversation, whether that be a topic, reply tree, or similar.
I am happy to report that we have made some wonderful inroads this past few months!
- jesseplusplus@mastodon.social has been working closely with the Mastodon team to allow software to backfill from Mastodon — this pull request has now been approved!
- I have been working with both nutomic@lemmy.ml and rimu@piefed.social to allow software to backfill from Lemmy and Piefed, respectively. (Lemmy PR, Piefed issue)
This marks a major milestone in the adoption of conversational contexts. With Mastodon on board backfill will be possible with the majority of the microblogiverse. With Lemmy and Piefed on board, backfill will be possible with the majority of the threadiverse.
Remember that pfefferle@mastodon.social was an early adopter of conversational contexts, and we have been able to backfill from WordPress blogs for quite awhile now (so that’s the blogiverse too)
I for one, am eagerly awaiting the next version of all of these softwares!!
threaded - newest
Whoaa that's super exciting!!
Can anyone ELI15 what backfilling conversations means?
Sometimes you see a post on your home instance and it looks like it has no replies but then you go look at it on the remote view and there are lots of replies. Backfilling is like making sure if you see a post it has all the comments
I don’t use mastodon, but is that an issue for the Threadiverse? I thought all replies are collected by the host instance of the community.
It may still be missing stuff from before the first person from your home instance joined the community which can make younger communities feel empty.
Ah ok. That’s the problem that the major instances solved by having bots auto join all new communities in each other, right?
Yup, this removes that need.
Edited to add: and more BTW, including addressing brief network outages messing with sync and other issues, depending on the server type/activitypub implementation.
everything is weaved together! What happenes when a mastodon account responds to a piefed account and then that gets shared by a Friendica account etc.
I have no idea :)
My point is that there is never a hard line between any other parts of the fediverse so if there is a new feature or way one part does something it will always have an effect on the other parts.
Wow that is a badly written sentence! I think you get my point tho
You're not wrong about that. Threadiverse software (Lemmy, Piefed, and NodeBB too) use something called 1b12 to synchronize content between instances.
If you follow a community, you set up the synchronization from your server to that community.
This is similar, but about backfill, which is what happens if you follow a community mid-way and don't have any of that stuff from before.
Also you can't 1b12 synchronize with Mastodon, so that unfortunately is a dead-end, but you would hopefully be able to backfill from Mastodon.
AFAIK, It’s to solve or mitigate the “Replies from other servers may be missing” issue.
Essentially imagine you are signed in on server A, responding to or looking at a comment from server B. But people on servers C, D and E have faved, boosted or replied to that same comment. Unless you or someone on your server had followed people on the other servers, you can’t see those comments or their contributions to the boost count, unless you go to view the comment on server B’s site.
Backfilling means server A fetching those other actions from other servers somehow, so that they will show up when you view it from your own server reliably. Examples of that somehow could be, obtaining all the info from server B (localized single source of truth), it could be collected individually from other servers, from a centralized server, or other means.
So cool
For some reason I assumed this was already how the fediverse worked, but I haven’t been here very long and it does explain some things, including the “empty” vibe in some lesser-populated places.
This is super exciting for the fediverse and, naturally, I have questions. While this change will mostly bring positives and a better experience for users, there could also be more opportunities for shenanigans.
What considerations are being given to data integrity/mutability and trust? Will all servers that touch a post have a distributed record of all comments and give network confirmation (a la blockchain)? Or does one server (e.g. the originator of each post, or the server with the most resources) act as a single authority of that post? Something else?
Could one server be instructed to “go rogue” and submit bad content to the network, or go on a deletion/overwriting spree that ends up becoming permanent?
What about resources? What impact will backfilling have on your average dude hosting a small instance?
This is just where my mind goes, you see. I’m sure all this and more have been discussed and figured out already. If a public discussion is available to look at, I would love a link!
There have been monthly meetings about this issue and surrounding ones that are open to the public if you really want to know more. There are years worth of discussions about it too but they are very spread around
Those are all very good questions, and exactly the sort of things that would be discussed at the ForumWG.
Backfill is just one of the things (the main thing, currently) we touch on, but one of the more important ones, because the potential to ensure you have the entire conversation is important from a data completeness standpoint.
The thing to remember is that there's no one "owner" of a conversation. Right now it's a pretty loose association... individual posts and notes can declare that they are part of a context, even if that isn't the case. This beats the current system where there is no association at all.
The difference here is that as a consumer of backfill, I can actually go to the context and verify this. We can extend this later on with context ownership, and defer responsibilities like moderation, interaction policies, forking/merging, etc.
The long view of this is we intend to increasingly solidify the association between context and object over time.
Julian is an awesome guy! I just spoke with him on the latest episode of Fireside Fedi if anyone is interested: https://video.firesidefedi.live/w/q3z3C6JrpitEUxwFmwiTUo
Thanks for the link back, I appreciate it!
One neat side effect of this is that if the microblogiverse backfills from the threadiverse, and vice versa, then you have in effect built a bridge connecting the two disparate 'verses.
We're not there yet — this step only exposes conversational contexts so that consumers (like NodeBB) can backfill from them. Neither Lemmy nor Piefed have signalled intent to consume contexts yet, but as it is a relatively new extension, there is no telling what may happen in a week, a month, a year... :slightly_smiling_face:
Also @rimu@piefed.social is pretty darn responsive and if you as (very) nicely that would go a long way.
Thanks for the context context!