Making the case for richer HTML in ActivityPub
from julian@community.nodebb.org to swicg-threadiverse-wg@community.nodebb.org on 27 Jun 2024 04:30
https://community.nodebb.org/post/100142

Tonight I set aside some time to listen to @johnonolan@mastodon.xyz on @mike@flipboard.social's DotSocial podcast.

A lot a lot a lot of what John says mirrors the very same potential that many ActivityPub devs see as well. There are far too many points in that podcast that made me nod my head in agreement (and wish I was a third guest too!), but there was one that was incredibly timely:

Mike: ... you've been thinking about actually embedding the whole article in the ActivityPub post, which is a mind-blowing thing... it's not a link to something else... the whole article is in the post.
John: Yes, this is something that makes perfect sense but is somehow completely new, which is weird...
Mike: You can have formatted text... images? video?
John: ActivityPub is fairly agnostic, you could in theory shove almost anything into it. The question is what is the client on the other side prepared to receive? Do they have some way to display it?
John: If we get platforms in the ActivityPub network to start innovating with content types, it might cause those things to be adopted and it might drive the standard and what it is possible to display

Emphasis mine.

John, Mike, this is almost word-for-word exactly what the Forum and Threaded Discussions working group has been working towards! The main problem is we need buy-in from implementers to push this forward.

We can do this, we can send richer HTML across the protocol in such a way that all those things you two mentioned — in-line images, embedded videos, tables, etc. — can all show up as intended by the sender.

We've got commitment from (but not limited to) representatives from NodeBB, Discourse, and WordPress, and having Ghost and Flipboard sign on would help push this forward just that much more.

Let's do this, let me get you caught up with the state of the protocol re: the Article object type. Let's chat (but publicly, since I can't receive DMs here on NodeBB).

#activitypub #dotsocial #forumwg #swicg-threadiverse-wg

threaded - newest

mikedev@fediversity.site on 27 Jun 2024 07:55 next collapse

but is somehow completely new, which is weird...


Laughs. Not weird at all because it isn't new. We've always had rich multi-media HTML of unlimited length in the fediverse. Years before Mastodon was created. And we still have it today.

Somebody is literally stuck in a filter bubble.
silverpill@mitra.social on 27 Jun 2024 10:48 next collapse

@julian @mike @johnonolan @mike As other commenters have noted, embedding whole articles is not really new, they are just not rendered properly by Mastodon. However, I don't think anyone have tried to embed an interactive application:

https://webxdc.org/docs/get_started.html

mikedev@fediversity.site on 27 Jun 2024 11:28 collapse

Evan had a Farmville clone for ActivityPub's predecessor pump.io. We've dabbled in some other server-side apps, like putting chess  games into your  stream -  but the killer is letting strangers execute javascript in your stream. We ran some experimental stuff in sandboxed iframes, but finally decided it was a bad look for the free web and said yeah,nah.

mike@flipboard.social on 27 Jun 2024 13:46 next collapse

@mikedev @silverpill yeah JavaScript in your stream would be problematic.

It would be interesting to figure out a safe and standard way to provide limited interactivity on the level of things like polls. Farcaster seems to be experimenting with ideas like this though i have not looked at it closely.

julian@community.nodebb.org on 27 Jun 2024 14:03 collapse

@mike@flipboard.social I think even @mikedev@fediversity.site could agree that Mastodon has done the bare minimum of support. They used to only support straight text (and even now, composing in Mastodon is plaintext only), but have since allowed a heavily locked-down subset of HTML.

Allowing arbitrary JavaScript execution is a bad idea, but relaxing html sanitisation to the minimum would allow for much more expressive content. Inline images and tabular data are two tags I'd like to see allowed, but herein lies the problem... What subset of HTML do you feel comfortable with? I wager that different implementers would answer differently, and Mastodon eschewing certain tags in order to prioritize the textual microblogging style is their choice, too!

The fact of the matter is Mastodon needn't support all the tags under the sun, but they do need to acknowledge that received content is not faithfully represented in all cases, and so a prompt to view the content on the original site would be a good intermediate step.

There already exists an ActivityStreams object type for this kind of content: as:Article. It gives other implementers a chance to explicitly say "hey, this might be better viewed natively if you don't elect to show everything we send you". I'd love to make this a reality.

evan@cosocial.ca on 27 Jun 2024 14:01 collapse

@mikedev @silverpill it might be time to dust off the OpenSocial spec and consider how this could work in stream. I don't think arbitrary JS is a good idea, but it might be interesting to have user-installable widgets in a sandbox.

ricmac@mastodon.social on 30 Jun 2024 09:18 collapse

@evan @mikedev @silverpill Federated start pages? Getting Netvibes vibes here :)

angus@socialhub.activitypub.rocks on 27 Jun 2024 08:03 next collapse

@jupiter_rowland I'm curious if you actually posted twice or if your post appearing twice is a bug in one of our implementations.

julian:

We can do this, we can send richer HTML across the protocol in such a way that all those things you two mentioned — in-line images, embedded videos, tables, etc. — can all show up as intended by the sender.

I agree with you Julian. I think we'll need to tackle this piece by piece, but I'm on-board with the vision. I know this is what the users of Discourse want (they've been telling me!).

I wonder whether the most extensible way to do this is by sending markdown in addition to the HTML we already send. Markdown

  • has canonical (word of the day) markup for media objects (e.g. we'll all send images as ![image](https://imageurl.com))
  • is more secure than HTML
  • is (mostly?) already supported by the platforms who'd want to federate with each other for this richer content.

If we put an additional markdown representation which included the richer content we'd also avoid the HTML being rejected or subject to parsing we don't control on the grounds it contained unsupported tags or overly-complex structures.

julian@community.nodebb.org on 27 Jun 2024 11:06 collapse

@angus@socialhub.activitypub.rocks said in Making the case for richer HTML in ActivityPub:

I wonder whether the most extensible way to do this is by sending markdown in addition to the HTML we already send.

NodeBB already sends markdown through, although I'm not sure whether sending different markdown from the HTML in content is the right approach (something something two wrong don't make a right)

@johnonolan@mastodon.xyz is right about one thing, which is that if a critical mass expanded support, then other implementers would likely follow suit. It's always been this way, but usually it's Mastodon that has gained, and everybody else who's has to adapt.

I see two paths forward:

  1. Compromise (@mikedev@fediversity.site suggested this years ago) — as:Note contains a limited subset of HTML (and images are attached), as:Article contains the richer set.
    • Downsides: it's a technically contrived distinction as opposed to one based on the kind of content relative to others (e.g. content is an article because it contains an image, vs content is an article because it is considered a standalone work)
  2. Ignore — send as:Article if we want to, and send the richer HTML set because we can.
    • Downside: End users don't care, they just see that their post content doesn't make it through to Mastodon and complain

As much as I dislike the "best viewed in" phrase, Mastodon would ideally need to show an info label instructing users to go to the original site, or render richer HTML outside of their feed (like in a modal).

laurenshof@indieweb.social on 27 Jun 2024 12:28 next collapse

@julian @johnonolan @mike

Also summarised as "federation happens in the client"

lutindiscret@mastodon.libre-entreprise.com on 27 Jun 2024 12:37 collapse

@laurenshof @julian @johnonolan @mike I really hope that by "rich content", you mean "structured content" (headings, images, lists) not "formatted content" (font size, font colors, alignments). While I would accept the firsts, we should reject the seconds in both the standards and the apps: i don't want to live with people poor choice about fonts and colors (theming and accessibility will be issues).

mike@flipboard.social on 27 Jun 2024 13:40 next collapse

@julian @johnonolan I’d love to learn more about this!

KevinMarks@xoxo.zone on 30 Jun 2024 11:32 next collapse

@julian @johnonolan @mike the origins of ActivityPub is Atom, which had the name/summary/content split, and we preserved that through the various Activity Streams iterations, from Atom to JSON to the current JSON plus LD.
Mastodon undermines this by ignoring post names and treating summary as a Content Warning.

ShadSterling@mastodon.social on 04 Jul 2024 18:20 next collapse

@julian @johnonolan @mike it still baffles me that the content encoding standards aren’t the same as email. The spec is already there, why not use it?

osma@mas.to on 04 Jul 2024 19:13 next collapse

@julian @johnonolan @mike
Would be remiss to not mention that Hometown, which is a "light" Mastodon fork supports the Article post type and has no issues rendering long form.
https://github.com/hometown-fork/hometown?tab=readme-ov-file#reading-more-content-types

diyrebel@lemmy.dbzer0.com on 07 Sep 2024 07:55 collapse

First, I must say that it’s disturbing to find a “threadiverse working group” hosted inside a centralised exclusive walled garden (community.nodebb.org is a Cloudflare site that restricts access and undermines transparency). A threadiverse working group should be aligned with the core fediverse principle of decentralisation.

As for the topic at hand, we must separate the good ideas from the bad.

The good:

  • Embedding the whole article in the post – great idea for many reasons:
    • the web has become mostly exclusive (Cloudflare, paywalls, tor hostility, loss of netneutrality), thus most links being shared are to places that restrict some demographics of people.
    • enshitification of the web means everyone visiting the link will have to fight with dark patterns, tracking, cookie walls, popups, and countless other shitty downgrades on everyone’s attempt to consume content. By copying the content to an accessible enshitification-free post, everyone commenting can enjoy equal access to the content

The bad:

  • Enrichening the HTML risks bringing enshitification into the threadiverse. Of course there would be tooling that makes it easy to mirror a webpage into the post in an automated way. The garbage on the webpage that downgrades everyone’s UX will be replicated. Users are lazy as fuck. Instead of cleaning up the garbage they will just let it ride and burden everyone else.
  • Try looking at websites in lynx. The web has become a shitshow of chasing the shiny and breaking our basic need for reading text. It’s inherently exclusive because those on capped internet connections cannot afford the bandwidth for eye candy. A text-based client should be able to function well and interact with all the content. It should be accessible from an a11y standpoint.
eeeee@community.nodebb.org on 07 Sep 2024 15:58 collapse

First, it’s disturbing to find a “threadiverse working group” hosted inside a centralised exclusive walled garden (community.nodebb.org is a Cloudflare site that restricts access and undermines transparency). A threadiverse working group should be aligned with the core fediverse principle of decentralisation. It signals unfitness to lead the threadiverse. I will not be subscribing to this exclusive community or following it.

Hi
Just so I can follow, isnt Cloudfare the domain register? Why does it reflect badly in your opinion on the website.
Im not necessarily disagreeing with you, just wondering why having that company involved in the domain, tars whatever is hosted via them?

julian@community.nodebb.org on 07 Sep 2024 17:18 collapse

@eeeee don't engage, some people will always be unhappy unless they get everything for free.