from Agosagror@lemmy.dbzer0.com to fediverse@lemmy.world on 13 Apr 18:56
https://lemmy.dbzer0.com/post/42130098
I was playing around with Lemmy statistics the other day, and I decided to take the number of comments per post. Essentially a measure of engagement – the higher the number the more engaging the post is. Or in other words how many people were pissed off enough to comment, or had something they felt like sharing. The average for every single Lemmy instance was 8.208262964 comments per post.
So I modeled that with a Poisson distribution, in stats terms X~Po(8.20826), then found the critical regions assuming that anything that had a less than 5% chance of happening, is important. In other words 5% is the significance level. The critical regions are the region either side of the distribution where the probability of ending up in those regions is less than 5%. These critical regions on the lower tail are, 4 comments and on the upper tail is 13 comments, what this means is that if you get less than 4 comments or more than 13 comments, that’s a meaningful value. So I chose to interpret those results as meaning that if you get 5 or less comments than your post is “a bad post”, or if you get 13 or more than your post is “a good post”. A good post here is litterally just “got a lot of comments than expected of a typical post”, vice versa for “a bad post”.
You will notice that this is quite rudimentary, like what about when the Americans are asleep, most posts do worse then. That’s not accounted for here, because it increases the complexity beyond what I can really handle in a post.
To give you an idea of a more sweeping internet trend, the adage 1% 9% 90%, where 1% do the posting, 9% do the commenting, and 90% are lurkers – assuming each person does an average of 1 thing a day, suggests that c/p should be about 9 for all sites regardless of size.
Now what is more interesting is that comments per post varies by instance, lemmy.world for example has an engagement of 9.5 c/p and lemmy.ml has 4.8 c/p, this means that a “good post” on .ml is a post that gets 9 comments, whilst a “good post” on .world has to get 15 comments. On hexbear.net, you need 20 comments, to be a “good post”. I got the numbers for instance level comments and posts from here
This is a little bit silly, since a “good post”, by this metric, is really just a post that baits lots and lots of engagement, specifically in the form of comments – so if you are reading this you should comment, otherwise you are an awful person. No matter how meaningless the comment.
Anyway I thought that was cool.
EDIT: I’ve cleared up a lot of the wording and tried to make it clearer as to what I am actually doing.
threaded - newest
Don’t be too mad at me
/c/theydidamath
Everybody stop commenting! Wait, did I just…dammit!
Bamboozled!
Fun break down! More comments is more interesting than more posts for me
Well then, here, have a comment.
Doing my part to make this a good post, cause it was.
I support your support.
Haha nice bait, which I took to get some actually interesting statistics, well executed !
Here is your comment, you deserve it. Now your post made it to “average”! You’re welcome.
(Was there any correlation between upvote count and the comment-based metrics? That could also be pretty interesting)
I don’t have any data for upvote count/ comment based metrics. If you have any sites that happen to have that data, send it my way, that’d be amazing!
Cool !
No, you did your math wrong
Also, something about politics.
(Just kidding. This is neat 😎)
Thanks. That was the toxicity I was expecting. Even if it’s not sincere, I appreciate it. I’ve been kinda withdrawing after switching to Lemmy, and I really needed a dose of Reddit hostility.
You didn't factor in the variability of federation vs. a single platform and how not only can it affect how long it takes for everyone to see a post, if they do at all, but also how many duplications there may be floating around. And I don't know if you can predict that reliably, as we're all still trying to figure it out.
Oh, you can absolutely increase how meaningful this is. I only took college level statistics so this is very rudimentary. But yeah, I don’t know how you would predict how long something would take to be seen etc.
We had the chance to upvote this heavily without leaving any comments, but we blew it
post is too [good] unfortunately
they don’t think it be like it is but it do
accidentally a word
that meme deserves to come back
I accidentally the whole thing! I miss that one.
What the hell is this?
A dying gasp collected from the old internet.
Considering the upvotes, you guys seem to have filled in just fine
I upvoted you, because it was funny and made me laugh :D
Why would you make a comment while knowing how this works. Everybody stop commenting! Don’t even do it as a meta-comment joke!
Right! Sorry.
Shhhhhhhhhhhhh!
pawb.social/u/LemUrun ruined it!
this is a certified good post
Where can I get my 9% badge?
Sorry for the delay. Here you go:
<img alt="" src="https://lemmy.world/pictrs/image/cf81468e-7c27-414e-8514-198cfd434c44.jpeg">
In my mind, that shows that “copying reddit” was not the best idea and people should really have copied things like phpBB or SMF for the flagship “community-based” fediverse platform, at least to start out.
On traditional forums, even relatively small communities cause interesting content to appear all the time, by thread bumping and back-and-forth discussion that can go over many pages. However it is obvious that this structure doesn’t scale well to communities with thousands of active users writing thousands of comments in one thread. The reddit structure works better for such communities, but most communities we have here on the threadiverse just aren’t that big yet.
I grew up with traditional forums and discovered other structures for “social media” much later; I still consider traditional forums way superior to any “social media” structure that is nowadays popular.
While I love traditional forum structure, I don't think it would be great to transition later from one structure to another completely different. In my mind, the only thing that needs a bit of tuning is the "Hot" algorithm of the front page
ActivityPub is flexible enough that we don’t have to choose. Someone could implement a piece of compatible software that displays threadiverse communities as “boards” and everyone could join whatever they liked best. NodeBB is already doing something similar to that.
I think meaningful commenting only works in trees, for example the old mailing lists.
With classic linear forums, I quickly loose track of different discussions. Good luck finding replys to a comment on page #3 when the post has 300 comments.
True, that structure does also have its own peculiar problems.
It’s just what I grew up with (from when I was a preteen, only first became active on reddit ~10 years later), so I’m kinda nostalgic for it. :D One aspect of linear forums is that you gradually got to know the people regularly commenting, much more so than on reddit or Lemmy.
I still see you quite often 😄
Yes, you’re also one of the few usernames I keep seeing repeatedly.
There are definitely around 50 usernames I see in a lot of communities, or sometimes just in one. Feels nice to feel like talking to real humans compared to bots.
I was also a very active user of traditional forums but, in my experience, small niche subreddits (when I was on Reddit) were a decent substitute in terms of content, since posts could stay on their front page for several days. Lemmy isn’t big enough to have those yet but I hope it will be. The thing I miss most about forums isn’t the format but rather the community. The forum I posted on the most had only a few dozen regulars and I knew them.
There was the guy with a kind, insightful take on controversial issues and a fetish for women with more than two arms. The active duty marine who reliably posted harsh truths. The feminist I didn’t get along with at all despite agreeing with her about most things. The dedicated father who bought real razor wire for his daughter when she wanted a UN-peacekeeper-base themed birthday party. The very determined conservative who defended his position no matter how outnumbered he was and once bragged that he had given his wife several dozen orgasms in a row…
I suppose I was the young man with strange views about what was or wasn’t fair and a great deal of anger over any perceived unfairness. (I don’t think I was particularly well-liked.) The internet is so much less personal now.
PieFed is trying a bunch of new stuff that even Reddit does not have, enabling the democratization of moderation by putting more power into the hands of individual users (e.g. mods don't have to be as aggressive as removing many posts with keywords when users who want such can set their own preferences via the built-in keyword filtering, which enables All, Some, and None).
Lemmy to me looks more like a straight attempt to copy, although the modlog is a great addition - unfortunately in the absence of notifications of a moderation event, lack of modmail, and presence of an obscured moderator name, Lemmy has somehow become even more authoritian than Reddit.🤷😳
Though with a MUCH more friendly userbase, and most admins, and ofc lack of profit incentive which all by its lonesome helps a ton.
Between (old) reddit, phpbb and smf… the old reddit model wins by a landslide in my opinion. I’ve used all of these over the years and absofuckinglutely HATE phpBB and SMF, with SMF earning slightly less hatred than phpBB.
I’m also old enough to have downloaded
.QWKfiles for offline BBS forum reading/replying and even connecting to the internet over a FidoNet-internet bridge. Lemmy did the exact right thing in copying reddit’s format.“New Comments” helps with that
Thread bumping is still possible on Reddit-like social media too. Just use a sort that responds to activity, like the Active sort on Mbin or Piefed.
Finally, I know why.
This does happen with comments sometimes. I go into a post and someone has already eloquently said what I would have said (often better than I would have). So I upvote it and move along
I wonder how much that statistic would change if you exclude news or politics communities.
That’s a good point. Please take this two thirds of a comment as tribute.
So, like, I do have a background in stats and network analysis, and I’m not sure what you are trying to say here.
Statistically significant what? What hypothesis are you testing? Like, how are you setting this question up? What is your null?
Because I don’t believe your interpretation of that conclusion. It sounds like mostly you calculated the parameters of a poisson and then are interpreting them? Because to be clear, thats not the same as doing hypothesis testing and isn’t interpretable in that manner. Its still fine, and interesting, and especially useful when you are doing network analysis, but on its on, its not interpretable in this manner. It needs context and we need to understand what test you are running, and how you are setting that test up.
I’m asking these questions not to dissuade you, but to give you the opportunity to bring rigor to your work.
Should you like, to further your work, I have set up this notebook you can maybe use parts of to continue your investigations or do different investigations.
Oh yeah ok, so I was going to figure out to put “H0 : L = 8.2”, and “H1 != 8.2, X~Po(8.2), P(c<=X<=c2) => c=?, c2=?” but I left it out because I couldn’t format it in a way that looked half decent in a Lemmy post.
I found the critical regions of the Poisson distribution, that takes the mean to be the average comments/post for the fediverse. I then interpreted those numbers, which I where I assume I’ve made a mistake. As if it was outside of the critical region, that would mean H1, but we know H1 is wrong, since we already have a value for L. It sounds like your interpretation of what I did is bang on. Yeah I get that it isn’t a hypothesis test, but at the level of my stats exams - finding the critical regions was 99% of the work in a hypothesis test.
I only took college level statistics like I said in another reply. I just thought it was cool to see all the instances comments/post ratio. It doesn’t help that my stats teacher was the most boring man alive, and I was always much preferred the pure side of the maths course.
So lets just cover a few things…
Hypothesis testing:
The phrase “if your post got less than 4 comments, that was statistically significant” can be misleading if we don’t clearly define what is being tested. When you perform a hypothesis test, you need to start by stating:
Without a clearly defined H₀ and H₁, the statement about significance becomes ambiguous. The p-value (or “significance” level) tells you how unusual an observation is under the assumption that the null hypothesis is true. It doesn’t automatically imply that an external factor caused that observation. Plugging in numbers doesn’t supplant the interpretability issue.
“Statistical significance”
The interpretation that “there is a 95% probability that something else caused it not to get more comments” is a common misinterpretation of statistical significance. What the 5% significance level really means is that, under the null hypothesis, there is only a 5% chance of observing an outcome as extreme as (or more extreme than) the one you obtained. It is not a direct statement about the probability of an alternative cause. Saying “something else caused” can be confusing. It’s better to say, “if the observed comment count falls in the critical region, the observation would be very unlikely under the null hypothesis.”
Critical regions
Using critical regions based on the Poisson distribution can be useful to flag unusual observations. However, you need to be careful that the interpretation of those regions aligns with the hypothesis test framework. For instance, simply saying that fewer than 4 comments falls in the “critical region” implies that you reject the null when observing such counts, but it doesn’t explain what alternative hypothesis you’re leaning toward—high engagement versus low engagement isn’t inherently “good” or “bad” without further context. There are many, many reasons why a post might end up with a low count. Use the script I sent you previously and look at what happens after 5PM on a Friday in this place. A magnificent post at a wrong time versus a well timed adequate post? What is engagement actually telling us?
Model Parameters and Hypothesis Testing
It appears that you may have been focusing more on calculating the Poisson probabilities (i.e., the parameters of the Poisson distribution) rather than setting up and executing a complete hypothesis test. While the calculations help you understand the distribution, hypothesis testing requires you to formally test whether the data observed is consistent with the null hypothesis. Calculating “less than 4 comments” as a cutoff is a good start, but you might add a step that actually calculates the p-value for an observed comment count. This would give you a clearer measure of how “unusual” your observation is under your model.
Look, I survived statistics class. I will stride to defend some of my post.
Namely that much of the aim of it was to show that an metric like comment count doesn’t imply that it was a good or bad post - hence the bizarre engagement bait at the end. And also why all of the “good posts” were in quotes.
I’m under the impression that whilst you can do a Hypothesis test by calculating the probability of the test statistic occurring, you can also do it by showing that the result is in the critical regions. Which can be useful if you want to know if a result is meaningful based on what the number is, rather than having to calculate probabilities. For a post of this nature, it makes no sense to find a p value for a specific post, since I want numbers of comments that anyone for any post can compare against. Calculating a p-value for an observed comment count makes no sense to me here, since it’s meaningless to basically everyone on this platform.
Truthfully I wasn’t doing a hypothesis test - and I don’t say I am in the post - although your original reply confused me - so I thought I was, I was finding critical regions and interpreting them, however I’m also under the impression that you can do 2 tailed tests, although I did make a mistake by not splitting the significance level in half for each tail. :(. I should have been clearer that I wasn’t doing a hypothesis test, rather calculating critical regions.
It doesn’t seem like you are saying I’m wrong, rather that my model sucks - which is true. And that my workings are weird - it’s a Lemmy post not a science paper. That said, I didn’t quite expect this post to do so well, so I’ve edited the middle section to be clearer as to what I was trying to do.
Well I appreciate the effort regardless. If you want any support in getting towards a more “proper” network analysis, I’ve dm’d you a link you can use to get started. If nothing else it might allow you to expand your scope or take your investigations into different directions. The script gets more into sentiment analysis for individual users, but since Lemmy lacks a basic API, the components could be retooled for anything.
Also, you might consider that all a scientific paper is, at the end of the day, is a series of things like what you’ve started here, with perhaps a little more narrative glue, and the repetitive critique of a scientific inquiry. All scientific investigations start with exactly the kind of work you are presenting here. Then you PI comes in and says “No you’ve done this wrong and that wrong and cant say this or that. But this bit or that bit is interesting”, and you revise and repeat.
The other chance that you got no comments on your post for is that you are banned from the remote instance/community, or federation is broken (still happens intermittently).
Lemmy will still allow you to post from your home instance since you are not banned there, but your content will simply get black-holed by the remote instance if you’re banned there. Sometimes you have to check the remote instance directly to see if your post was federated or not.
Is there a way to check without manually going through each instance individually?
You can just check the modlog of your local instance and search for your own username. Most of the time the ban action will federate (but again, sometimes not, never really sure why). If nothing shows up locally check the modlog of the remote instance you’re trying to post to.
Omg, I’m so glad there isn’t any entity trying to boost this KPI like it’s the only thing that matters in the world.
Thank you for your engagement. As a reward, please enjoy an additional upvote as well as this comment.
Ok.
Can’t think of anything I care about less.
Elon Musk’s Path of Exile II advice?
I don’t understand what this is supposed to mean. The commenter’s account, or the community they posted to is on .world/.ml? Because those aren’t necessarily the same.
Presumably where you posted it, given that local feeds show posts based, not on if someone is on the instance, but rather which instance the post is made on. The model I used is litterally the most basic thing in the world, so I just cobbled something together that was somewhat meaningful. I only took college stats, so complex models are out of my range.
I’ve happily found that there is much more interaction here than on Mastodon :)
Mastodon mainly only looks like there is no interaction happening because of their federating logic. Which is being worked on to be fixed sometimes this year
Does that mean we would see more posts / accounts being recommended?
It means that if you see a post, you will finally see all replies and interactions to that post. Currently this is not working.
Got it, thanks for clarifying.
Oh holy shit is that when I tried mastodon it felt like a ghost town with people only posting and no one engaging in discussions?
I mean, in microblogging a lot of posts will have no discussions simply because thats the nature of microblogging. But if you saw no interaction and discussion under posts of people who have lots of followers, then certainly yes.
I was pretty much only seeing people from my instance in comments. Or people that they followed. Joind an explicitly leftist instance because it had more charecterst to play with and it felt empty.
When I post here I get replies. On mastodon I don’t.
I see that you have a very good quota of comments that you get on every lemmy post you make. I dont think that’s true for every poster, especially when posting to niche lemmy communities.
But yes: of course the lemmy format invites comments way more than the microblogging format of mastodon
It’s a different model.
Mastodon, like Twitter, is a person-centered setup. You can use hashtags, but most people don’t. You follow people not communities. As a result it’s basically microblogs, where most people are just posting into the void. Celebrities are followed more, so they get more replies, so there are more conversations. But, fundamentally it’s not really inviting interactions.
Lemmy, like Reddit, is a topic-centered setup. It has a bunch of communities and people post something because they think it might be interesting for people who are also interested in that community. Every post is basically an invitation to have a discussion about something.
I think the friction to posting something on Lemmy is slightly higher, but when you do, it’s more likely to generate comments.
I sometimes comment, just for the sake of comment and to support this community. For example now.
Your post is good!
I feel like we shouldn’t do that or stray off on a tangent, like commenting about comments.
Yes, or even continuing such discussions for no apparent reason, but to feel a bit of the normal human interaction… Or to have fun playing some meta together :3
Oof. That would be problematic for sure. I’m glad we’re cooperating to oppose that kind of nonsense.
At this depth, I consider you a friend, however you are :) Thank you for the good time!
Likewise. We are now officially Lemmy friends.
.
Thank a lot for sharing this with us, i love those data :)
Your correct use of the plural makes me oddly uncomfortable.
Thank you for your insights.
Good post bro. 😉
Even though I appreciate this post, I don’t think I will comment.
Yeah, it was very informative, but I agree that we should test some of these hypotheses by avoiding a comment chain. Therefore, I, too, will forego commenting in the interest of science.
does this have anything to do with the girl in the picture? or is this just “GIIIIIRLLL!!!” moment?
That’s the bait to get you here.
No. What makes you say that?
<img alt="" src="https://lemmy.world/pictrs/image/d7a7a040-ad8f-4286-9874-64fea023d3ab.jpeg">
goes back to lurking in the shadows
You are doing a terrible job of lurking. Thank you.
By my count, this comment will take your post from one with 12 comments to one with 13 comments, therefore I’m conferring on you the title of “good post”. Congratulations!!
However, I’m assuming that you’re including your own comments in the comment tally. If you’re not, then your 2 comments so far to this post don’t count, and you’ll only be at 11, and therefore “not good”.
If you are counting your own comments on your own post, can you juice the numbers by adding lots of comments? In other words, can you make a post good by interacting with the people who are interacting with the post? Like some kind of um… conversation? Sounds like cheating to me.
Hey, just tossing in a comment here, I think this post is a good post!
I guess those without ego stroking don’t care.
Post in a obscure sub
If average comments/post is lowest on .ml, medium on .world and highest on hexbear, it might correlate those instances with post meaningfulness, or with the innate tendency of their users to comment. Or with both, or some other thing entirely. All I can really say about it is, “Huh, interesting.” Not interesting because it leads to any particular conclusion, but interesting that there’s a pattern.
You know, I’m really just waiting for the day voyager supports gif insertion.
This should be a picture of Nicole, the Fediverse chick.
Another likely cause: you’re posting to a non-local community and you got hit by federation issues, while your instance thinks the post got created, the target instance doesn’t know about it.
Happened to me a few times.
I’m a bit confused but I think I liked this post
Okay. Look. We both said a lot of things that you’re going to regret. But I think we can put our differences behind us. For science. You monster.
It makes sense that it would be highly dependent on comments because for one, Lemmy’s default filter is activity based so the more activity a new post has, the higher it will rank, until displaced by a newer post. The second part is that if there aren’t any comments there people might be less likely to leave comments and the post is more likely to do poorly as it’ll get bumped down by posts with higher activity. Obviously not everyone uses the activity sort feature, some sort by new, top, or scaled, but since activity is the default most will use that. Especially since it shows posts with the most discussion and activity, the ones most likely to find other people interacting on.
I think the community matters a lot more than the instance. Hexbear has a bunch of coping bubble communities but they keep posting the same low-quality comments, so that’s probably why the threshold of 20 comments is so high. Another example, I make posts to my own blog community !dginovker_blog@lemmy.ml, but there’s no subscribers so there’s never gonna be any comments.
Basically I’m saying you should do this same analysis across a sample of random communities ^^
You need a factor for niche communities. A post with 4 comments in a backpacking community with 20 subscribers is way “gooder” than 40 comments in a 5k subscriber news community.
I.E. add a community size factor.
Any details you could share about how you obtained and processed the data? It seems like there’s a lot of interesting things that could be done with this but I’m not sure where the best place to start would be
Write a few comments, go to sleep and see for yourself what happens when you wake up. This is how I do it.
I think one needs to include parameters like how soon after the topic was created the comment was made and how deep is it in the comment tree. If you for instance consistently comment on 1 month old topics or reply on comments ten levels deep you will get very few interactions.
Well exactly, I’ve said this elsewhere in this thread, this was mostly something that I thought was cool. That said I might try and figure out how to include that data, if I can find it.
Average Fediverse Experience:
Post comment
Waits 24 hours
zero replies
zero votes
not even a downvote
check post viewed from other instances
can’t find the comment
realizes that the comment never federated
now too much time has passed since the original time of the post, and the joke you commented is no longer funny anymore
😭
Or other people created the same joke without ever seeing your post
A post by fediversechick
I follow instructions, I think. Good post
Interesting numbers, it would be great to see how the statistics look for different “categories” of communities. Interaction based communities (c/ask X) and political communities will naturally garner more comments than information communities. E.g. while you may enjoy the content of blogs posted on !godot@programming.dev or !programming@programming.dev, you’re probably less likely to comment than on !asklemmy@lemmy.world or !casualconversation@lemmy.world
I actually plotted the top 50 or so instances, with user size against comments/post. One of the many outlier instances was lemmynsfw.com which obviously lacks all that much engagement, with a score of around 1 c/p. Which makes quite a bit of sense when you think about it.
!casualconversation@lemm.ee
I comment very seldom and only if i think that I can contribute. I see no need to write anything if I got nothing of significance to add.
Maybe I should. Add comments that is uplifting and kind more often.
I comment a shit ton and often with absolute banalities. Especially on posts with 0 comments.
My reasoning is twofold: first of all I want to encourage posters by engaging with their content so they don’t stop posting. Second I want to invite others to comment and it’s much more inviting to do so if a post has at least one comment. People tend to think it’s dead otherwise and not bother.
I think at the current level of MAUs there is no comment too small, and every little bit helps just by virtue of breaking the silence.
I feel guilty now. Yes, everything you just said is true.
I shall become a better… Lemming(?) and comment a few times every day.
Well said
My meagre contributions pale in comparison to your efforts, but I do what I can.
I try to be positive, but my way of life are very different from other people’s; and i end up doing more harm than good, if i’m forcing myself to be friendly and nice.
This comment will be sad if you don’t engage with it.
Ohhh poor thing here have an upvote and a comment.
The comment is very happy to be a good comment with 8 updoots and two replies.
10/9.5 🥳
2 upvotes and 1 reply but I wish you the best.
This comment is part of a tree-datastructure that represents the branches of discussion.
Add a TLDR or this post won’t get a lot of traction either
Confirmed. I see “Poisson distribution” I start skimming lol
I disagree that commenting for the sake of commenting is a good idea. Quality over quantity, a single meaningful discussion is superior to a sea of low effort garbage. I also want the fediverse to take off, but not at the cost of adopting modern Reddit culture.
Baiting anything is bad.
Well exactly, that was kind of the point of this post. Hence “good post” being in air quotes. It being a silly idea as well.
Completely agree with you on that last point.
Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure.”
Not entirely sure how this applies to the discussion, it just came to mind lol
Ah nice, I encountered a Poisson-distribution in the wild today. I shall recount this encounter to my children.