Do we need a crawler to load content from other lemmy instances? How are we supposed to federate without one?
from uhdeuidheuidhed@thelemmy.club to fediverse@lemmy.world on 21 Aug 20:59
https://thelemmy.club/post/33306278

I’m trying to start my own lemmy instance, and reading through the documentation at join-lemmy.org/…/federation_getting_started.html it looks like we need to manually search for just about everything that can be found on other lemmy instances.

It works. I’m able to search for communities/comments/posts and then my instance will load only that community/comment chain/post and only after I’ve searched it. No updates.

Is this what we’re supposed to be doing? Seems pretty tedious to have someone that’s supposed to go to other instances and then search for all of their content on my instance in order to federate with it.

I have a feeling there is an easier way, but I’m legitimately not seeing it.

Right now, it looks like the only way users on my instance will get to see content from other instances is if I manually search for just about everything they’ll get to see.

I can write a crawler to automate all of this, but I’d rather not unless it’s completely necessary.

#fediverse

threaded - newest

tofu@lemmy.nocturnal.garden on 21 Aug 21:23 next collapse

That’s basically how it’s done. Searching for them isn’t enough I think, in order to get updates, a user from your instance needs to subscribe, so a crawler might create a list of communities, but it won’t see news.

Yes, starting a new instance is tedious work.

frongt@lemmy.zip on 21 Aug 21:26 next collapse

There’s a bot you can use to jump-start federation. No idea what it’s called, but I’m sure you can find it with a few seconds of searching.

FrostyTrichs@crazypeople.online on 21 Aug 21:36 next collapse

lemmy-federate.com

Coelacanth@feddit.nu on 21 Aug 22:00 next collapse

This is the answer, I hope it’s common knowledge by now.

uhdeuidheuidhed@thelemmy.club on 21 Aug 22:00 collapse

Thanks, this looks interesting.

I tried adding my instance under “Manage Instances” but got this error: Request timed out: GET https://{INTERNAL_IP}/nodeinfo/2.1

I’m not sure what this means. Could anyone help provide an explanation?

FrostyTrichs@crazypeople.online on 21 Aug 22:10 collapse

Sorry but I’m not sure. Every time I’ve used that tool the setup has gone flawlessly. The first time you use it you put in your instance URL and it should output something for you to add to your DNS records. If you just made the DNS changes it can take a few minutes to several hours for those changes to be pushed out.

uhdeuidheuidhed@thelemmy.club on 22 Aug 00:15 collapse

Thank you. I think I figured it out.

In my lemmy.hjson file, I accidentally put the internal IP address of my server instead of the domain name. This was a carry-over from testing on my main machine by using an IP address and port forwarding because I didn’t have a domain name to use.

I changed the hostname setting in lemmy.hjson to my domain name, and now it seems to be working. Sort of.

Now I have a different error, “Your instance is not guaranteed in Fediseer. Please check fediseer.com”, but this one seems less of a technical issue. I will have to learn what this is about and hopefully I can get it to work :)

FrostyTrichs@crazypeople.online on 22 Aug 00:48 collapse

You’ll have to claim your instance on fediseer and have another instance endorse yours. It’s a system of trust to help keep bad actors from having too much influence over the platform and causing others grief.

uhdeuidheuidhed@thelemmy.club on 22 Aug 01:14 collapse

Thanks yet again.

I tried claiming my instance, but now I’m getting the error: API Key PM failed

Slowly and steadily, this problem will be solved!

FrostyTrichs@crazypeople.online on 22 Aug 01:36 collapse

IIRC the way it works is by PMing you the API key to get started and you’re failing at that point. Are you sure your instance is federating? I don’t know anything about how fediseer works on the backend but if it’s not able to send you a PM that suggests you might not be federating at all or maybe you’re running a whitelist or something? I’d start investigating if your instance is reachable from other instances and if you’re able to send yourself a PM from another instance.

uhdeuidheuidhed@thelemmy.club on 22 Aug 02:00 collapse

You’re probably right.

I’m trying to look up a user on my instance from another insance with theirinstance.com/u/myuser@myinstance.com and it’s not working.

I’m not sure what could be causing this, it could be something simple like the previous issue I had earlier with the domain name. I’m going to go through my settings and try to figure this one out, but I’m still kind of new to all of this so it may take some time.

FrostyTrichs@crazypeople.online on 22 Aug 02:13 collapse

Good luck mate. It’s a little intimidating at first but it usually boils down to something simple. A slight misconfiguration or minor syntax error can cause pretty significant issues. You might have a look at your logs or try running in debug mode to see if anything jumps out at you.

EDIT: Do you have federation enabled in the admin panel? Also probably worth making sure you DON’T have private instance checked.

<img alt="federation" src="https://crazypeople.online/pictrs/image/66964952-f80d-4f3d-89ff-cee08d4343e2.png">

vk6flab@lemmy.radio on 21 Aug 21:53 next collapse

Make sure you’re aware of any legal liability implications for content posted by your users.

Steve@communick.news on 21 Aug 21:54 next collapse

Right now, it looks like the only way users on my instance will get to see content from other instances is if I manually search for just about everything they’ll get to see.

They can do that themselves.

jollyroberts@jolly-piefed.jomandoa.net on 22 Aug 01:39 next collapse

Piefed has a couple of different ways to populate a new server. Pulling lemmyverse data and subscribing to comms that way, or scanning a remote Lemmy/mbin/pieces server for comms. You can look at that code if you want inspiration.

squaresinger@lemmy.world on 22 Aug 07:05 collapse

That’s how federation is supposed to work. If one user searches content, you get it once. If a user of your instance subscribes to a community, you get updates for that community.

The point is to reduce load on your instance by not federating content that nobody on your instance needs.

It’s based on a flawed understanding of how communities work, specifically that reddit-style forums only work well when you have huge communities that everyone sibscribes to. So, most instances end up with most content anyway.

Also, keep in mind, all conent that has been federated to your instance is on your srver and thus legally your responsibility. If there’s illegal content on there, you are liable if you don’t delete it.