Custom remote backup
from rehydrate5503@lemmy.world to selfhosted@lemmy.world on 16 Jul 22:15
https://lemmy.world/post/33067955
from rehydrate5503@lemmy.world to selfhosted@lemmy.world on 16 Jul 22:15
https://lemmy.world/post/33067955
Hi everyone,
I’m visiting some family in another area of the country soon, and have the opportunity to set up a little remote backup server.
Essentially I would like to set something up that I can ssh into and backup photos/videos/documents from my main server periodically once a month or so. Ideally it would be off until I need to turn it on.
I’m looking for ideas on how to best approach this. What kind of hardware would you use in my shoes? I have a couple of spare raspberry pi’s I was thinking to use with an external drive. I was also considering something like those ugreen nas devices that have been popping up. I would ideally set it up and do a sync before I head there, and then just plug it in. Would wake on lan be advised for this?
threaded - newest
raspberry pi + USB raid enclosure in mirror mode with 2x HDD
Do you think a pi 3b would be enough for this?
I have two spare low performance 4TB drives I was thinking to use for this. It’s not quite enough though. Would be ideal to find a way to be able to have someone there easily add a drive when needed 🤔
The 3b just has USB 2, so even with slow spinning rust, that’s going to be a bottleneck. But it’s probably still plenty fast as a remote storage device for media storage.
Edit: said I didn’t know OP’s use case but in re-reading they did say. Edited accordingly.
Hmm good point on it being usb2. I would only occasionally power it on to sync, and the sync could be 50Mb or 20Gb, but even then I think that should be ok.
.
What are you using for your main backup? It probably has a feature for doing remote backup / duplication. You’re best off using that.
If you don’t, then I think that’s probably your first order of business. There are a bunch of good COTS NAS devices that support remote backup to a similar device or to the cloud. Synology generally seems to be the easiest to use based on reviews, but recently they’ve been getting picky about hard drive support.
If you’d rather DIY then there are some FOSS software options to let you build your own NAS and then back it up to the cloud or to a remote device running the same software. These can get pretty complicated from what I can tell (I’m in the process of doing something similar, been researching). Options include OpenMediaVault, and TrueNAS. TrueNAS seems to be “better” but more complicated and easy to fuck up.
Unraid is also very popular, but it costs money to get a software license. Users swear by it, though.
And on the outside HexOS - a fork (or maybe alternative front end?) of TrueNAS, by some former Unraid devs, with the goal of making TrueNAS as easy to use as Unraid. But it’s both paid and beta, so probably not a good choice yet.
These will all allow remote backup to cloud or to a remote device running the same software. They also typically support some kind of virtualization with an app store, so you can use your NAS to host other servers like a media server or immich or home assistant, etc (although app ecosystem abundance will vary).
Wrt hardware, you’ll have to look up system requirements for the software you want to use. For example, TrueNAS uses ZFS filesystem, which wants a lot of ram if you need it to perform well.
If your r-pi can run the software you want, then you can get a SATA hat for your pi, to run a couple hard drives. You can also get NAS cases for your pi.
I probably wouldn’t recommend leaving a mess of cables and parts at your friend’s house across the country, it’s better for both of you if the system is fairly well contained - enough for them to move it without risk of parts getting disconnected.
Thanks for the detailed reply.
So my main NAS is Unraid, and I also have a couple of proxmox boxes. Though I’m less concerned about the proxmox boxes as the main files are on the NAS, and I have a proxmox backup server vm set up on Unraid with regular backups there.
For most of my important files on unraid, I have an external drive that I periodically sync and store in a safe.
I also have access to a VPS with over 1TB of space which I am still figuring out how to best integrate into my backup strategy.
For what I’m asking here, I just want to have a simple solution that I can tuck away and have remote access to and just use syncthing or something to keep it updated.
Ah, ok then, never mind my answer. I greatly misjudged what you were really looking for
Depends a little of how you set it up. But for the target system at least use some kind of raid/raidz. With ZFS you can do “zfs-send” perhaps? Or something like good old rsync would work to.
Wake on LAN won’t work remotely, so you’d either need to have access to a VPN at their location, or have a 2nd always on device that you can connect to and that could then WoL to your device… or… get a device with an IPMI which you remote into. (All non-VPN forms of remote connection are open to abuse)
I suspect (guess) you’re not going to be able to setup a VPN, so perhaps an always on pi is going to be necessary - so maybe it’ll be that with drives set to spin down when idle?
OpenMediaVault was my preferred choice until everything went docker on it which was getting too complex for a NAS… so I just created my own, which powers on at certain times of the day and off again when CPU / network IO was low enough.
Data transfer with syncthing is great, but I don’t really recommend sync for snapshot backups… (consider your files are all corrupted, it’ll happily sync those corruptions) but I have enough space for a few versions of my files, so in theory I can roll back, but it’s cetainly not a Grandfather, Father, Son strategy.
Ahh gotcha, I misunderstood that then. I could probably set up a VPN there but don’t want to over complicate it. An always on Pi will be fine I think, they are low power. I could also add a smart switch and set up a schedule or something but I don’t think thats worth the hassle considering the low power usage of a pi.
Hmmm that’s a good point about syncthing backing up corrupt files. I was thinking to use it because I already use it extensively and I wouldn’t need to mess with port forwarding or anything of the sort.
I had multiple copies of files previously as a backup “strategy” and it got way out of hand where I have like 1.5m photos lol. What do you recommend as an alternative to syncthing?
I would consider also the case where something goes wrong. Maybe the whole thing crashes, maybe you misconfigure something, maybe there’s a power outage or something else happens and you lose the connectivity. Is there someone on site who can do anything to your hardware as you can’t easily just go and figure it out by yourself?
If the answer is ‘no’ then I would strongy reconsider the whole approach. On a worst case scenario the system goes down before you’re even back home from the trip and then your hardware is just gathering dust.
This scenario is why my offsite backup is primarily a backblaze b2 bucket, while also running a large media backup to an external HDD once a month which I keep in a storage unit. Janky but effective
Realistically I could coordinate with my brother to set up a backup system at my family’s place but it feels like a hassle
I was thinking about backblaze but the cost would be too high I think, and I have so much spare hardware laying around I may as well use it.
This is a good point. The answer in my case is yes, there is someone on site who is tech savvy and can help get it back up and running in the even that something goes down.
I have a remote backup setup at my parents, which consists of a rpi4 and an external hdd (old NAS HDD). The rpi also has a wireguard interface running. If I need to back stuff up (via a restic-rest container) I open the vpn from my side, start the container, mount/decrypt the hdd and trigger the restic update. Afterwards everything gets disconnected again.
The rpi needs to be always on and you need to be able to forward a port to the rpi.
The data should be secure though, as the disk is encrypted and not mounted. (You can even store the decryption key on your side of the network, so you need the vpn to be able to mount the hdd)
I think this is the play. I’ll likely just get an enclosure for the two 4TB drives I have and can always buy an external drive in the future and get them to plug it in.
I don’t have any experience in setting up wire guard so I’ll have to look into that. I was thinking to use syncthing since that skips the need for that, but I think someone in the thread mentioned that may not be ideal in case of file corruption.
Do you just have raspbian on the pi?