How do you backup?
from Zenlix@lemm.ee to linux@lemmy.ml on 19 Mar 10:01
https://lemm.ee/post/58869487

I recently implemented a backup workflow for me. I heavily use restic for desktop backup and for a full system backup of my local server. It works amazingly good. I always have a versioned backup without a lot of redundant data. It is fast, encrypted and compressed.

But I wondered, how do you guys do your backups? What software do you use? How often do you do them and what workflow do you use for it?

#linux

threaded - newest

Strit@lemmy.linuxuserspace.show on 19 Mar 10:03 next collapse

My systems are all on btrfs, so I make use of subvolumes and use brkbk to backup snapshots to other locations.

a14o@feddit.org on 19 Mar 12:30 collapse

Same! This works really well.

poinck@lemm.ee on 19 Mar 10:07 next collapse

This looks a bit like borgbackup. It is also versioned and stores everything deduplicated, supports encryption and can be mounted using fuse.

Zenlix@lemm.ee on 19 Mar 10:44 collapse

Thanks for your hint towards borgbackup.

After reading the Quick Start of Borg Backup they look very similar. But as far as I can tell, borg can be encrypted and compressed while restic is always. You can mounting your backups in restic to. It also seems that restic supports more repository locations such as several cloud storages and via a special http server.

I also noticed that borg is mainly written in python while restic is written in go. That said I assume that restic is a bit faster based on the language (I have not tested that).

ferric_carcinization@lemmy.ml on 19 Mar 12:23 next collapse

I haven’t used either, much less benchmarked them, but the performance differences should be negligible due to the IO-bound nature of the work. Even with compression & encryption, it’s likely that either the language is fast enough or that it’s implemented in a fast language.

Still, I wouldn’t call the choice of language insignificant. IIRC, Go is strongly typed while Python isn’t. Even if type errors are rare, I would rather trust software written to be immune to them. (Same with memory safety, but both languages use garbage collection, so it’s not really relevant in this case.)

Of course, I could be wrong. Maybe one of the tools cannot fully utilize the network or disk. Perhaps one of them uses multithreaded compression while the other doesn’t. Architectual decisions made early on could also cause performance problems. I’d just rather not assume any noticeable performance differences caused by the programming language used in this case.

Sorry for the rant, this ended up being a little longer than I expected.

Also, Rust rewrite when? :P

drspod@lemmy.ml on 19 Mar 12:36 collapse

It was a while ago that I compared them so this may have changed, but one of the main differences that I saw was that borg had to backup over ssh, while restic had a storage backend for many different storage methods and APIs.

anamethatisnt@sopuli.xyz on 19 Mar 10:22 next collapse

Most of my data are on 2x16TB HDDs running an mdraid1 and then I backup it all to a usb drive with Borg Backup.
The os.qcow2 files live on my m.2 NVMe and are manually backuped to the mdraid1 before running the borg backup.
I should automate the borg backup but currently I just do it manually a few times a month.
Would also like to have two usb drives and keep one offline in another part of the house but that’s another future project.

savvywolf@pawb.social on 19 Mar 10:40 next collapse

I recently bought a storagebox from Hatzner and set up my server to run borgmatic every day to backup to it.

I’ve also discovered that Pika Backup works really well as a “read only” graphical browser for borg repos.

ouch@lemmy.world on 23 Mar 20:47 collapse

Do you use some kind of encryption on the VPS?

savvywolf@pawb.social on 24 Mar 01:48 collapse

Yep, borgmatic encrypts it before it sends data to the server.

Vintor@lemm.ee on 19 Mar 10:55 next collapse

I’ve found that the easiest and most effective way to backup is with an rsync cron job. It’s super easy to setup (I had no prior experience with either rsync or cron and it took me 10 minutes) and to configure. The only drawback is that it doesn’t create differential backups, but the full task takes less than a minute every day so I don’t consider that a problem. But do note that I only backup my home folder, not the full system.

For reference, this is the full line I use: sync -rau --delete --exclude-from=‘/home/<myusername>/.rsync-exclude’ /home/<myusername> /mnt/Data/Safety/rsync-myhome

“.rsync-exclude” is a file that lists all files and directories I don’t want to backup, such as temp or cache folders.

(Edit: two stupid errors.)

atzanteol@sh.itjust.works on 19 Mar 12:24 next collapse

You might be interested in “rsnapshot” which uses rsync and manages daily, monthly, etc. snapshots.

everett@lemmy.ml on 19 Mar 17:29 next collapse

only drawback is that it doesn’t create differential backups

This is a big drawback because even if you don’t need to keep old versions of files, you could be replicating silent disk corruption to your backup.

suicidaleggroll@lemm.ee on 19 Mar 20:45 collapse

It’s not a drawback because rsync has supported incremental versioned backups for over a decade, you just have to use the --link-dest flag and add a couple lines of code around it for management.

everett@lemmy.ml on 19 Mar 21:19 collapse

Sure, but that’s not in their answer.

suicidaleggroll@lemm.ee on 19 Mar 21:35 collapse

They didn’t provide an rsync example until later in the post, the comment about not supporting differential backups is in reference to using rsync itself, which is incorrect, because rsync does support differential backups.

I agree with you that not doing differential backups is a problem, I’m simply commenting that this is not a drawback of using rsync, it’s an implementation problem on the user’s part. It would be like somebody saying “I like my Rav4, it’s just problematic because I don’t go to the grocery store with it” and someone else saying “that’s a big drawback, the grocery store has a lot of important items and you need to be able to go to it”. While true, it’s based on a faulty premise, because of course a Rav4 can go to the grocery store like any other car, it’s a non-issue to begin with. OP just needs to fix their backup script to start doing differential backups.

everett@lemmy.ml on 19 Mar 22:02 collapse

My one and only purpose was to warn them that their “drawback” is more of a gator pit. It’s noble that you’re here defending rsync’s honor, but maybe let them know instead? My preferred backup tool has “don’t eat my data” mode on by default.

dihutenosa@lemm.ee on 19 Mar 18:29 collapse

Rsync can do incremental backups with a command-line switch and some symlink jugglery. I’m using it to back up my self-hosted stuff.

blade_barrier@lemmy.ml on 19 Mar 11:05 next collapse

Since most of the machines I need to backup are VMs, I do it by the means of hypervisor. I’d use borg scheduled in crontab for physical ones.

BlackEco@lemmy.blackeco.com on 19 Mar 11:15 next collapse

My work flow is pretty similar to yours:

For my desktop and laptops: systemd timer and service that backups every 15 minutes using restic to my NAS.

For my NAS : daily backup using restic + ZFS snapshots.

All restic backups are then uploaded daily to Backblaze B2.

ItTakesTwo@feddit.org on 19 Mar 14:21 collapse

Do you create ZFS snapshots and let those be backed up to B2 via restic or do you backup different types of data, one with ZFS snapshots and one with restic?

BlackEco@lemmy.blackeco.com on 19 Mar 14:24 collapse

Only restic snapshots are backed-up to B2. ZFS snapshots are for undoing mistakes, though I enabled them recently and I have yet to use them.

ItTakesTwo@feddit.org on 19 Mar 14:29 collapse

Gotcha, thanks! Similar setup to mine then.

Mio@feddit.nu on 19 Mar 11:55 next collapse

Timeshift for snapshots and deja backups for files

MentalEdge@sopuli.xyz on 19 Mar 11:56 next collapse

I recently switched to Kopia for my offsite backup solution.

It’s apparently one of the faster options, and it can be set up so that the files of the differential backups are handled by a repository server on the offsite end, so file management doesn’t need to happen over the network at a snails pace.

The result is a way to maintain frequent full backups of my nextcloud instance, with almost no downtime.

Nextcloud only goes into maintenance mode for the duration of a postgres database dump, after which the actual file system backup occurs using a temporary btrfs snapshot, containing a frozen filesystem at the time of the database dump.

mybuttnolie@sopuli.xyz on 19 Mar 12:06 next collapse

Nice try, mister ransonware attacker hacker!

privateX@lemmy.world on 19 Mar 12:19 next collapse

I keep all of my documents on a local server so all that is on any of my computers is software. So if I need to reinstall Linux I cab just do it without wording about losing anything.

Gieselbrecht@feddit.org on 19 Mar 12:22 next collapse

I’m curious, is there a reason why noone uses deja-dup? I use it with an external SSD on Ubuntu and (receently) Mint, where it comes pre-installed, and did not encounter Problems.

mazzilius_marsti@lemmy.world on 19 Mar 12:57 next collapse

What do you backup with dejadup? Everything under /home?

Gieselbrecht@feddit.org on 20 Mar 07:00 collapse

Mostly, with some folders excepted (e.g. my Nextcloud folder)

Kolossos@lemmy.world on 19 Mar 23:03 collapse

The restore process takes forever and sometimes fails. Last time I was forced to try every daily backup to several days before last backup to find one that could actually be restored. I have switched to borg (using Pika Backup for desktop and Borgmatic for servers). No restores have failed since.

Gieselbrecht@feddit.org on 20 Mar 07:00 collapse

Thanks for your feedback, maybe I should look into switching

zeca@lemmy.eco.br on 19 Mar 12:27 next collapse

i do backups of my home folder with Vorta, tha uses borg in the backend. I never tried restic, but borg is the first incremental backup utility i tried that doesnt increase the backup size when i move or rename a file. I was using backintime before to backup 500gb on a 750gb drive and if I moved 300gb to a different folder, it would try to copy those 300gb again onto the backup drive and fail for lack of storage, while borg handles it beautifully.

as an offsite solution, i use syncthing to mirror my files to a pc at my fathers house that is turned on just once in a while to save power and disc longevity.

melfie@lemmings.world on 19 Mar 12:29 next collapse

I currently use rclone with encryption to iDrive e2. I’m considering switching to Backrest, though.

I originally tried Backblaze b2, but exceeded their API quotas in their free tier and iDrive has “free” API calls, so I recently bought a year’s worth. I still have a 2 year Proton subscription and tried rclone with Proton drive, but it was too slow.

djsaskdja@reddthat.com on 19 Mar 12:41 next collapse

There’s nothing saved on my system I couldn’t afford to lose. All my work stuff is saved in Google Drive for better or worse. I have a few small files in a personal Proton Drive that I backup manually. I wipe my own system a few times a year and I rarely ever save anything first. Honestly very refreshing to live your life like that. Other than my cat, pretty much all my possessions could disappear tomorrow and I’d get over it pretty quickly.

isgleas@lemmy.ml on 19 Mar 12:49 next collapse

Backup? What?

thenextguy@lemmy.world on 19 Mar 14:36 collapse

Your car.

rutrum@programming.dev on 19 Mar 12:58 next collapse

I use borg the same way you describe. Part of my nixos config builds a systemd unit that starts a backup on various directories on my machine at midnight every day. I have 2 repos: one to store locally and on a cloud backup provider (borgbase) and another thats just stored locally. That is, another computer in my house. That local only is for all my home media. I havent yet put the large dataset of photos and videos on the cloud or offsite.

Pika@sh.itjust.works on 19 Mar 13:20 next collapse

for my server I use proxmox backup server to an external HDD for my containers, and I back up media monthly to an encrypted cold drive.

For my desktop? I use a mix of syncthing (which goes to the server) and windows file history(if I logged into the windows partition) and I want to get timeshift working I just have so much data that it’s hard to manage so currently I’ll just shed some tears if my Linux system fails

hallettj@leminal.space on 19 Mar 15:49 next collapse

My conclusion after researching this a while ago is that the good options are Borg and Restic. Both give you incremental backups with cheap timewise snapshots. They are quite similar to each other, and I don’t know of a compelling reason to pick one over the other.

Zenlix@lemm.ee on 19 Mar 17:38 collapse

As far as I know, by definition, at least restic is not incremental. It is a mix of full backup and incremental backup.

doubtingtammy@lemmy.ml on 19 Mar 16:01 next collapse

rsync 😪

bubbalouie@lemmy.ml on 19 Mar 17:28 next collapse

I rsync ~/ to a USB nub. A no brainer.

lattrommi@lemmy.ml on 19 Mar 18:10 next collapse

I want to say I’m glad you asked this and thank you for asking. In this day and age there are a lot of valid concerns for privacy and anonymity and the result is that people do not share how their system(s) work, not openly or very often. I’m still fairly new to Linux (3.5 years) and at times, I feel like I am doing everything wrong and that there is probably a better way. Posts like these help me learn about possible improvements or mistakes I might have made.

I previously used Vorta with Borgbackup locally, automatically backing up my Home (sans things like .cache and .mozilla) to a secondary internal drive every other day. I also would manually back up a smaller set of important documents (memes and porn #joke) to a USB flash drive, to keep on my person, which also would be copied across several cloud storage providers (dropbox, mega, proton), depending on how much space their free versions provided, with items removed according to how much I trusted the provider.

Then I built a new system. In the process of setting it all up, I had a few hiccups. It took longer than I expected to have a stable system. That was over a year ago (stat / …Birth: 2024-02-05 04:20:53…) and I still haven’t gotten around to setting up any backup system on it. I want to rethink my old solution and this post is useful for learning about the options available. It’s also a reminder to get it done before it is too late. Where I live, tornado season in starting. I lost a lot in 2019 after my city had 4 tornados in one day.

beeng@discuss.tchncs.de on 19 Mar 18:23 next collapse

Borg to a NAS.

500GB of that NAS is “special” so I then rsync that to a 500GB old laptop hdd, of which is is duplicated again to another 500GB old laptop hdd.

Same 500GB rsync’d to Cloud Server.

CrabAndBroom@lemmy.ml on 19 Mar 18:44 next collapse

I use BorgBackup with Vorta for a GUI, and I keep the 3-2-1 backup rule for important stuff (IE: 3 copies, 2 on different media, 1 off-site.)

limelight79@lemm.ee on 19 Mar 20:29 next collapse

My kmymoney file goes on an old compact flash memory card.

My home directory (including that file), /etc, databases, and a few other things get backed up weekly on to a USB stick.

Media raid array is automatically backed up to a large drive in another computer each evening. (The raid5 array isn’t that large. It was when I built it, but now I can buy a single drive that is nearly as large as the array…)

Pictures are backed up to Amazon’s glacier deep freeze. I pay about $1/month to back up all of my pictures. I intend to put other important things there too but haven’t gotten there yet.

suicidaleggroll@lemm.ee on 19 Mar 21:03 next collapse

My KVM hosts use “virsh backup begin” to make full backups nightly.

All machines, including the KVM hosts and laptops, use rsync with --link-dest to create daily incremental versioned backups on my main backup server.

The main backup server pushes client-side encrypted backups which include the latest daily snapshot for every system to rsync.net via Borg.

I also have 2 DASs with 2 22TB encrypted drives in each. One of these is plugged into the backup server while the other one sits powered off in a drawer in my desk at work. The main backup server pushes all backups to this DAS weekly and I swap the two DASs ~monthly so the one in my desk at work is never more than a month or so out of date.

tasankovasara@sopuli.xyz on 19 Mar 21:14 next collapse

  • daily important stuff (job stuff, Documents folder, Renoise mods) is kept synced between laptop, desktop and home server via Syncthing. A vimwiki additionally also syncs with the phone. Sync happens only when on home network.

  • the rest of the laptop and desktop I’ll roll into a tar backup every now and then with a quick bash alias. The tar files also get synced onto home server’s big file system (2 TB ssd) via Syncthing. Home server backs itself up on it’s own once a week.

  • clever thing is that the 2 TB ssd replaced an old 2 TB spinning disk. I kept the old disk and set up a systemd thing that keeps it spun down, but starts and mounts it once a week and rsyncs the changes to the ssd over, then unmounts it so that it sleeps again for a week. That old drive is likely to serve for years still with this frugal use.

ouch@lemmy.world on 23 Mar 20:45 collapse

How do you make sure the disk spins down? Is unmounting enough?

tasankovasara@sopuli.xyz on 24 Mar 12:19 collapse

Unmounting is enough if the disk has spindown configured. I’ve got this in /etc/udev/rules.d/ : ACTION==“add”, SUBSYSTEM==“block”, KERNEL==“sd[a-z]”, ENV{ID_SERIAL_SHORT}==“S2H7J9FZB02854”, RUN+=“/usr/bin/hdparm -S 70 /dev/%k”

jwt@programming.dev on 19 Mar 21:41 next collapse

Right now, nothing 😈 (but my dotfiles/etc configs live on several machines)

Before, I used Restic (incremental, encrypted backup over network), which I really liked. I think I should set it up again.

tiny@midwest.social on 19 Mar 23:16 next collapse

All my configs are in gitlab or a self hosted forgejo server and all files are in seafile or a self hosted service running on proxmox. Then I use proxmox backup server on a storage VPS for off-site backup

golden_zealot@lemmy.ml on 20 Mar 00:40 next collapse

Using timeshift. Very, very easy, works great.

Earflap@reddthat.com on 20 Mar 02:53 next collapse

I have a server with a RAID-1 array, that makes daily, weekly, and monthly read only btrfs snapshots. The whole thing (sans snapshots) is sync’d with syncthing to two rPi’s in two different geographic locations.

I know neither raid nor syncthing are “real” backup solutions, but with so many copies of the files living in so many locations (in addition to my phone, laptop, etc.) I’m reasonably confident its a decent solution.

data1701d@startrek.website on 20 Mar 03:40 next collapse

Borg Backup, whenever I feel like it - usually monthly.

ColdWater@lemmy.ca on 20 Mar 06:04 next collapse

I use external drive for my important data and if my system is borked (which never happen to me) I just reinstall the OS

floquant@lemmy.dbzer0.com on 20 Mar 08:32 collapse

External drives are more prone to damage and failures, both because they’re more likely to be dropped/bumped/spilled on etc, and because of generally cheaper construction compared to internal drives. In the case of SSDs the difference might be negligible, but I suggest you at least make a copy on another “cold” external drive if the data is actually important

hamburger@discuss.tchncs.de on 20 Mar 08:16 next collapse

  • Offline Backup on 2 separate HDD/SSD
  • Backup on HDD within my desktop pc
  • Backup offsite with restic to Hetzner Storage Box
haque@lemm.ee on 20 Mar 09:01 next collapse

I use Duplicacy to backup to my TrueNAS server. Crucial data like documents are backed up a second time to my GDrive, also using Duplicacy. Sadly it’s a paid solution, but it works great for me.

tankplanker@lemmy.world on 20 Mar 13:45 next collapse

Borg daily to the local drive then copied across to a USB drive, then weekly to cloud storage. Script is triggered by daily runs of topgrade before I do any updates

bitcrafter@programming.dev on 21 Mar 02:45 next collapse

I created a script that I dropped into /etc/cron.hourly which does the following:

  1. Use rsync to mirror my root partition to a btrfs partition on another hard drive (which only updates modified files).
  2. Use btrfs subvolume snapshot to create a snapshot of that mirror (which only uses additional storage for modified files).
  3. Moves “old” snapshots into a trash directory so I can delete them later if I want to save space.

It is as follows:

#!/usr/bin/env python
from datetime import datetime, timedelta
import os
import pathlib
import shutil
import subprocess
import sys

import portalocker

DATETIME_FORMAT = '%Y-%m-%d-%H%M'
BACKUP_DIRECTORY = pathlib.Path('/backups/internal')
MIRROR_DIRECTORY = BACKUP_DIRECTORY / 'mirror'
SNAPSHOT_DIRECTORY = BACKUP_DIRECTORY / 'snapshots'
TRASH_DIRECTORY = BACKUP_DIRECTORY / 'trash'

EXCLUDED = [
    '/backups',
    '/dev',
    '/media',
    '/lost+found',
    '/mnt',
    '/nix',
    '/proc',
    '/run',
    '/sys',
    '/tmp',
    '/var',

    '/home/*/.cache',
    '/home/*/.local/share/flatpak',
heythatsprettygood@feddit.uk on 21 Mar 20:25 collapse

I use Pika Backup (GUI that uses Borg Backup on the backend) to back up my desktop to my home server daily, then overnight that server has a daily backup using Borg to a Hetzner Storage Box. It’s easy to set it and forget it (other than maybe verifying the backups every once in a while), and having that off site back up gives me peace of mind.