What's with all these hip filesystems and how are they different?
from rutrum@lm.paradisus.day to linux@lemmy.ml on 07 Dec 2023 11:47
https://lm.paradisus.day/post/70709

You know, ZFS, ButterFS (btrfs…its actually “better” right?), and I’m sure more.

I think I have ext4 on my home computer I installed ubuntu on 5 years ago. How does the choice of file system play a role? Is that old hat now? Surely something like ext4 has its place.

I see a lot of talk around filesystems but Ive never found a great resource that distiguishes them at a level that assumes I dont know much. Can anyone give some insight on how file systems work and why these new filesystems, that appear to be highlights and selling points in most distros, are better than older ones?

Edit: and since we are talking about filesystems, it might be nice to describe or mention how concepts like RAID or LUKS are related.

#linux

Cwilliams@beehaw.org on 07 Dec 2023 11:52 next collapse

Not sure about the other ones, but I use Btrfs because of subvolumes and backups.

Subvolumes are like special folders inside of your partition that mount separately. Ex. In my btrfs partition, I have a @home partition that is mounted to /home

This makes it easier to choose what you are backing up, because you can say, “just copy everything in @home to the backup location”

If I got any of that wrong, feel free to correct me!

zephr_c@lemm.ee on 07 Dec 2023 12:12 collapse

I mean, is it actually easier to copy everything in @home than it is to copy everything in /home? Btrfs has always kinda felt like it’s a bunch of extra steps to solve problems I don’t have.

mindfive@lemm.ee on 07 Dec 2023 13:38 next collapse

Kinda. You can copy your snapshots from @home too, meaning a restore from backup also restores your local file version history. There are also tools to push snapshots around as a large archive instead of dealing with smaller files directly.

The COW can also reduce the chances of running rsync on a large file that is currently being accessed, and getting a partial file in your backup. Or I suck at rsync 🤷‍♂️

Chewy7324@discuss.tchncs.de on 07 Dec 2023 14:06 collapse

You’re right, atomic snapshots are a big advantage of CoW fs.

Rsync backups done while the system is running have a chance of being broken, while CoW fs snapshots are instant and seem basically as if the system suddenly lost power.

narc0tic_bird@lemm.ee on 07 Dec 2023 13:41 next collapse

Not sure, but maybe you can snapshot these subvolumes independently?

anothermember@beehaw.org on 07 Dec 2023 15:45 collapse

The real power for btrfs for me is incremental backups; you can take a snapshot of your home partition and send it to a backup device, then you can take a second snapshot a week later and just send the differences between them. I do my weekly backups like this. You can keep many multiple snapshots to roll back if needs be since only the differences between snapshots take up space. This is the tutorial that got me started.

zephr_c@lemm.ee on 07 Dec 2023 15:53 collapse

Yeah, alright, I see how that could be useful for someone who isn’t me. I don’t have much that’s important on my computer, and for what little there is I just have a second ssd I drag and drop it onto. That one has Mint installed on it in case I do something stupid to my main drive, because I routinely do stupid things to my main drive.

anothermember@beehaw.org on 07 Dec 2023 16:04 collapse

I suppose it depends on how much stuff you have, doing a full back up of my home every week is too time consuming to be practical but takes a couple of minutes with this method.

Keeping multiple past snapshots is overkill for me but I do it because I can, more-or-less. It would be useful if I accidentally delete a file and only remember it months later.

i_am_not_a_robot@feddit.uk on 07 Dec 2023 12:03 next collapse

I use f2fs on my Raspberry Pis, it’s designed for flash storage and appears to have much better performance than ext4 on the same device. I’m not sure whether it’s suitable for SSDs, or just SD cards and USB (these devices are optimised for FAT and f2fs utilises that optimisation). When I tried to use f2fs on a proper laptop it was too early and the distro didn’t support booting from it. I assume that has changed now.

As for the others, I usually stick with ext4 as I’ve never seen a compelling reason not to.

Chewy7324@discuss.tchncs.de on 07 Dec 2023 14:12 collapse

Interesting. f2fs supports file-based encryption and compression. It is designed for flash and is used for many smartphones.

i_am_not_a_robot@feddit.uk on 07 Dec 2023 14:55 collapse

What I missed mentioning is it does wear-levelling so as its name suggests it is “flash friendly” and stops SD cards wearing out so quickly.

aBundleOfFerrets@sh.itjust.works on 07 Dec 2023 15:05 collapse

I don’t believe this is true. F2FS is still meant for use on devices with a FTL (flash transition layer) meaning that the device is doing wear leveling itself and a filesystem doing it twice is redundant and counter-productive. The flash-friendly part is referring to other filesystem features (there are many)

i_am_not_a_robot@feddit.uk on 07 Dec 2023 15:08 collapse

I bow to your superior knowledge. It definitely doesn’t wear out SD cards as quickly though, but that might be due to other factors not wear levelling.

aksdb@feddit.de on 07 Dec 2023 12:05 next collapse

As with every software/product: they have different features.

ZFS is not really hip. It’s pretty old. But also pretty solid. Unfortunately it’s licensed in a way that is maybe incompatible with the GPL, so no one wants to take the risk of trying to get it into Linux. So in the Linux world it is always a third-party-addon. In the BSD or Solaris world though …

btrfs has similar goals as ZFS (more to that soon) but has been developed right inside the kernel all along, so it typically works out of the box. It has a bit of a complicated history with it’s stability/reliability from which it still suffers (the history, not the stability). Many/most people run it with zero problems, some will still cite problems they had in the past, some apparently also still have problems.

bcachefs is also looming around the corner and might tackle problems differently, bringing us all the nice features with less bugs (optimism, yay). But it’s an even younger FS than btrfs, so only time will tell.

ext4 is an iteration on ext3 on ext2. So it’s pretty fucking stable and heavily battle tested.

Now why even care? ZFS, btrfs and bcachefs are filesystems following the COW philisophy (copy on write), meaning you might lose a bit performance but win on reliability. It also allows easily enabling snapshots, which all three bring you out of the box. So you can basically say “mark the current state of the filesystem with tag/label/whatever ‘x’” and every subsequent changes (since they are copies) will not touch the old snapshots, allowing you to easily roll back a whole partition. (Of course that takes up space, but only incrementally.)

They also bring native support for different RAID levels making additional layers like mdadm unnecessary. In case of ZFS and bcachefs, you also have native encryption, making LUKS obsolete.

For typical desktop use: ext4 is totally fine. Snapshots are extremely convenient if something breaks and you can basically revert the changes back in a single command. They don’t replace a backup strategy, so in the end you should have some data security measures in place anyway.

*Edit: forgot a word.

Fizz@lemmy.nz on 07 Dec 2023 12:20 next collapse

So ext4 is the best for desktop gaming performance?

noddy@beehaw.org on 07 Dec 2023 12:28 next collapse

I remember reading somewhere that btrfs has good performance for gaming because of deduplication. I’m using btrfs, haven’t benchmarked it or anything, but it seems to work fine.

aberrate_junior_beatnik@lemmy.world on 07 Dec 2023 12:29 next collapse

I’d be surprised to find out there was one filesystem that consistently did better than others in gaming performance. ext4 is a fine choice, though.

lurch@sh.itjust.works on 07 Dec 2023 13:33 collapse

does tmpfs count?

pastermil@sh.itjust.works on 07 Dec 2023 16:44 collapse

No.

aksdb@feddit.de on 07 Dec 2023 12:30 next collapse

It likely has an edge. But I think on SSDs the advantage is negligible. Also games have the most performance critical stuff in-memory anyway so the only thing you could optimize is read performance when changing scenes.

Here are some comparisons: www.phoronix.com/news/Linux-5.14-File-Systems

But again … practically you can likely ignore the difference for desktop usage (also gaming). The workloads where it matters are typically on servers with high throughput where latencies accumulate quickly.

flashgnash@lemm.ee on 07 Dec 2023 12:30 next collapse

Having tried NTFS, ext4 and btrfs, the difference is not noticeable (though NTFS is buggy on Linux)

Btrfs I believe has compression built in so is good for large libraries but realistically ext4 is the easiest and simplest way to do so I just use that nowadays

Cwilliams@beehaw.org on 07 Dec 2023 13:35 next collapse

Well that’s because any support for it is unofficial. NTFS is made for Windows

MonkderZweite@feddit.ch on 07 Dec 2023 16:25 collapse

And proprietary and an old piece of garbage.

Cwilliams@beehaw.org on 07 Dec 2023 22:56 collapse

I didn’t want to sound to harsh, but yea

Flaky@iusearchlinux.fyi on 07 Dec 2023 14:29 collapse

I had a pretty bad experience with the Paragon NTFS3 drivers a couple years ago. Basically the kernel hung, maybe from this, maybe not, but it ended up with filesystem corruption on my hard drives.

Thankfully, Windows was able to fix it but until recently I relied on NTFS-3G. Paragon’s NTFS3 driver seems to be faring a lot better nowadays.

Auli@lemmy.ca on 07 Dec 2023 12:56 next collapse

Going to be they or XFS. There was a benchmark of the different filesystems I heard about never found it though. It was recent and included bcachefs

[deleted] on 07 Dec 2023 15:25 collapse
.
rutrum@lm.paradisus.day on 07 Dec 2023 12:27 next collapse

Perhaps I’m guilty of good luck, but is the trade off of performance for reliability worth it? How often is reliability a problem?

As a different use case altogether, suppose I was setting up a NAS over a couple drives. Does choosing something with COW have anything to do with redundancy?

Maybe my question is, are there applications where zfs/btrfs is more or less appropriate than ext4 or even FAT?

aksdb@feddit.de on 07 Dec 2023 12:37 next collapse

For fileservers ZFS (and by extension btrfs) have a clear advantage. The main thing is, that you can relatively easily extend and section off storage pools. For ext4 you would need LVM to somewhat achieve something similar, but it’s still not as mighty as what ZFS (and btrfs) offer out of the box.

ZFS also has a lot of caching strategies specifically optimized for storage boxes. Means: it will eat your RAM, but become pretty fast. That’s not a trade-off you want on a desktop (or a multi purpose server), since you typically also need RAM for applications running. But on a NAS, that is completely fine. AFAIK TrueNAS defaults to ZFS. Synology uses btrfs by default. Proxmox runs on ZFS.

4am@lemm.ee on 07 Dec 2023 13:52 collapse

ZFS cache will mark itself as such, so if the kernel needs more RAM for applications it can just dump some of the ZFS cache and use whatever it needs.

I see lots of threads on homelab where new users are like “HELP MY ZFS IS USING 100% MEMORY” and we have to talk them off that ledge: unused RAM is wasted RAM, ZFS is making sure you’re running fast AF.

aksdb@feddit.de on 07 Dec 2023 13:59 next collapse

ZFS cache will mark itself as such, so if the kernel needs more RAM for applications it can just dump some of the ZFS cache and use whatever it needs.

In theory. Practically unless I limit the max ARC size, processes get OOM killed quite frequently here.

MonkderZweite@feddit.ch on 07 Dec 2023 16:30 collapse

unused RAM is wasted RAM

In theory. But how it is implemented in current systems, reserved memory can not be used by other processes and those other processes can not just ask the hog to give some space. Eventually, the hog gets OOM-killed or the system freezes.

PixxlMan@lemmy.world on 08 Dec 2023 11:22 collapse

Even when, as the comment says, the memory is marked as cache?

Windows doesn’t have this problem

ReversalHatchery@beehaw.org on 07 Dec 2023 16:41 collapse

are there applications where zfs/btrfs is more or less appropriate than ext4 or even FAT?

Neither of them likes to deal with very low amounts of free space, so don’t use it on places where that is often a scarcity. ZFS gets really slow when free space is almost none, and nowadays I don’t know about BTRFS but a few years ago filling the partition caused data corruption there.

excitingburp@lemmy.world on 07 Dec 2023 13:17 next collapse

Btw COW isn’t necessarily (and isn’t at least for ZFS) a performance trade-off. Data isn’t really copied, new data is simply written elsewhere on the disk (and the old data is not marked as free space).

Ultimately it actually means “the data behaves as though it was copied,” which can be achieved in many ways. There are many ways to do that without actually copying.

teawrecks@sopuli.xyz on 07 Dec 2023 22:45 collapse

So let me give an example, and you tell me if I understand. If you change 1MB in the middle of a 1GB file, the filesystem is smart enough to only allocate a new 1MB chunk and update its tables to say “the first 0.5GB lives in the same old place, then 1MB is over here at this new location, and the remaining 0.5GB is still at the old location”?

If that’s how it works, would this over time result in a single file being spread out in different physical blocks all over the place? I assume sequential reads of a file stored contiguously would usually have better performance than random reads of a file stored all over the place, right? Maybe not for modern SSDs…but also fragmentation could become a problem, because now you have a bunch of random 1MB chunks that are free.

I know ZFS encourages regular “scrubs” that I thought just checked for data integrity, but maybe it also takes the opportunity to defrag and re-serialize? I also don’t know if the other filesystems have a similar operation.

d3Xt3r@lemmy.nz on 08 Dec 2023 07:21 collapse

Not OP, but yes, that’s pretty much how it works. (ZFS scrubs do not defrgment data however).

Fragmentation isn’t really a problem for several reasons.

  • Some (most?) COW filesystems have mechanisms to mitigate fragmentation. ZFS, for instance, uses a special allocation strategy to minimize fragmentation and can reallocate data during certain operations like resilvering or rebalancing.

  • ZFS doesn’t even have a traditional defrag command. Because of its design and the way it handles file storage, a typical defrag process is not applicable or even necessary in the same way it is with other traditional filesystems

  • Btrfs too handles chunk allocation effeciently and generally doesn’t require defragmentation, and although it does have a defrag command, it’s almost never used by anyone, unless you have a special reason to (eg: maybe you have a program that is reading raw sectors of a file, and needs the data to be contiguous).

  • Fragmentation is only really an issue for spinning disks, however, that is no longer a concern for most spinning disk users because:

    • Most home users who still have spinning disks use it for archival/long term storage/media that rarely changes (eg: photos, movies, other infrequently accessed data), so fragmentation rarely occurs here and even if it does, it’s not a concern.
    • Power users typically have a DAS or NAS setup where spinning disks are in a RAID config with striping, so the spread of data across multiple sectors actually has an advantage for averaging out read times (so no file is completely stuck in the slow regions of a disk), but also, any performance loss is also generally negated because a single file can typically be read from two or more drives simultaneously, depending on the redundancy config.
  • Enterprise users also almost always use a RAID (or similar) setup, so the same as above applies. They also use filesystems like ZFS which employs heavy caching mechanisms, typically backed by SSDs/NVMes, so again, fragmentation isn’t really an issue.

teawrecks@sopuli.xyz on 08 Dec 2023 20:22 collapse

Cool, good to know. I’d be interested to learn how they mitigate fragmentation, though. It’s not clear to me how COW could mitigate the copy cost without fragmentation, but I’m certain people smarter than me have been thinking about the problem for my whole life. I know spinning disks have their own set of limitations, but even SSDs perform better on sequential reads over random reads, so it seems like the preference would still be to not split a file up too much.

ReversalHatchery@beehaw.org on 07 Dec 2023 16:32 next collapse

In case of ZFS and bcachefs, you also have native encryption, making LUKS obsolete.

I don’t think that it makes LUKS obsolete. LUKS encrypts the entire partition, but ZFS (and BTRFS too as I know) only encrypt the data and some of the metadata, the rest is kept as it is.

openzfs.github.io/…/zfs-load-key.8.html#Encryptio…

Data that is not encrypted can be modified from the outside (the checksums have to be updated of course), which can mean from a virus on a dual booted OS to an intruder/thief/whatever.
If you have read recently about the logofail attack, the same could happen with modifying the technical data of a filesystem, but it may be bad enough if they just swap the names of 2 of your snapshots if they just want to cause trouble.

But otherwise this is a good summary.

lemann@lemmy.one on 07 Dec 2023 17:19 collapse

BTRFS has encryption now? Yay!! I have been wrapping it inside a LUKS partition for years at this point…

KiranWells@pawb.social on 07 Dec 2023 23:39 collapse

They said bcachefs; I don’t think BTRFS has it, at least not since I last checked.

mcepl@lemmy.world on 07 Dec 2023 21:25 collapse

ZFS is not really hip. It’s pretty old. But also pretty solid. Unfortunately it’s licensed in a way that is maybe incompatible with the GPL, so no one wants to take the risk of trying to get it into Linux. So in the Linux world it is always a third-party-addon. In the BSD or Solaris world though …

Also ZFS has tendency to have HIGH (really HIGH) hardware/CPU/memory requirements.

bamboo@lemm.ee on 08 Dec 2023 09:36 collapse

It was originally designed for massive storage servers (“zettabyte” file system) rather than personal laptops and desktops. It was before the current convergence trend too, so allocating all of the system resources to the file system was considered very beneficial if it could improve performance.

mcepl@lemmy.world on 24 Dec 2023 23:04 collapse

I haven’t meant it as the criticism of ZFS. It is just so, and perhaps there were good reasons for it. Now (especially with the convergence trend) it hurts.

bjoern_tantau@swg-empire.de on 07 Dec 2023 12:16 next collapse

I’ve started using BTRFS on my laptop with OpenSUSE and on my Steam Deck. It does two things for me, which I’m interested in. On OpenSUSE it does a snapshot before every system update. So if anything goes wrong I can easily roll back.

On the Steam Deck I love the deduplication. It’s really great for a ton of Windows games that all need their own little “Windows” environment which amounts to a GB or two per game. With BTRFS I only use that space once.

rutrum@lm.paradisus.day on 07 Dec 2023 12:32 collapse

Can you elaborate more on deduplication? Is this a feature you setup, or does it sort of work out of the box? This is a new concept to me, but sounds incredibly useful, especially in that scenario.

bjoern_tantau@swg-empire.de on 07 Dec 2023 12:57 next collapse

I used a script that did everything for me, so I’m not 100 % sure. But as far as I know you enable the feature at mount time and then every time you copy something only a reference is copied until you actually do a change to the new or old file.

For everything else a cronjob runs every week or so to search for unnecessary duplicates.

Chewy7324@discuss.tchncs.de on 07 Dec 2023 14:02 collapse

And if a copied file is changed, btrfs only stores the difference instead of two complete files. E.g. if the 1GB file1 is copied to file2, they will take 1GB total. If 100MB is appended to file2, the total storage usage is 1,1GB

[deleted] on 07 Dec 2023 15:31 collapse
.
Discover5164@lemm.ee on 07 Dec 2023 13:35 next collapse

it’s not automatic since it will eat resources while it’s running. but it’s a feature of btrfs.

[deleted] on 07 Dec 2023 17:26 collapse
.
BeefPiano@lemmy.world on 07 Dec 2023 12:28 next collapse

I don’t know about the new ones, but ReiserFS was a killer back in the day.

n2burns@lemmy.ca on 07 Dec 2023 13:14 next collapse
rotopenguin@infosec.pub on 07 Dec 2023 16:45 collapse

The horrible part is it was. Your other choice was ext2, which wasted so many lifetimes with its hours long fsck times. Reiserfs was a cut above the rest, we would all be using it today if it weren’t for that one teensy-weensy legal issue.

cybersandwich@lemmy.world on 08 Dec 2023 00:42 collapse

I have an old drive with it in there. The drive is going bad so I haven’t messed with it too much. I never knew at the time why the development and shine faded so quickly.

Frederic@beehaw.org on 07 Dec 2023 12:40 next collapse

After using ext4 for yyyeeeaaaarrrrrsss, when I upgraded my MX21 to MX23 I used btrfs, with subvolumes, especially for easy backup/snapshot/timeshift.

Just at install, super easy, create a small ext4 boot partition on the SSD, then a big LUKS partition, format with btrfs, create subvolumes for / /home /var /swap and that’s it. No hassle with sizing correctly.

btrfs seems pretty stable. I see no diff in performance compared to ext4 because my application are not that dependant to FS speed, and with SSD anyway?

oh yeah, built-in compression too!

actual_patience@programming.dev on 26 Dec 2023 13:19 collapse

ext4 boot partition? Does that mean you have Coreboot, not UEFI?

Frederic@beehaw.org on 26 Dec 2023 22:28 collapse

EFI with grub

actual_patience@programming.dev on 26 Dec 2023 22:39 collapse

Don’t you need FAT 32 for compatibility?

Frederic@beehaw.org on 28 Dec 2023 02:52 collapse

Yes for /boot/efi you’re right but /boot is ext4

jokro@feddit.de on 07 Dec 2023 13:15 next collapse

btrfs = B-tree filesystem

B-trees are a data structure.

kpw@kbin.social on 07 Dec 2023 13:35 collapse

Great. So what?

Chewy7324@discuss.tchncs.de on 07 Dec 2023 13:56 collapse

You know, ZFS, ButterFS (btrfs…its actually “better” right?), and I’m sure more.

tastysnacks@programming.dev on 07 Dec 2023 15:13 collapse

I call it butterface

teawrecks@sopuli.xyz on 07 Dec 2023 23:06 collapse

Butterface?! That’s my wife!

christian@lemmy.ml on 07 Dec 2023 14:58 next collapse

I know I’m not making a helpful contribution here, but I’ve been wondering about this stuff for a while myself and this thread has some great answers. Thanks for asking this OP.

possiblylinux127@lemmy.zip on 07 Dec 2023 15:24 next collapse

ZFS is a crazy beast that’s best for high end server systems with tiered storage and lots of RAM.

ext4 is really just a basic file system. Its superior to NTFS and fat as it does have extra features to try to prevent corruption but it doesn’t have a large feature set.

Btrfs is kind of the new kid on the block. It has strong protection against corruption and has better real world performance than ext4. It also has more advanced features like sub volumes and snapshots. subvolumes are basically virtual drives.

Another few older options include things like XFS but I won’t go into those.

List of filesystems: en.m.wikipedia.org/…/Comparison_of_file_systems

Lojcs@lemm.ee on 07 Dec 2023 16:44 collapse

and has better real world performance than ext4

Source? Most benchmarks I’ve seen it lags behind

possiblylinux127@lemmy.zip on 07 Dec 2023 17:43 next collapse

My personal testing

the_third@feddit.de on 07 Dec 2023 17:48 collapse

Let’s agree on: it has a different performance for various use cases and hardware below. Sometimes better, sometimes worse.

possiblylinux127@lemmy.zip on 07 Dec 2023 18:12 collapse

Well I use it pretty much exclusively now for bare hardware. For VMs it doesn’t matter so I use ext4

the_third@feddit.de on 07 Dec 2023 18:20 collapse

Not much time, I’ll be brief with three examples that come to mind from my experience:

Great use: Large filestorage with regular changes, daily snapshots, stream snapshots offsite as backup.

Not so great use: Storage backend for qcow2 backed VMs on spinning RAID. CoW made a mess of access times.

Really not great use: Large Postgres-DB with queries that creted large ondisk temp tables.

It really depends.

AMDIsOurLord@lemmy.ml on 07 Dec 2023 17:59 next collapse

Benchmarks are also usually very different from real world usage, tbh

Pantherina@feddit.de on 07 Dec 2023 22:17 collapse

www.phoronix.com/news/Linux-5.14-File-Systems

It seems actually F2FS is the best?? Thats used in Android and optimized for Flash storage, does that include SSDs?

Lojcs@lemm.ee on 07 Dec 2023 23:08 next collapse

Yes, but most filesystems are already optimized for flash storage. Arch wiki says f2fs is prone to corruption on power loss. Based on that and the lack of information on its anti-corruption measures I’m inclined to think it doesn’t have one and that’s why it’s faster. I wouldn’t use it in a non-battery operated device.

Pantherina@feddit.de on 07 Dec 2023 23:39 collapse

So basically all laptop users can safely use it.

Crazy how PC users rely on such a steady power supply. Arent there small UPS devices for a few seconds with auto shutdown?

Gabu@lemmy.ml on 07 Dec 2023 23:54 collapse

Catastrophic battery failure isn’t really any less likely than catastrophic power supply failure (conceptually. If you use a brandless grey power supply, results may vary).

d3Xt3r@lemmy.nz on 08 Dec 2023 06:32 collapse

That link is for kernel 5.14, so I’d say those results are pretty much invalid for most users (unless you’re actually on it, or the 5.15 LTS kernel). There have been a ton of improvements in every filesystem since then, with pretty much every single kernel release.

A more relevant test would be this one - although it talks about bcachefs, other filesystems are also included in it. As you can see, F2FS is no longer the fastest - bcachefs and XFS beat it in several tests, and even btrfs beats it in some tests. F2FS only wins in the Dbench and CockroachDB benchmarks.

Pantherina@feddit.de on 08 Dec 2023 16:16 collapse

Thanks. Bcachefs is for SSD-HDDs isnt it?

d3Xt3r@lemmy.nz on 08 Dec 2023 19:55 collapse

Not quite. Bcachefs can be used on any drive, but it shines the best when you have a fast + slow drive in your PC (eg NVMe + HDD), so the faster drive can be used as a cache drive to store frequently accessed data.

ReversalHatchery@beehaw.org on 07 Dec 2023 16:08 next collapse

ext4 certainly has its place, it’s a fine default file system, there’s really no problems with it.

But others, like ZFS and BTRFS, have features that you may want to use, but ext4 doesn’t do: fs snapshots, data compression, built in encryption (to a degree, usually only happening for data and some of the metadata, so LUKS is often better IMHO), checking for bitrot and restoring it when possible (whether it is depends on your config), quotas per user group or project, spanning multiple disks like with RAID but safer (to a degree), and others.

lolcatnip@reddthat.com on 07 Dec 2023 16:30 next collapse

ButterFS (btrfs…its actually “better” right?),

I’m still waiting to find out who the BCA Chefs are.

callyral@pawb.social on 07 Dec 2023 17:02 next collapse

related question, although i don’t think it’s big enough for a post of its own.

if i use btrfs subvolumes, does it mean that i can have one EFI partition and one root partition, and then subdivide the root partition using subvolumes? how would that work during the installation process? or is it done after installation?

the_third@feddit.de on 07 Dec 2023 17:47 next collapse

if i use btrfs subvolumes, does it mean that i can have one EFI partition and one root partition, and then subdivide the root partition using subvolumes?

Yes.

how would that work during the installation process? or is it done after installation?

That depends on your distro. What do you plan to achieve with the separate subvolumes?

callyral@pawb.social on 07 Dec 2023 19:38 collapse

I already have a partition layout in btrfs where I have a /home and a /root partition, since when I installed I didn’t know about btrfs subvolumes. I use Void Linux and I think it’s after installation, since I don’t remeber having a subvolumes step during the installation.

I’ll make sure to remind about btrfs subvolumes in case I reinstall. There’s a btrfs program that has a subvolume argument, so I’m guessing that’s what I could use.

usage: btrfs [global]  [...]  []

...
Command groups:
  subvolume         manage subvolumes: create, delete, list, etc
mhz@lemm.ee on 07 Dec 2023 18:31 next collapse

One EFI + one ROOT partition is what I do on both my laptop and desktop for years, /home is a subvolume to my root partition. This setup suits my needs as I don’t have to worry about how big should my root or home (gaming) partition should be.

I use Arch on my desktop and Opensuse on my laptop. They both have options to set up subvolumes from their installer, Debian does not, and I’m not sure about other distros, but you can always set that up after installation, just make your home partition the last one (after the root partition) so you can easily delete it after and grow the root partition without much blocks relocation.

danielquinn@lemmy.ca on 07 Dec 2023 20:08 collapse

I’ve never heard of sub volumes. What do they do for me? Why not just partition the disk or store everything on the one partition?

mhz@lemm.ee on 07 Dec 2023 22:22 collapse

I like to think a subvolume is a directory on my filesystem that:

  • Acts as an independent filesystem.
  • Shares it’s parent size (unless quotas are set in place)
  • Can be mounted/unmounted any time
  • Excluded from their parent partition’s snapshots. (a /home subvolume is exluded from / snapshots).
  • Can be snapshot-ed independently.

This is by no mead a definition for BTRFS subvolume, but I hope you get the idea.

lupec@lemm.ee on 07 Dec 2023 20:07 next collapse

Pretty much, yeah. At some point I remember the recommendation being having a separate /boot as well due to incompatibilities with GRUB’s save default option iirc, not sure that’s a thing anymore.

Anyway, you usually set that up during the install process, although I’m not sure graphical installers let you handcraft btrfs subvolume mount points or even select them as such these days. Last I checked at least they either just used a default layout (@ and @home with Ubuntu, for instance) or treated it as a single volume with no further options.

RustyNova@lemmy.world on 08 Dec 2023 11:56 collapse

Exactly. But if you tend to be on the hoarder side, put a swap partition in there too. Even 32GB ram isn’t enough sometimes

AMDIsOurLord@lemmy.ml on 07 Dec 2023 17:58 next collapse

Using Btrfs you can do some pretty cool snapshotting: It’s basically like system restore of Windows but MUCH faster and pretty seamless. Even if you annihilate the whole operating system you can restore the snapshot and voila, have fun! It also has compression which can save some wear on SSDs and of course give you some more free™ storage space, which is cool [actual benefits depend on workload*]

ipkpjersi@lemmy.ml on 07 Dec 2023 20:39 next collapse

ZFS also has snapshotting too FWIW.

AMDIsOurLord@lemmy.ml on 08 Dec 2023 09:29 collapse

ZFS has almost everything ever conceived for filesystems lol it’s a whole ass volume manager and filesystem into one

mcepl@lemmy.world on 07 Dec 2023 21:40 next collapse

This is twelve years old, but it nicely illustrates what BTRFS (and ZFS on other OS) can do … youtu.be/9H7e6BcI5Fo?t=206

PipedLinkBot@feddit.rocks on 07 Dec 2023 21:40 collapse

Here is an alternative Piped link(s):

https://piped.video/9H7e6BcI5Fo?t=206

Piped is a privacy-respecting open-source alternative frontend to YouTube.

I’m open-source; check me out at GitHub.

jlow@beehaw.org on 07 Dec 2023 21:57 next collapse

I wasn’t cool enough to figure out how to “just boot into a snapshot” when I tried btrfs a while ago. I mean I did figure it out (maybe?) but somehow the read/write rights where messed up and the snapshot couldn’t actually boot/I wasn’t able to log in +___+ Just reinstalled the system eith good old ext4. It sounds really cool, though …

Pantherina@feddit.de on 07 Dec 2023 22:13 next collapse

Fedora out of the box just works

AMDIsOurLord@lemmy.ml on 08 Dec 2023 09:27 collapse

Well, sounds like a setup or distro issue. It should work without problems on Debian/Ubuntu/Mint. Linux Mint even really supports it as a setup-less default with TimeShift

Pantherina@feddit.de on 07 Dec 2023 22:12 collapse

Do you know how I could split my default /var/home/user into /var/home/user/.var, /var/home/user/Torrents and the rest?

Think that would be great for use with btrbk, when I find out how to use that.

Damn BTRFS and btrbk need an easy GUI, I have the feeling its great for backups

AMDIsOurLord@lemmy.ml on 08 Dec 2023 09:25 collapse

There’s no GUI, but following the wiki pages on BTRFS subvolumes you should be able to make a subvolume for those with like 2 simple commands (take a look at the man page for BTRFS subvolumes as well)

Pantherina@feddit.de on 07 Dec 2023 22:18 next collapse

Fedora uses BTRFS so I get the features are the best argument for it

www.phoronix.com/news/Linux-5.14-File-Systems

But it seems F2FS is by far the fastest in many areas! Its used in Android, optimized for Flash storage.

sexy_peach@feddit.de on 07 Dec 2023 23:20 next collapse

you’re forgetting about the all new bcachefs

dunno I also wonder if it’s worth it for personal use

rutrum@lm.paradisus.day on 08 Dec 2023 16:48 collapse

Now THAT is hip

sexy_peach@feddit.de on 08 Dec 2023 16:57 collapse

Well I am a very cool guy 😏

duncesplayed@lemmy.one on 08 Dec 2023 00:27 next collapse

The principled “old” way of adding fancy features to your filesystem was through block-level technologies, like LVM and LUKS. Both of those are filesystem-agnostic, meaning you can use them with any filesystem. They just act as block devices, and you can put any filesystem on top of them.

You want to be able to dynamically grow and shrink partitions without moving them around? LVM has you covered! You want to do RAID? mdadm has you covered! You want to do encryption? LUKS has you covered? You want snapshotting? Uh, well…technically LVM can do that…it’s kind of awkward to manage, though.

Anyway, the point is, all of them can be mixed and matched in any configuration you want. You want a RAID6 where one device is encrypted split up into an ext4 and two XFS partitions where one of the XFS partitions is in RAID10 with another drive for some stupid reason? Do it up, man. Nothing stopping you.

For some reason (I’m actually not sure of the reason), this stagnated. Red Hat’s Strata project has tried to continue pushing in this direction, kind of, but in general, I guess developers just didn’t find this kind of work that sexy. I mentioned LVM can do snapshotting "kind of awkward"ly. Nobody’s done it in as sexy and easy way to do as the cool new COWs.

So, ZFS was an absolute bombshell when it landed in the mid 2000s. It did everything LVM did, but way way way better. It did everything mdadm did, but way way way better. It did everything XFS did, but way way way better. Okay it didn’t do LUKS stuff (yet), but that was promised to be coming. It was Copy-On-Write and B-tree-everywhere. It did everything that (almost) every other block-level and filesystem previously made had ever done, but better. It was just…the best. And it shit all over that block-layer stuff.

But…well…it needed a lot of RAM, and it was licensed in a way such that Linux couldn’t get it right away, and when it did get ZFS support, it wasn’t like native in-the-kernel kind of stuff that people were used to.

But it was so good that it inspired other people to copy it. They looked at ZFS and said “hey why don’t we throw away all this block-level layered stuff? Why don’t we just do every possible thing in one filesystem?”.

And so BtrFS was born. (I don’t know why it’s pronounced “butter” either).

And now we have bcachefs, too.

What’s the difference between them all? Honestly mostly licensing, developer energy, and maturity. ZFS has been around for ages and is the most mature. bcachefs is brand spanking new. BtrFS is in the middle. Technically speaking, all of them either do each other’s features or have each other’s features on their TODO list. LUKS in particular is still very commonly used because encryption is still missing in most (all?) of them, but will be done eventually.

OmnipotentEntity@beehaw.org on 08 Dec 2023 04:06 collapse

ZFS has encryption now, dunno about the rest

chitak166@lemmy.world on 08 Dec 2023 15:09 next collapse

I just use ext4.

MigratingtoLemmy@lemmy.world on 08 Dec 2023 17:23 collapse

Did BTRFS fix the Raid-5 problem or no?

manifesto7473@lemmy.ml on 10 Dec 2023 09:08 collapse

No, but according to this Phoronix article, they will fix the RAID56 issues soon:

The support for RAID56 is in development and will eventually fix the problems with the current implementation. This is a backward incompatible feature and has to be enabled at mkfs time.

MigratingtoLemmy@lemmy.world on 11 Dec 2023 09:09 collapse

That is taking a surprisingly long time TBH