EXT4 Has A Very Nice Performance Optimization For Linux 6.11 (Phoronix) (www.phoronix.com)
from thingsiplay@beehaw.org to linux@lemmy.ml on 18 Jul 2024 14:46
https://beehaw.org/post/15059028

Ted Ts’o sent out the EXT4 updates today for Linux 6.11. He explained in that pull request:

“Many cleanups and bug fixes in ext4, especially for the fast commit feature. Also some performance improvements; in particular, improving IOPS and throughput on fast devices running Async Direct I/O by up to 20% by optimizing jbd2_transaction_committed().”

#linux

threaded - newest

autotldr@lemmings.world on 18 Jul 2024 14:50 next collapse

This is the best summary I could come up with:


With the maturity of the EXT4 file-system it’s not too often seeing any huge feature additions for this commonly used Linux file-system but there’s still the occasional wild performance optimization to uncover… With Linux 6.11 the EXT4 file-system can see upwards of a 20% performance boost in some scenarios.

Ted Ts’o sent out the EXT4 updates today for Linux 6.11.

He explained in that pull request: "Many cleanups and bug fixes in ext4, especially for the fast commit feature.

Up to 20% faster for fast devices using async direct I/O thanks to JBD2 optimizations.

Indeed the patch from Huawei’s Zhang Yi to speed up jbd2_transaction_committed() shows off some great improvements:

It’s great continuing to see EXT4 uncover new performance optimizations.


The original article contains 144 words, the summary contains 120 words. Saved 17%. I’m a bot and I’m open source!

GolfNovemberUniform@lemmy.ml on 18 Jul 2024 14:57 next collapse

Wow my favorite FS is still being developed. Nice

thingsiplay@beehaw.org on 18 Jul 2024 15:06 next collapse

Mine too. I could not bring myself to adopt a new or different FS at the moment. I wonder what “fast devices” and “slow” in this context means.

entropicdrift@lemmy.sdf.org on 18 Jul 2024 17:21 collapse

NVME SSDs vs HDDs, perhaps?

thingsiplay@beehaw.org on 18 Jul 2024 17:45 collapse

Might be. It’s just guess work. My guess was maybe NVME SSD vs SATA SSD. Or maybe they refer to the CPU+RAM too.

boredsquirrel@slrpnk.net on 18 Jul 2024 15:45 next collapse

Fedora people would say that BTRFS is better because it allows maintenance that EXT4 doesnt even have :)

GolfNovemberUniform@lemmy.ml on 18 Jul 2024 15:51 next collapse

I dislike Fedora anyways.

[deleted] on 18 Jul 2024 16:25 next collapse

.

GolfNovemberUniform@lemmy.ml on 18 Jul 2024 16:37 collapse

Reported

boredsquirrel@slrpnk.net on 18 Jul 2024 18:18 collapse

Haha okay fine if you want to XD I just made aware that such a comment is totally useless.

GolfNovemberUniform@lemmy.ml on 18 Jul 2024 20:14 collapse

And it is totally useless indeed but tbh I don’t think it’s necessarily bad to make useless comments if it’s not a user asking for help.

bsergay@discuss.online on 18 Jul 2024 20:29 collapse

Why do you dislike Fedora?

I’m genuinely curious.

Samueru@lemmy.ml on 19 Jul 2024 01:37 next collapse

If you want my answer, this video sums it up pretty good: www.youtube.com/watch?v=x9qCqRTEVz0

More recently fedora pulled this move which causes headaches to everyone: gitlab.com/gnuwget/wget2/-/issues/661

To this day I notice that there is some skepticism with Btrfs, and I think it is because fedora also pushed it early.

bsergay@discuss.online on 19 Jul 2024 05:58 collapse

Thank you for sharing your thoughts!

Fedora’s tendency to default to (potentially) premature software, can definitely be a legit reason to prefer other distros instead.

I’m a “(sweet) summer child” in that I’ve only been using Fedora for over two years now. Therefore, I haven’t experienced the commonly cited ‘shifts’ that have caused many issues to other users. Regardless, I do (somewhat) understand.

Regarding wget2, I didn’t even know that was a thing. Thank you for mentioning it! I have yet to understand why or how Fedora unanimously agreed to push that change.

To this day I notice that there is some skepticism with Btrfs, and I think it is because fedora also pushed it early.

This, however, I can’t agree with. And perhaps you’re conflating matters. Btrfs was not ready when it was first supported. However, Fedora was not an early adopter. They only defaulted to it in 2020. By contrast, AFAIK openSUSE was the first to default it in 2014. Heck, the next year it was defaulted by SLE as well. By the time Fedora did the same, the severe issues and instabilities were already ironed out. So, I’d attribute the scepticism towards Btrfs as the community’s PTSD after many community members lost valuable data early in Btrfs’ lifetime.

GolfNovemberUniform@lemmy.ml on 19 Jul 2024 05:09 collapse

Extremely slow package manager (the most important one), confusing installer, fast deprecation of important technologies and testing of new technologies on its users (making major upgrades risky) is what I can remember now.

bsergay@discuss.online on 19 Jul 2024 06:18 collapse

Thank you for the reply!

Extremely slow package manager (the most important one)

Fair. Though, IIRC, it’s in the same order of magnitude as apt and zypper. But yeah; apk, pacman and xbps are definitely faster by a wide margin. Hopefully, dnf5 will be able to close the gap significantly.

confusing installer

I often hear this. But I’m not sure if I understand. Is it because Anaconda does not walk you (explicitly) through all parts of the installation (at least by default)? And, instead, chooses to give the user an overview (at some point) in which the user is expected to go over each one of them by themselves.

fast deprecation of important technologies and testing of new technologies on its users (making major upgrades risky)

Fair. I think this is the most legitimate concern. Thankfully, over the last two years, I have yet to bang my head against a brick wall for reasons related to this. But I understand why others are more reluctant based on Fedora’s (less recent) track record.

GolfNovemberUniform@lemmy.ml on 19 Jul 2024 06:53 collapse

apt

Apt can be improved with frontends and it doesn’t take 10 minutes to sync the repos.

zypper

Idk much about it but I heard it’s slow too.

But I’m not sure if I understand. Is it because Anaconda does not walk you (explicitly) through all parts of the installation (at least by default)?

Yes.

And, instead, chooses to give the user an overview (at some point) in which the user is expected to go over each one of them by themselves.

Yea even archinstall might be better than this design lol.

less recent

X11. Though I don’t remember if they decided to drop it before explicit sync was introduced for NVidia drivers or after.

bsergay@discuss.online on 19 Jul 2024 07:06 collapse

Apt can be improved with frontends

nala is indeed pretty cool.


Thank you for clarifying/confirming the parts related to how Fedora’s installation is confusing.

X11. Though I don’t remember if they decided to drop it before explicit sync was introduced for NVidia drivers or after.

Totally forgot about this one. Blame AMD 😛. Thank you for correcting me!

thingsiplay@beehaw.org on 18 Jul 2024 16:02 next collapse

I really like the idea of BTRFS and what it can do. For my recent system, build in end of 2023 (not a year ago) I really thought about and compared the systems, but end up using EXT4. Here some thoughts I had:

I want to use BTRFS as my main system FS, but I wasn’t sure which alternative FS to use (there are other contenders too), if I need the extra functionality, if its 100% stable for me on a non Fedora system and I also did not want to spent the time learning and experimenting with it, yet. But I will. And if other distributions I install or boot into would work well with BTRFS, if they are not on the newest Kernel yet.

drwho@beehaw.org on 18 Jul 2024 16:19 next collapse

If you’re interested I have a fairly thorough “I use this” post on my website (last time I updated it was in early 2023) about btrfs.

thingsiplay@beehaw.org on 18 Jul 2024 16:25 collapse

Sure! I’m interested into the “current” state or real world experience of it. Wouldn’t mind if you post it here. Although I am not sure how relevant it is 1 year later, because the filesystem is quite under development.

drwho@beehaw.org on 18 Jul 2024 17:55 collapse

Here you go: …virtadpt.net/…/experimenting-with-btrfs-in-produ…

That its under development means that it’s being maintained. EXT4 is still being developed, so is xfs. And the other ones that are fairly popular, for that matter.

thingsiplay@beehaw.org on 18 Jul 2024 18:05 collapse

Thanks. But it’s important to note your experience report is based on the experience of 2019 and the slight edits aren’t changing that. That its being developed is not the same as under maintenance. EXT4 is fully developed and there are only optimizations in performance expected, if anything, while BTRFS still needs active development to improve compatibility and some other features.

I’m still curious to how to work with it and such a report is still welcome. I’ll give it a read. Edit: Hopefully my reply didn’t sound too negative. I’m interested in the process of going all of this, so the article is useful in a practical sense.

drwho@beehaw.org on 23 Jul 2024 16:49 next collapse

My experience is based on running that btrfs array since 2019. It’s still running in production on my server, I still use it daily, and the data I keep on it is still accessed, processed, indexed, and backed up every day. It’s not an experiment for the sake of a blog post, it’s a thing that is part of my personal infrastructure. The reason I update that post periodically is because I learn something new, or something minor has changed and the text should be updated to reflect that. If using btrfs on a busy server every day is the experience of 2019, I don’t know what to say to that.

thingsiplay@beehaw.org on 23 Jul 2024 20:40 collapse

Oh that’s hugely different then. It was not apparent that you were using this setup since the initial blog post. Maybe make a note at the top of the post, so your message (like here) is understood. Don’t get me wrong, I appreciate the clear information you give here and your feedback on how this works. From your report and given its the default FS for Fedora, its absolutely clear that I need to review my bias and view as BTRFS as a contender for my next system drive.

drwho@beehaw.org on 24 Jul 2024 16:31 collapse

I had intended that the dates on the edits would have suggested otherwise (the last was 20230422), but I also get how easy it is to miss them if you’re looking for something specific. I can’t change the publication datestamp because that’s part of the slug, and it would break links both internal and any that are external.

soundconjurer@mstdn.social on 18 Jul 2024 18:07 collapse

@thingsiplay @drwho , as soon as RAID5/6 is fully ready (and I am aware it looks like it'll never be), I'll be switching over to it.

drwho@beehaw.org on 23 Jul 2024 16:52 collapse

What’s your use case? Do you need the storage capacity and parity of RAID-5 or -6? The write performance?

soundconjurer@mstdn.social on 23 Jul 2024 16:54 collapse

@drwho I have a data server with data I hold dear and want to ensure losing drives (using RAID6) won't lead to me losing my data.

drwho@beehaw.org on 23 Jul 2024 18:12 collapse

That’s entirely fair.

narc0tic_bird@lemm.ee on 18 Jul 2024 16:30 collapse

I’m not quite sure why people are still worried about the stability of btrfs when it has been rock solid for years. Synology has been using it for quite a while now in their NAS systems, they surely wouldn’t if it’d mean a lot of customers were at risk of losing their data.

There are valid reasons not to be using btrfs (although I’d argue most ordinary use cases don’t have a valid reason), but stability certainly isn’t one of them, independent of the distribution used (unless it’s ancient).

ArchAengelus@lemmy.dbzer0.com on 18 Jul 2024 17:02 next collapse

Wait til your table with all the checksums gets messed up on an “older” btrfs install. Happened to me on a VM because I didn’t know copy-on-write should be disabled for large frequently partially updated files. It also slowed that VMs IO down a lot.

Like most file systems, BTRFS is great if you know the edge cases. I recently moved to ZFS on my new work system, which has been a great change in terms of in-line snapshots and the like.

If EXT4 meets your needs, that’s awesome. If you understand how to use a different FS well or are willing to learn (and risk), I would also encourage other options as well.

thingsiplay@beehaw.org on 18 Jul 2024 17:06 next collapse

Note my research of BTRFS is almost a year old now. And there was still a few headlines making round of problems with BTRFS in some cases. A controlled NAS system is not the same as random user configuration of a random desktop user. And as said in my comment, I was not sure if it would be stable for my installation (when I did my research) and did not claim it to be unstable. On the other side, I know for a fact that EXT4 is stable and I did not research more or experiment to find out which one is better for me.

Current state of BTRFS: btrfs.readthedocs.io/en/latest/Status.html (note when I did my research, Linux was at Kernel v6.4, therefore BTRFS was in a worse situation than today)

leopold@lemmy.kde.social on 19 Jul 2024 02:59 collapse

Because of widespread fearmongering, itself caused by the filesystem taking too long to become stable and garnering a bad reputation as a result which it has never shaken off.

narc0tic_bird@lemm.ee on 18 Jul 2024 16:21 next collapse

Tumbleweed people like me would say it’s a great filesystem because it enables snapper to work effortlessly.

Swedneck@discuss.tchncs.de on 12 Aug 2024 10:07 collapse

i’m using fedora on ext4 lol, the single unavoidable problem with BTRFS is that i’d have to switch to it :P

teawrecks@sopuli.xyz on 19 Jul 2024 01:49 collapse

I understand ext4 being dependable, but favorite? I’m curious what makes it your favorite.

laurelraven@lemmy.blahaj.zone on 19 Jul 2024 02:16 next collapse

Is dependable not a good enough reason to be someone’s favorite?

teawrecks@sopuli.xyz on 19 Jul 2024 16:09 collapse

That’s fine, I just consider that the absolute bare minimum for a FS to be usable.

GolfNovemberUniform@lemmy.ml on 19 Jul 2024 05:11 collapse

It’s stable and well refined. Imo these are the most important things for a file system.

teawrecks@sopuli.xyz on 19 Jul 2024 16:11 collapse

Sure, yeah, I would say you should only trust stable, well refined filesystems as a daily driver. Which is why I was curious what set ext4 apart from the rest for you.

lemmyreader@lemmy.ml on 18 Jul 2024 18:57 next collapse

Huge news 🎉 Thanks OP for sharing.

It feels like a relief after reading earlier Lemmy comments in other posts about btrfs vs ext4 and having read this Wikipedia page paragraph :

In 2008, the principal developer of the ext3 and ext4 file systems, Theodore Ts’o, stated that although ext4 has improved features, it is not a major advance, it uses old technology, and is a stop-gap. Ts’o believes that Btrfs is the better direction because “it offers improvements in scalability, reliability, and ease of management”.[29] Btrfs also has “a number of the same design ideas that reiser3/4 had”.[30] 😢

Oh no, wait a minute, I overlooked the next sentence last time 😀 :

However, ext4 has continued to gain new features such as file encryption and metadata checksums.

beejjorgensen@lemmy.sdf.org on 18 Jul 2024 19:46 collapse

On the last system I put together I used xfs because I was thinking ext4 development was waning. TBH I can’t really tell the difference in my regular usage.

Word on the street is that xfs sometimes corrupts files, but I’m not sure if that’s true anymore.

Maybe on the next system I’ll be back to ext4.

possiblylinux127@lemmy.zip on 18 Jul 2024 23:38 collapse

Btrfs is newer and has more features

possiblylinux127@lemmy.zip on 18 Jul 2024 23:37 next collapse

For those who still use it.

Ok I’m just giving it a hard time as I still use it for VMs

thingsiplay@beehaw.org on 19 Jul 2024 00:20 next collapse

I still use it (Ext4) exclusively and its great. How can you have a hard time with Ext4? It’s the most proven and most polished FS. Its not like slowing down your system or being buggy or like that.

Samueru@lemmy.ml on 19 Jul 2024 01:34 next collapse

Btrfs snapshots made me stop using ext4 all together.

shirro@aussie.zone on 19 Jul 2024 07:40 next collapse

Same here. Ext4 is an excellent general purpose file systems and a sensible default. It lacks features that are useful, even critical, for some use cases which sometimes rules it out but it certainly isn’t obsolete.

possiblylinux127@lemmy.zip on 19 Jul 2024 11:22 collapse

It is slower than btrfs and lacks protection against corruption

bsergay@discuss.online on 19 Jul 2024 11:51 collapse

It is slower than btrfs

What evidence supports your statement, given that it contradicts the overall results of tests conducted in the past year by both DJ Ware and Michael Larabel from Phoronix?

PrivateNoob@sopuli.xyz on 19 Jul 2024 05:46 collapse

Well that’s the default for debian based systems. So a lot of us use it.

peppy@lemmy.ml on 19 Jul 2024 05:42 next collapse

ext4 is better than btrfs in terms of speed right?

addie@feddit.uk on 19 Jul 2024 07:01 collapse

Yes, because it doesn’t do as much to protect you from data corruption.

If you have a use case where a barely-measurable increase in speed is essential, but not so essential that you wouldn’t just pay for more RAM to keep it in cache, and also it doesn’t matter if you get the wrong answer because you’ve not noticed the disk is failing, and you can afford to lose everything in the case of a power cut, then sure, use a legacy filesystem. Otherwise, use a modern one.

thingsiplay@beehaw.org on 19 Jul 2024 07:29 next collapse

Ext4 is not legacy, just because something newer is out there. Ext4 is proven and rock solid, not without reason the standard for most Linux systems. It doesn’t randomly corrupt your files. If someone would read your reply, one would think that Ext4 is abandoned since decades and a risk to use.

If one has to ask and don’t understand Btrfs, should just use Ext4 by default; a safe and good option without risking anything. There are no downsides to this. Use Btrfs only, if you know what you are doing, if you understand it and actually need the extra functionality.

bsergay@discuss.online on 19 Jul 2024 07:44 collapse

Use Btrfs only, if you know what you are doing, if you understand it and actually need the extra functionality.

And, may I add, if your chosen distro defaults to it.

jeremias@social.jears.at on 19 Jul 2024 11:32 collapse

and you can afford to lose everything in the case of a power cut

But ext4 is a journaling filesystem, so a power cut shouldn’t harm it.

[deleted] on 19 Jul 2024 07:17 next collapse

.

LeFantome@programming.dev on 21 Jul 2024 04:21 collapse

I notice that this is from Huawei. If it was commercial software, this could not be used by anybody using US federal funding. Because this is Open Source, it will probably be put into use by the NSA, CIA, FBI, and NASA. Fascinating.

dragonfly4933@lemmy.dbzer0.com on 22 Jul 2024 06:49 collapse

There is a lot of development from China in the linux kernel. Also, to my knowledge there is a lot of chinese work in qemu and libvirt as well.