I wrote an ebook on GNU awk with hundreds of examples and exercises
from learnbyexample@programming.dev to linux@lemmy.ml on 28 Mar 13:32
https://programming.dev/post/27665706

Hello!

I am pleased to announce a new version of my CLI text processing with GNU awk ebook. This book will dive deep into field processing, show examples for filtering features, multiple file processing, how to construct solutions that depend on multiple records, how to compare records and fields between two or more files, how to identify duplicates while maintaining input order and so on. Regular expressions will also be discussed in detail.

Book links

To celebrate the new release, you can download the PDF/EPUB versions for free till 06-April-2025.

Or, you can read it online at learnbyexample.github.io/learn_gnuawk/

Interactive TUI apps

Feedback

I would highly appreciate it if you’d let me know how you felt about this book. It could be anything from a simple thank you, pointing out a typo, mistakes in code snippets, which aspects of the book worked for you (or didn’t!) and so on.

Happy learning :)

#linux

threaded - newest

Jimbabwe@lemmy.world on 28 Mar 14:21 next collapse

Awesome! Thanks so much for doing this and sharing!

eldavi@lemmy.ml on 28 Mar 14:39 next collapse

i’m in awe everytime people do this.

i learned how to do this before code sharing sites like github existed and it forced me to turn everything i’ve learned into muscle memory and i think that this next generation of greybeards are going to be so much better than my generation’s greybeards because of it.

vfsh@lemmy.blahaj.zone on 29 Mar 04:13 collapse

I thought your comment was going in a totally different direction, it’s nice to hear appreciation of improved teaching methods instead of the old “well I figured it out myself so everyone else should too”

Magister@lemmy.world on 28 Mar 14:46 next collapse

I like awk, I started using it in the 90s in university, I’m not a pro, but this is so powerful!

swab148@lemm.ee on 28 Mar 14:57 next collapse

Finally, time to learn how to use awk! Sed, you’re next.

Enkers@sh.itjust.works on 28 Mar 15:25 next collapse

Could someone perhaps explain the major use cases or give a real life example of a time you’ve needed to use awk? I’ve been using Linux casually for quite a long time now, and although I learned the basics of the tool, I can’t recall having ever felt I had a need for it. If I want to glue a bunch of cli stuff together and need to do some text processing, it generally seems like it’d be easier to just use a simple python script.

Is it more for situations that need to be compatible with most *nix systems and you might not necessarily have access to a higher level scripting language?

learnbyexample@programming.dev on 28 Mar 15:37 next collapse

Well, if you are comfortable with Python scripts, there’s not much reason to switch to awk. Unless perhaps you are equating awk to Python as scripting languages instead of CLI usage (like grep, sed, cut, etc) as my ebook focuses on. For example, if you have space separated columns of data, awk ‘{print $2}’ will give you just the second column (no need to write a script when a simple one-liner will do). This of course also allows you to integrate with shell features (like globs).

As a practical example, I use awk to filter and process particular entries from financial data (which is in csv format). Just a case of easily arriving at a solution in a single line of code (which I then save it for future use).

balder1993@programming.dev on 28 Mar 15:46 collapse

Also AWK is made to be fast, right? I suppose doing something in CPython in a non efficient way might not be noticeable with a bit of text, but would show up with a large enough data stream.

commander@lemmy.world on 28 Mar 15:43 next collapse

I’m not expert in sed or awk. I always have to Google. For me though, it’s generally that you can do a great deal in just one line of awk or sed. They’re standard on any Linux distribution I’ve ever used. When building out pipelines, scripts that you want run from an installer you built post install and when removing, sed and awk rather than needing python.

All really nice when you have strict configuration management and versioning and there’s something deployed but it doesn’t have the python packages installed that would make it easy in python and you can’t just pip install it on hundreds+ of computers without going through a process of approval and building a new tagged version release but sed/awk/etc can do the job. If it’s hard enough, python and whatever packages you can install. If simple enough to do in a small bash script, no python just what’s standard in your Linux distro

N0x0n@lemmy.ml on 28 Mar 17:46 collapse

I’m not expert in sed or awk. I always have to Google. For me though, it’s generally that you can do a great deal in just one line of awk or sed.

Same here ! I recently used a one liner awk piped into sed, piped into another command to find duplicated lines and merge both files.

Writing a python script would have taken an unknown amount of time !

Danitos@reddthat.com on 28 Mar 16:04 next collapse

I manage some servers and awk can be useful to filter data. If you use commands like grep, and use the pipe operator (the " | " command), awk can be very handy.

Sure, a Python script can do that as well, but doing a one-liner in Bash is waaay faster to program.

AnAmericanPotato@programming.dev on 28 Mar 16:45 collapse

Is it more for situations that need to be compatible with most *nix systems and you might not necessarily have access to a higher level scripting language?

Yes, and also because integrating Python one-liners into shell pipelines is awkward in general. I’m more likely to write my entire script in Python than to use it just for text processing, and a lot of the time that’s just a pain. Python isn’t really designed for one-liners or for use as a shell. You can twist it into working in those use cases, but then I’d ask the reverse question: why would you do that when you could “just” use awk?

On macOS, Python is not installed by default. So if you are writing scripts that you want to be portable across platforms, or for general Mac administration, using Python is a burden.

This is also true when working with some embedded devices. IIRC I can ssh into my router and use awk (thanks to it being included in Busybox), but I’m definitely not going to install an entire Python environment there. I’m not sure there’d even be enough storage space for that.

silverhand@reddthat.com on 28 Mar 15:46 next collapse

Beautiful, thank you so much!

harsh3466@lemmy.ml on 28 Mar 19:14 next collapse

Perfect. I’m feeling comfortable enough with bash that next on my list is AWK. Gonna download this when I get home!

dino@discuss.tchncs.de on 29 Mar 12:47 collapse

You misstyped fish.

harsh3466@lemmy.ml on 29 Mar 13:14 collapse

Heh. I haven’t started exploring other shells yet.

tuckerm@feddit.online on 28 Mar 19:38 next collapse

Looking forward to reading it! awk has been a huge blind spot for me for a long time now.

jagged_circle@feddit.nl on 29 Mar 00:38 next collapse

Thanks but omg no. I’m happy to use awk like cut.

MonkderVierte@lemmy.ml on 29 Mar 12:31 next collapse

Btw, there’s asciidoctor-epub3.

starbrite@lemmy.zip on 02 Apr 04:28 collapse

Could you do grep next?