Funny how this was one of the first tools I learnt once I “seriously” started my linux journey, lol

github.com/johnkerl/miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON

I use it occasionally but every time I need to do something a tiny bit more complex than “extract field from an object” I have to spend half an hour studying its manual, at which point it’s faster to just write a Python script doing exactly what I need it to do.

yq can do both JSON and YAML :)

I just started the process of switching from nginx Proxy Manager to Caddy yesterday, and even before setting up a single rule, I’m enjoying it more than NPM. Really wish I would have heard about it sooner!

I’d recommend tmux for that particular use. Screen has a lot of extras that are interesting but don’t really follow the GNU mentality of “do one thing and do it well.”

nohup is similar

There is also zellij, which can do the same but also has modern functionality specific for development workspaces!

(Although screen or tmux will still probably be more widely available on remote machines etc)

Woah screen is seeing active development again? There was like a decade where it stagnated. So much so that different distros were packaging different custom feature patches (IIRC only Ubuntu had a vertical split patch by default?) Looking at it now, the new screen maintainers had to skip a version to not conflict with forks that had become popular.

When tmux stabilized I jumped ship immediately and never looked back.

I know everyone likes tmux but screen is phenomenal. I have a .screenrc I deploy everywhere with a statusbar at the bottom, a set number of pre-defined tabs, and logging to a directory (which is cleaned up after 30 days) so I can go back and figure out what I did. Great tool.

I heard about helix from you and I’ve used it for a year and a half or so now, it’s by far the best editor I’ve used so far and I can definitely vouch for it

Could you explain them in more depth? I opened them and don’t know

Just commenting to give more love to helix. It’s my favorite “small quick edits” editor.

I’ve actually been testing with fish recently coming from zsh, though I might wait until 4.0 fully releases before I make a more conclusive decision to move or not.

With that said, I remember looking through omf themes and stumbled onto Starship that branched off one of the themes and really liked the concept.

Helix is great thanks

Once Helix gets plugin support and someone makes a Clojure REPL plugin as good as Conjure I am never touching vim again!

Do you have experience with either ranger, lf, or yazi? I’m wondering how broot compares. Big fan of file ranger, and this looks very similar.

From your example, I have a hard time inferring what is it doing.

@bamboo @mfat , DD, great tool. Utilize it so often, but it is powerful and dangerous. I always double, triple, quadruple check my target disks with multiple programs to avoid destroying my production workstation. Might be best if I just designated a RPi for the job. 😅

ddrescue (or gddrescue) is a great version if you have a sick drive. It’ll try to copy the good areas first then go back to hammer on the sick areas.

Not perfect as it doesn’t know about the file system so it tries to copy the entire surface, but generally a good tool.

You can destroy it all the same with cp or cat.

Wow. This might come in handy!

Great call! pv is deceptively powerful. Being able to see progress and rate limit a pipe is incredibly useful.

I’ve switched from using dd to using pv to write disk images to removable media.

Do you have to wear the fedora to run this command?

I love locate! I have a cronjob that updates the db every night, then I can just find a file without having to think about where it is

I immediately had flashbacks of diagnosing bad I/O performance on CentOS 5 servers. That was the week when I learned what updatedb is and why it was always running in the background (there was a lot of files)

-g is not documented, what does it do?

Note: this made me discover topless (SFW) and its Caveat section.

Also Ctrl+D to exit any shell and Ctrl+R for reverse searching your history!

Yes! I use less all the time, combine it with grep, etc.

cat <your file> | more

I’d like to interject for a moment. There is also a tool called bat that is just cat with extra features. It prints out and works just like cat, but when the contents get too big, it works like less. The is syntax highlighting and works with git.

It’s replaced my need for cat and less.

The first time I used wget I felt so awesome. I was grabbing some extra music files I think it was for Ur-Quan Masters.

I love httpie for hitting urls when i want to see the headers or body without downloading to a file eg testing an api

Cool. I’d never heard of this and I’ve used Debian for years.

Yeah, to this day vim still isn’t intuitive for me, so I just use nano as it’s either often included or simple to install on most Distros.

Unless a script is hardcoded for vim I haven’t had to use it.

It's for people to memorize hundreds of arcane shortcuts and shit so they can feel like a smug hacker and gloat over the rest of us using other editors and getting just as much done as they are.

Also for graybeards that haven't realized it's not 1985 anymore.

vim must be used for certain files??

I used nano when I started but now I am using vim for one year already. I’d recommend taking a few days where you only use vim and I think you will see why people like it. With a few motions you can be much faster than you would be in Nano.

You can change that by changing your editor global variable

You can change your hate to love by using vim

Wow you triggered a lot of vim users !

Maybe give micro a shot :) It’s nano but more sane defaults and comes with customization in mind.

Nano is hella confusing too. Since when is ^ = Ctrl?

And why dont they tell you that Ctrl+S Ctrl+C Ctrl+X works?

seems like you need to try micro. It’s like nano, but with more sensible standard keybinds imho, as well as syntax highlighting and global clipboard use.

igtfo<ESC>
:q!

vim isn’t required for any files, you just followed online tutorials for how to edit those files instead of RTFM

terminal text editing is convoluted because it has to strike a balance between figuring out when a keypress is part of the text you’re typing, vs when it’s a command you’re using, and making sure that all the editor commands the designer wanted are accessible.

vim is great because it allows for thousands more editing commands and macros, and much more customization of the editor, up to allowing plugins that emulate other functionality. As it stands, my setup basically functions as a full, lightweight-ish, multi-language IDE that rivals Emacs or Visual Studio.

On top of all that, I don’t have to move my hands away from the homerow of keys to navigate or edit, which may not seem like much, but adds up to a lot of avoid typos and time saved from moving my hands to reach the arrows/delete/home/end/pgup/pgdn.

Some examples:

h, j,k,l move left, down, up, and right respectively, but they can be combined with a number to move that many rows or columns; e.g. 6j will move down 6 rows

dd deletes a line, but using a number + d + a movement will delete that many characters/lines in the path of the cursor: e.g. 34dl will delete 34 characters to the right of the cursor, 12dk will delete 12 lines up.

gg will take you to the first line, G will take you to the last, and number + either will take you to that line: e.g. 3275gg or 3275G will take you to line 3275

and finally you can use /text or regex pattern you want to search for and Enter to search the document for the first occurence below your current location, and then use n to search for the next occurence, or N to search for the previous

That doesn’t even scratch the surface (that’s just the cheatsheet, which only scratches the surface), but if you can get a handle on only what I’ve said, and switching between input and command mode (i and Esc respectively), the speedup to navigation alone will make it seem more sensible.

And as always, don’t forget to :wq (write to file and quit)

.

Also works on Twitch with the added benefit of NOT playing ads (you still get breaks, just with a placeholder screen instead of the commercial).

mpv has yt-dlp support built in, so it can just play the streams directly.

Since everyone keeps mentioning yt-dlp I gotta ask: what’s wrong with the original youtube-dl? I keep using it, it works, it’s still being updated.

The huge list of sites can be found here github.com/yt-dlp/yt-dlp/tree/master/…/extractor

It also supports ripping playlists. Fantastic to archive a set locally…

awk and sed have always been intimidating for me with that cryptic syntax.

Just be careful with files with spaces in the name. There’s an incantation with xargs that I always have to look up when I want to use it safely.

I loved glances until I switched to btop with Hot Purple Traffic Light theme, and I doubt I’ll switch again. Check it out!

This looks very cool. Thanks for sharing.

Relevant xkcd

I can never get grep to work consistently on Mac and Linux. Now, ripgrep OTOH…

Check out my chapter on GNU grep BRE/ERE for those wanting to learn this regex flavor: …github.io/…/breere-regular-expressions.html (there’s also another chapter for PCRE)

I love flexibility with regex, personally I use ugrep as it also allows utilization of boolean and/or/not logic for more complicated searches.

what is the difference between eza and lsd? I switched to eza a while back and i havent looked at any other ls replacement.

Love LSD. I always install it first with btop and zoxide

So just clone a git and delete the .git folder? Seems pretty useless to me.

tmux is my religion

Tmux is so much better than screen, and yes that is the hill I will die on

Specially when confined with tmuxp , it’s how I handle Game servers that can run headless to start at boot without losing access to giving commands to the server via its server console

Why not just use python as a calculator then?

from math import *
x = sin(12) + pi * 3.2
y = tan(x)
y
# prints 0.09200389785419612

FFMpeg Simple and underrated? Not sure about that.

similar to ImageMagick, jpegtran is great for lossless jpeg transformations. You can even extend jpegs using the “crop” function, which can be very useful for batch-processing images, even though it’s hardcoded to middle grey.

web.archive.org/web/…/jpeg-lossless-extend.html

That’s fantastic, I can’t wait to go home and install it

Game changer!

If you’d map it to just f it’s even more handy

I use fuckit to fix exceptions

That sounds dangerous. I hope it never tries to fix anything with rm

this just reminds me of please which runs the previous command with sudo

sudo !!

Holy shit I was just talking about cp with progress today. Awesome

Lazygit perfectly fits my lifestyle:) thanks for sharing

What is it?

Also useful in this regard, python comes with a sìmple file server built in, python -m http.server --directory /dir/ would serve /dir/ on port 8000.

I’ve used it in high-throughput production environments to do things that netcats can’t.

See also github.com/TomWright/dasel

Can it handle a file that has corrupt json? Or does it just tell you “no”

I used jq for something similar before, recently I’ve discovered Nu Shell and have been using that for converting and analyzing data since a full shell is a lot more powerful than a command (e.g. open a yaml, for each element on key X grab the first element of list Y and export to a CSV)

simple

What is AIX

We install dnf on our AIX boxes. Works okayish.

Don’t run killall on aix before reading the man page!

github.com/BurntSushi/ripgrep

Cool cal trick, it knows about the Gregorian calendar reform: cal 9 1752

Very useful for having a quick glance at calendar. Although it’s annoying that there’s no option to have week start on monday.

My fav for the past few years is github.com/ClementTsang/bottom

Check out BTOP++ – it blows htop out of the water.

Bashtop if you want something more graphical

OpenDoas, or simply doas, as in do [command] as [user]

permit persist setenv {PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin} :wheel

Why should I use a sudo alternative?

task-spooler (ts aka tsp)

This looks amazing. Do you know how well it works in dispatching and tracking jobs over remote servers (over SSH)?

You can do zoxide path2 data

I usually would just do z 2data. Yes, I’m lazy. It’s the perfect tool for lazy people.

Better than fasd?

Sounds a lot like autojump

y

yes no maybe

🖕😤🖕

Hey, still the quickest way to copy any damn text from a terminal elsewhere is with the mouse. Tmux is a joke for this.

Why can’t terminal emulators just make this easy

Gripes:

• starship and all these shell frameworks are overbloated. Just write your own prompt command and be done with it.

• restic, ongoing issue with the author to allow people to backup without a password. Seems like a no-brainer but he’s being difficult

Wow, I might try learning this over the next few days. Nice replacement for tidyverse on the cli

It can also format minimized JSON from cURL API requests

Combine with jc to process CSV files. This is how I get data into my plain text accounting system.

jq and yq are both things I install on pretty much every machine I have.

I love jq, but I wouldn’t call it “surprising simple” for anything but pretty-formatting json. It has a fairly steep learning curve for doing anything with all but the simplest operations on the simplest data structures.

Lftp is fast. Those parallel chunked downloads are not a joke

The terminal-based file browser space is so filled today but for my part I love what vifm has done for the dual-pane midnight commander concept - it’s the same basic idea, uses (somewhat) vim-like bindings by default and is super extensible.

Great call out! I first used ftp about 30 years ago, and lftp has been my go to for about the last decade. I rarely need it anymore, but I still use it for quickly transferring files with my homebrew switch.

What’s the syntax here? Do I go

command && yes

I’m not sure if I’ve had a use case for it, but it’s interesting.

yes | command-that-asks-a-lot-of-questions

yes > /dev/null

That’s really neat but also seems like it could be quite dangerous in a lot of use-cases!

Also, you can make yes return anything:

yes no

Very useful for shell scripts that need to do maths as well. I use it to make percentages when stdout has values between 0.0 and 1.0

It has not taken over NGINX and Apache yet.

Came here for the pipe.

i just cat grep .bash_history lol

but this does sound more convenient xD

So did I until reading this post. Thanks!

Works much better with fzf, but even just default bash it’s useful.

dude

wait, shell could make matrix multiplication ?

I like batcat aka bat, but nominate the humble ‘cat’ instead.

Want to copy a disk image to a device? You can use cat for that: cat file.iso>/dev/sdf

What to copy local stdout over ssh? Use cat.

ls -l | ssh myhost ‘cat >out.txt’

That’s simple and surprisingly powerful.

When you’re ready for a full monitoring package, check out Frigate.

Very true. I used to do magic with xargs when working as a sysadm. Also a good way to mess up on a grand scale. Ask me how I know.

netstat is kind of deprecated, ss is more modern (from the iproute2 package) and uses very similar parameters.

Converts any rich text format to any other.