zcat shouldn't error out if you try to zcat an uncompressed file, it should just output the damned file !
from interdimensionalmeme@lemmy.ml to linux@lemmy.ml on 18 Dec 07:47
https://lemmy.ml/post/23707648

There I said it !

#linux

threaded - newest

allywilson@lemmy.ml on 18 Dec 08:04 next collapse

Yeah, it’s a pain. Leads to bad one liners:

for i in $(ls); do zcat $i || cat $i; done

interdimensionalmeme@lemmy.ml on 18 Dec 08:16 next collapse

Thanks !

But still we shouldn’t have to resort to this !

Also, can’t get the output through pipe

for i in $(ls); do zcat $i || cat $i; done | grep mysearchterm

this appears to work

find . -type f -print0 | xargs -0 -I{} sh -c ‘zcat “{}” 2>/dev/null || cat “{}”’ | grep “mysearchterm”

Still, that was a speed bump that I guess everyone dealing with mass compressed log files has to figure out on the fly because zcat can’t read uncompressed files ! argg !!!

for i in $(ls); do zcat $i 2>/dev/null || cat $i; done | grep mysearchterm

MonkderVierte@lemmy.ml on 18 Dec 15:12 collapse

Btw, don’t parse ls. Use find |while read -r instead.

find -maxdepth 1 -name "term" -print |while read -r file
   do zcat "$file" 2>/dev/null || cat "$file"
done
allywilson@lemmy.ml on 18 Dec 19:14 next collapse

Won’t this cause cat to iterate through all files in the cwd once zcat encounters an issue, instead of just the specific file?

MonkderVierte@lemmy.ml on 18 Dec 22:08 next collapse

Yeah, i was tired and had $file there first, then saw that you wanted to cat all in directory. Still tired, but i think this works now.

LemoineFairclough@sh.itjust.works on 19 Dec 00:44 collapse

You are correct. This probably produces something more similar to what you’d want the original command to do, but with better safely:

find -- . -type f -regex '^\./[^/]*$' -exec sh -c -- 'for file in "${@}"; do zcat "${file}" || cat "${file}" || exit; done' sh '{}' '+'

That assumes you want to interact with files with names like .hidden.txt.gz though. If you don’t, and only intend to have a directory with regular files (as opposed to directories or symbolic links or other types of file), using this is much simpler and even safer, and avoids using files in a surprising order:

for i in *; do zcat -- "$i" || cat -- "$i" || exit; done

Of course, the real solution is to avoid using the Shell Command Language at all, and to carefully adapt any program to your particular problem as needed: sipb.mit.edu/doc/safe-shell/

gnuhaut@lemmy.ml on 19 Dec 13:36 collapse

You can just do for f in * (or other shell glob), unless you need find’s fancy search/filtering features.

The shell glob isn’t just simpler, but also more robust, because it works also when the filename contains a newline; find … | while read -r will crap out on that. Also apparently you want while IFS= read -r because otherwise read might trim whitespace.

If you want to avoid that problem with the newline and still use find, you can use find -exec or find -print0 … | xargs -0, or find -print0 … | while IFS= read -r -d ‘’. I think -print0 is not standard POSIX though.

MonkderVierte@lemmy.ml on 19 Dec 15:33 collapse

because it works also when the filename contains a newline

Doesn’t that depend on the shell?

gnuhaut@lemmy.ml on 19 Dec 15:48 collapse

I don’t think so and have never heard that, but I could be wrong.

elmicha@feddit.org on 18 Dec 08:18 next collapse

I agree. zgrep also works for uncompressed files, so we could use e.g. zgrep ^ instead of zcat.

interdimensionalmeme@lemmy.ml on 18 Dec 09:26 collapse

Thanks, didn’t know that existed

That’s basically everything I was looking for !

huf@hexbear.net on 18 Dec 09:39 next collapse

zgrep . * should do the trick

oh, there’s also zcat -f *

Morphit@feddit.uk on 19 Dec 16:58 collapse 

       -f --force
              If the input data is not in a format recognized by gzip, and if the option --stdout is also given,
              copy the input data without change to the standard output: let zcat behave as cat.

I don’t know why this isn’t the top comment. I guess there might be some scenario where you’d want to know about non-gzip files where you don’t expect them so changing the defaults would probably cause some subtle breakage. For shell use though, just an alias could be used; alias zcat=gzip -cdf

huf@hexbear.net on 19 Dec 19:08 collapse

in that case, i’d prefer a ~/bin/zcat with the contents

#!/bin/sh
exec gzip -cdf "$@"

this way, it’s exec’able, unlike an alias or shell function.

makingStuffForFun@lemmy.ml on 18 Dec 09:49 next collapse

Celeste. Are you here? In a future search maybe?

Treczoks@lemmy.world on 18 Dec 15:39 next collapse

Well, the source code is available. Fix it if you need it that bad.

fool@programming.dev on 18 Dec 16:27 next collapse

Man, I have a minor inconvenience.

installs Gentoo

interdimensionalmeme@lemmy.ml on 18 Dec 21:09 collapse

Where is it? I can’t seen to find it github.com/zCat?tab=repositories

SteveTech@programming.dev on 18 Dec 22:03 collapse

It’s part of GNU Gzip, and zcat is basically just a shell script that runs exec gzip -cd “$@” meaning you can actually just do cat /usr/bin/zcat to get the source.

Morphit@feddit.uk on 19 Dec 16:58 collapse

Or even zcat -f /usr/bin/zcat

Malfeasant@lemm.ee on 18 Dec 20:01 next collapse

How do you propose zcat tell the difference between an uncompressed file and a corrupted compressed file? Or are you saying if it doesn’t recognize it as compressed, just dump the source file regardless? Because that could be annoying.

interdimensionalmeme@lemmy.ml on 18 Dec 21:08 collapse

Even a corrupt compressed files has a very different structure relative to plain text. “file” already has the code to detect exactly which.

Still, failing on corrupted compression instead of failing on plaintext would be an improvement.

Malfeasant@lemm.ee on 19 Dec 19:25 collapse

What even is plain text anymore? If you mean ASCII, ok, but that leaves out a lot. Should it include a minimal utf-8 detector? Utf-16? The latest goofy encoding? Should zcat duplicate the functionality of file? Generally, unix-like commands do one thing, and do it well, combining multiple functions is frowned upon.

LemoineFairclough@sh.itjust.works on 19 Dec 01:04 collapse

I think that providing an exit status that is not 0 when zcat is used with an uncompressed file is useful. Though my opinion is less strong regarding whether it should write more text after an error occurred, it’s probably more useful for a process to terminate quickly when an error occurred rather than risk a second error occurring and making troubleshooting harder.

I think that trying to change any existing documented features of widely used utilities will lead to us having less useful software in the future (our time is probably better spent making new programs and new documentation): www.jwz.org/doc/worse-is-better.html en.wikipedia.org/wiki/Worse_is_better

interdimensionalmeme@lemmy.ml on 19 Dec 03:31 collapse

Not improving existing software leads to stagnation.

It’s certainly a good part of why so much of linux is an awkward kludgy idiosyncratic mess to use.

Whatever the first implementation does ends up being a suicide pact by default.

Another option is to change cat to auto decompress compressed files, instead of printing gibberish.

LemoineFairclough@sh.itjust.works on 19 Dec 07:53 collapse

What operating system should I use with my laptop that isn’t an awkward kludgy idiosyncratic mess? I would say that Windows has plenty of kludges, like having problems with certain file names. Many versions of macOS are UNIX® Certified Products (for example, macOS version 15.0 Sequoia on Intel-based Mac computers and on Apple silicon-based Mac computers), so it’s surely not any less kludgy than Linux.

I suppose that it’s not bad to change documentation to be more specific, and change a program such that it matches the new documentation and wouldn’t cause any harm if it replaced all the existing versions of the program, but makes it possible to use the program to solve more problems. That would be to “add functionality in a backward compatible manner”.

You are also free to create new programs that are not an exact replacement for existing programs, but can enable some people to stop using one or more other programs. That would not be what I describe as stagnation.

“The cat utility shall read files in sequence and shall write their contents to the standard output in the same sequence.”, so I would be very annoyed if it did something different with a certain file but not others. I wouldn’t say that the contents of a file and the contents after the file is expanded are the same. In fact, I expect that some people use cat to process compressed files, and changing how cat acts with compressed files would probably cause them a large amount of annoyance, and would needlessly make a lot of existing documentation incorrect.