TheHobbyist@lemmy.zip
on 07 Jan 11:01
nextcollapse
Alternatively, you don’t even need podman or any containers, as open-webui can be installed simply using python/conda/pip, if you only care about serving yourself:
Yeah… No. What’s with the kids these days and shitty install scripts for Linux?
LainTrain@lemmy.dbzer0.com
on 07 Jan 12:58
nextcollapse
This is definitely not a “these days” thing, it’s been around for a while, it’s even less explicable now though that flatpak, snap, npm, conda, venvs and docker exist.
atzanteol@sh.itjust.works
on 07 Jan 13:46
collapse
This one happily modifies system repositories for you too which is “great”. This whole practice needs to stop.
LainTrain@lemmy.dbzer0.com
on 08 Jan 02:12
collapse
atzanteol@sh.itjust.works
on 07 Jan 15:29
collapse
I feel like it should be self evident - but I’ll outline a few of the specifics.
“Hey kids - just run this rando internet script directly without inspecting it first”
Firstly - you shouldn’t copy/paste directly into the terminal at all. What you copy and paste may not be what you see: example. So even “simple” things could turn bad. Paste to an editor first, then your terminal - especially if you’ve used “sudo” recently. It’s simply bad security hygiene to just run random code in a shell and to get people used to doing so.
Secondly - you’re just running some rando un-structured shell script. They can, and do, do whatever they want and put things wherever the developer felt they should go. It can re-configure your system in ways you may not want. In fact in this specific case it will add repositories to your system without asking. Did you want EPEL setup on your system? Did you want to add an external NVidia repo to your system? Too bad, it’s done. Hope you saw the “Installing NVIDIA repository…” message as it flew past because that’s all the notice you’ll get - and you only get that because the developers told you about it.
Thirdly - since these are completely unstructured there is no uninstall without sifting through the script (which you didn’t keep because it’s a “curl | sh”). Again lets use this one as an example:
How many things do you think that shell script installs?
It puts a binary in one of /usr/local/bin, /usr/bin or /bin
It then downloads a bunch of stuff to one of /usr/local/lib/ollama, /usr/lib/ollama or /lib/ollama.
It sets up external repositories to fetch nvidia dependencies
It proceeds to install dependencies from those external repos
It creates a service file for ollama and starts it.
It configures /etc/modules.d to load nvidia drivers
And that’s all I see on a cursory walk through the 300+ lines of script. All of that may be reasonable to get things working - but that’s a lot for you to find and undo if you wish to remove this later.
There are better ways to distribute software and handle dependencies.
You can grab that shell file and examine it. You can also check it out from their public git. I agree that this is bad practices, but not exactly uncommon.
atzanteol@sh.itjust.works
on 09 Jan 05:11
collapse
It being a bad practice and “not uncommon” is my complaint.
ollam runs on the 6700 XT, but you need to add an environment variable for it to work… I just don’t remember what it was and am away from my computer right now
dino@discuss.tchncs.de
on 07 Jan 13:35
nextcollapse
I tried it briefly, but its hot garbage if you dont have potent hardware.
The amount of iterations you have to do, to get proper answers and the time it takes to produce them is a waste of time.
theshatterstone54@feddit.uk
on 08 Jan 01:35
collapse
Personally I’d just recommend either Alpaca or GPT4All, both of which are on Flathub and much easier to set up (or at least GPT4All is; I haven’t tested Alpaca yet).
threaded - newest
Alternatively, you don’t even need podman or any containers, as open-webui can be installed simply using python/conda/pip, if you only care about serving yourself:
docs.openwebui.com/getting-started/quick-start/
Much easier to run and maintain IMO. Works wonderfully.
And llamafile is a binary you can just download and run, no installation required. “Uninstallation” is deleting the file.
You might instead just install the Alpaca flatpak. I found it a very easy and quick way to get started.
Yeah… No. What’s with the kids these days and shitty install scripts for Linux?
This is definitely not a “these days” thing, it’s been around for a while, it’s even less explicable now though that flatpak, snap, npm, conda, venvs and docker exist.
This one happily modifies system repositories for you too which is “great”. This whole practice needs to stop.
Fucking hell that’s awful
why no?
I feel like it should be self evident - but I’ll outline a few of the specifics.
“Hey kids - just run this rando internet script directly without inspecting it first”
Firstly - you shouldn’t copy/paste directly into the terminal at all. What you copy and paste may not be what you see: example. So even “simple” things could turn bad. Paste to an editor first, then your terminal - especially if you’ve used “sudo” recently. It’s simply bad security hygiene to just run random code in a shell and to get people used to doing so.
Secondly - you’re just running some rando un-structured shell script. They can, and do, do whatever they want and put things wherever the developer felt they should go. It can re-configure your system in ways you may not want. In fact in this specific case it will add repositories to your system without asking. Did you want EPEL setup on your system? Did you want to add an external NVidia repo to your system? Too bad, it’s done. Hope you saw the “Installing NVIDIA repository…” message as it flew past because that’s all the notice you’ll get - and you only get that because the developers told you about it.
Thirdly - since these are completely unstructured there is no uninstall without sifting through the script (which you didn’t keep because it’s a “curl | sh”). Again lets use this one as an example:
How many things do you think that shell script installs?
And that’s all I see on a cursory walk through the 300+ lines of script. All of that may be reasonable to get things working - but that’s a lot for you to find and undo if you wish to remove this later.
There are better ways to distribute software and handle dependencies.
Flawless comment, agreed with every word you said. This script is digital herpes
You can grab that shell file and examine it. You can also check it out from their public git. I agree that this is bad practices, but not exactly uncommon.
It being a bad practice and “not uncommon” is my complaint.
I still hate AI from an environmental and artistic point but I’m at least glad they didn’t mention AI ‘art’.
I did try to use it on Fedora but i have a Radeon 6700 XT and it only worked in the CPU. I wait until ROCM official support reaches my older Model.
I have the same setup, you have to add the line
Environment=“HSA_OVERRIDE_GFX_VERSION=10.3.0”for that specific GPU to the ollama.service fileollam runs on the 6700 XT, but you need to add an environment variable for it to work… I just don’t remember what it was and am away from my computer right now
I tried it briefly, but its hot garbage if you dont have potent hardware.
The amount of iterations you have to do, to get proper answers and the time it takes to produce them is a waste of time.
Personally I’d just recommend either Alpaca or GPT4All, both of which are on Flathub and much easier to set up (or at least GPT4All is; I haven’t tested Alpaca yet).
Alpaca is great, I can even run it on my oneplus 6t, albeit slowly and the max size I got running was llama 7b