autotldr@lemmings.world
on 12 Feb 2024 15:35
nextcollapse
This is the best summary I could come up with:
While there have been efforts by AMD over the years to make it easier to port codebases targeting NVIDIA’s CUDA API to run atop HIP/ROCm, it still requires work on the part of developers.
The tooling has improved such as with HIPIFY to help in auto-generating but it isn’t any simple, instant, and guaranteed solution – especially if striving for optimal performance.
In practice for many real-world workloads, it’s a solution for end-users to run CUDA-enabled software without any developer intervention.
Here is more information on this “skunkworks” project that is now available as open-source along with some of my own testing and performance benchmarks of this CUDA implementation built for Radeon GPUs.
For reasons unknown to me, AMD decided this year to discontinue funding the effort and not release it as any software product.
Andrzej Janik reached out and provided access to the new ZLUDA implementation for AMD ROCm to allow me to test it out and benchmark it in advance of today’s planned public announcement.
The original article contains 617 words, the summary contains 167 words. Saved 73%. I’m a bot and I’m open source!
kbal@kbin.melroy.org
on 12 Feb 2024 15:43
nextcollapse
For reasons unknown to me, AMD decided this year to discontinue funding the effort
Presumably they did not want to see Cuda becoming the final de-facto standard that everyone uses. It nearly did at one point a couple of years ago, despite the lack of openness and lack of AMD hardware support.
verdigris@lemmy.ml
on 12 Feb 2024 16:23
nextcollapse
They stopped funding the replacement, not CUDA.
kbal@kbin.melroy.org
on 12 Feb 2024 16:43
nextcollapse
That is contradicted by the headline. This easy confusion between CUDA (the API) and CUDA (the proprietary software package that is one implementation of it) illustrates the problem with CUDA.
ZLUDA seems to be an effort to fix that problem, but I don't know what it's chances of success might be.
By funding an API-compatible product, they are giving CUDA legitimacy as a common API. I can absolutely understand AMD not wanting a competitors invention and walled-off product to be anything resembling an industry standard.
conciselyverbose@kbin.social
on 12 Feb 2024 23:13
collapse
It already has legitimacy. It's their hardware that doesn't, despite the decent raw flops and high memory.
RandomLegend@lemmy.dbzer0.com
on 12 Feb 2024 16:32
nextcollapse
i heavily rely on CUDA for many things i do on my personal computer. If this establishes itself as a reliable method to use all the funky CUDA stuff on AMD cards, my next card will 100% be AMD.
TropicalDingdong@lemmy.world
on 12 Feb 2024 19:49
collapse
i heavily rely on CUDA for many things i do on my personal computer. If this establishes itself as a reliable method to use all the funky CUDA stuff on AMD cards, my next card will 100% be AMD.
If there were a drop in equivalent to CUDA with AMD, I’d have several AMD cards, right now.
ExtremeDullard@lemmy.sdf.org
on 12 Feb 2024 16:40
nextcollapse
All I need to know about AMD is this:
Whenever amdgpu.ko is insmoded, the display (or the system itself in some cases) is unstable. When it works great now, one day it won’t and the machine will inexplicably start crashing randomly or displaying garbage after an update.
This has held true for years for me on many machines I’ve installed Linux on, and it still does: not a week ago, I updated a laptop with a Renoir chipset in it (RX Vega 6) that had been stable for years, and now the display gets corrupted whenever I switch VT. Because amdgpu…
Not bashing on AMD or Nvidia. This has just been my reality. As a result, whenever I have a choice, I go with Intel graphics because it never causes me as much of a headache.
GenderNeutralBro@lemmy.sdf.org
on 12 Feb 2024 17:07
nextcollapse
Those benchmarks are impressive indeed. I certainly care much more about having extra VRAM than a little extra speed, comparing the 7900 XT or 7900 XTX to the RTX 4080.
I’d love to see some benchmarks for various LLMs and image generators.
Since when does ROCm require kernel modules? DRI exists?
chameleon@kbin.social
on 12 Feb 2024 18:47
collapse
Technically always has, ROCm comes with a "backported" amdgpu module and that's the one they supposedly test/officially validate with. It mostly exists for the ancient kernels shipped with old long-time support distros.
Of course, ROCM being ROCM, nobody is running an officially supported configuration anyway and the thing is never going to work to an suitably acceptable level. This won't change that, since it's still built on top of it.
cbarrick@lemmy.world
on 12 Feb 2024 20:13
nextcollapse
After two years of development and some deliberation, AMD decided that there is no business case for running CUDA applications on AMD GPUs. One of the terms of my contract with AMD was that if AMD did not find it fit for further development, I could release it. Which brings us to today.
mindbleach@sh.itjust.works
on 12 Feb 2024 20:38
nextcollapse
Now we’ll see if Nvidia drops CUDA immediately, or waits until next quarter.
LarmyOfLone@lemm.ee
on 12 Feb 2024 22:28
nextcollapse
Do LLM or that AI image stuff run on CUDA?
UraniumBlazer@lemm.ee
on 12 Feb 2024 23:40
nextcollapse
Cuda is required to be able to interface with Nvidia GPUs. AI stuff almost always requires GPUs for the best performance.
MalReynolds@slrpnk.net
on 13 Feb 2024 01:53
nextcollapse
Yes, llama.cpp and derivates, stable diffusion, they also run on ROCm. LLM fine-tuning is CUDA as well, ROCm implementations not so much for this, but coming along.
brianorca@lemmy.world
on 13 Feb 2024 04:45
nextcollapse
Nearly all such software support CUDA, (which up to now was Nvidia only) and some also support AMD through ROCm, DirectML, ONNX, or some other means, but CUDA is most common. This will open up more of those to users with AMD hardware.
Thanks that is what I was curious about. So good news!
redcalcium@lemmy.institute
on 13 Feb 2024 08:38
collapse
They are usually released for CUDA first, and if the projects got popular enough, someone will come in and port them to other platforms, which can take a while especially for rocm. Apple m series ports usually appear first before rocm, that’s show how much the devs community dislike working with rocm with famous examples such as geohot throwing the towel after working with rocm for a while.
as pointless as it sounds it would be a great way to test the system and call alternative implementations of each proprietary Nvidia library. It would also be great for debugging and development to provide an API for switching implementations at runtime.
threaded - newest
This is the best summary I could come up with:
While there have been efforts by AMD over the years to make it easier to port codebases targeting NVIDIA’s CUDA API to run atop HIP/ROCm, it still requires work on the part of developers.
The tooling has improved such as with HIPIFY to help in auto-generating but it isn’t any simple, instant, and guaranteed solution – especially if striving for optimal performance.
In practice for many real-world workloads, it’s a solution for end-users to run CUDA-enabled software without any developer intervention.
Here is more information on this “skunkworks” project that is now available as open-source along with some of my own testing and performance benchmarks of this CUDA implementation built for Radeon GPUs.
For reasons unknown to me, AMD decided this year to discontinue funding the effort and not release it as any software product.
Andrzej Janik reached out and provided access to the new ZLUDA implementation for AMD ROCm to allow me to test it out and benchmark it in advance of today’s planned public announcement.
The original article contains 617 words, the summary contains 167 words. Saved 73%. I’m a bot and I’m open source!
Presumably they did not want to see Cuda becoming the final de-facto standard that everyone uses. It nearly did at one point a couple of years ago, despite the lack of openness and lack of AMD hardware support.
They stopped funding the replacement, not CUDA.
That is contradicted by the headline. This easy confusion between CUDA (the API) and CUDA (the proprietary software package that is one implementation of it) illustrates the problem with CUDA.
ZLUDA seems to be an effort to fix that problem, but I don't know what it's chances of success might be.
It’s just a bad headline. They funded a CUDA replacement, then stopped funding it, as a result of which the project was released as open source.
By funding an API-compatible product, they are giving CUDA legitimacy as a common API. I can absolutely understand AMD not wanting a competitors invention and walled-off product to be anything resembling an industry standard.
It already has legitimacy. It's their hardware that doesn't, despite the decent raw flops and high memory.
i heavily rely on CUDA for many things i do on my personal computer. If this establishes itself as a reliable method to use all the funky CUDA stuff on AMD cards, my next card will 100% be AMD.
If there were a drop in equivalent to CUDA with AMD, I’d have several AMD cards, right now.
All I need to know about AMD is this:
Whenever amdgpu.ko is insmoded, the display (or the system itself in some cases) is unstable. When it works great now, one day it won’t and the machine will inexplicably start crashing randomly or displaying garbage after an update.
This has held true for years for me on many machines I’ve installed Linux on, and it still does: not a week ago, I updated a laptop with a Renoir chipset in it (RX Vega 6) that had been stable for years, and now the display gets corrupted whenever I switch VT. Because amdgpu…
Not bashing on AMD or Nvidia. This has just been my reality. As a result, whenever I have a choice, I go with Intel graphics because it never causes me as much of a headache.
.
Those benchmarks are impressive indeed. I certainly care much more about having extra VRAM than a little extra speed, comparing the 7900 XT or 7900 XTX to the RTX 4080.
I’d love to see some benchmarks for various LLMs and image generators.
Huh? What are these?
Since when does ROCm require kernel modules? DRI exists?
Technically always has, ROCm comes with a "backported" amdgpu module and that's the one they supposedly test/officially validate with. It mostly exists for the ancient kernels shipped with old long-time support distros.
Of course, ROCM being ROCM, nobody is running an officially supported configuration anyway and the thing is never going to work to an suitably acceptable level. This won't change that, since it's still built on top of it.
From github.com/vosen/ZLUDA?tab=readme-ov-file#faq
Now we’ll see if Nvidia drops CUDA immediately, or waits until next quarter.
Do LLM or that AI image stuff run on CUDA?
Cuda is required to be able to interface with Nvidia GPUs. AI stuff almost always requires GPUs for the best performance.
Yes, llama.cpp and derivates, stable diffusion, they also run on ROCm. LLM fine-tuning is CUDA as well, ROCm implementations not so much for this, but coming along.
Nearly all such software support CUDA, (which up to now was Nvidia only) and some also support AMD through ROCm, DirectML, ONNX, or some other means, but CUDA is most common. This will open up more of those to users with AMD hardware.
Thanks that is what I was curious about. So good news!
They are usually released for CUDA first, and if the projects got popular enough, someone will come in and port them to other platforms, which can take a while especially for rocm. Apple m series ports usually appear first before rocm, that’s show how much the devs community dislike working with rocm with famous examples such as geohot throwing the towel after working with rocm for a while.
Now let’s get this working on Nvidia hardware :P
as pointless as it sounds it would be a great way to test the system and call alternative implementations of each proprietary Nvidia library. It would also be great for debugging and development to provide an API for switching implementations at runtime.