Our first outage from LLM-written code
(sketch.dev)
from skip0110@lemmy.zip to programming@programming.dev on 01 Aug 02:00
https://lemmy.zip/post/45246325
from skip0110@lemmy.zip to programming@programming.dev on 01 Aug 02:00
https://lemmy.zip/post/45246325
cross-posted from: lemmy.bestiver.se/post/526967
threaded - newest
Why are we using tools that can’t parse the comment and code via syntax for refactoring?
The first problem is they’re letting AI touch their code.
The second problem is they’re relying on a human to pick up changes in moved code while using git’s built-in diff tools. There’s a whole bunch of studies that show how git’s diff algorithms are terrible, and how swapping to newer diff algos improves things considerably.
TL;DR on the studies:
There’s also a bunch of alternative diff algos you can use, but the best ones are paid, and the free ones have fewer features. See:
I gasped when I saw this:
This is like finding a live grenade under your bed and putting it under the rug.
They found a way to reproduce a system killing bug, and instead of taking the time to understand it, they threw away their test case.
They contained the impact. Root causing or “understanding” should come after impact mitigation. If needed find a safe way to reproduce the bug without customer impact.
Yeah me too but if you keep reading they didn’t actually “move on” in the way that it sounds.
Well done. More and more companies are deploying LLM-written code in production environments. Might as well be honest about the results so we can learn what does and doesn’t work.
It’s obvious that the LLM didn’t understand the code at all. It chose to refactor the way it did because of a silly comment.
It’s an inference model. It does not understand code no matter how much context it has. It can however output the most probable solution based on the context it has.