from cm0002@lemmy.world to programming@programming.dev on 10 Jun 06:51
https://lemmy.world/post/31121462
OC below by @HaraldvonBlauzahn@feddit.org
What called my attention is that assessments of AI are becoming polarized and somewhat a matter of belief.
Some people firmly believe LLMs are helpful. But programming is a logical task and LLMs can’t think - only generate statistically plausible patterns.
The author of the article explains that this creates the same psychological hazards like astrology or tarot cards, psychological traps that have been exploited by psychics for centuries - and even very intelligent people can fall prey to these.
Finally what should cause alarm is that on top that LLMs can’t think, but people behave as if they do, there is no objective scientifically sound examination whether AI models can create any working software faster. Given that there are multi-billion dollar investments, and there was more than enough time to carry through controlled experiments, this should raise loud alarm bells.
threaded - newest
Proceed to write a belief as a statement in the following paragraph
If you think LLMs doesnt think (I won’t argue that they arent extremely dumb), please define what is thinking, before continuing, and if your definition of thinking doesn’t apply to humans, we won’t be able to agree.
The burden of proof is on those who say that LLMs do think.
I asked for your definition, I cannot prove something if we do not agree on a definition first.
You also missread what I said, I did not said AI were thinking.
The burden of proof is on the one who made an affirmation.
I’m not the one who made an affirmation which field experts doesn’t know the answer.
But depending of your definition of thinking, some can be answered.
I don’t think y’all are disagreeing but maybe this sentence is somewhat confusing:
Maybe the “doesnt” shouldn’t be there.
No it is here because that’s what they claim.
Nobody yet know how it work, we don’t know how LLMs process information.
Anyone who claim it really think, or it isn’t thinking, is believing, this is not something the current ML field know.
Well, the neural network is given a prefix (series of tokens) and a token, and it spits out how likely is it that the token follows the prefix. Text is generated by calculating this probability for all known tokens, then picking one random, weighted based on the calculated probabilities.
And the brain is made out of neurons that sends electric signals between them and operate muscles.
That doesnt explain how the brain think.
It allows us to conclude that an LLM doesn’t “think” about what it is saying. Based on the mechanics, the LLM doesn’t even know it’s a participant in the conversation.
By that logic we also conclude that the human brain doesn’t “think” about what it is saying.
That does not follow. I can’t speak for you, but I can tell if I’m involved in a conversation or not.
Consciousness may be an illusion born from the ability of self reflection.
Also, like i showed before, you may act before consciously taking the decision of it.
en.m.wikipedia.org/…/Neuroscience_of_free_will
Theses study with the one presented by cgpgray, indicate that maybe we do stuff then we come up with a reasonable explanation after.
And how do you know LLMs can’t tell that they are involved in a conversation?
Unless you think there is something non-computational in the human brain, then you must accept that computers are - in theory - capable of thinking. With the right software and sufficiently powerful hardware.
Given that truth (which I think you can only avoid through religion or quantum quackery), you can’t just say “it’s only maths; it can’t be thinking” because we know that maths can think.
Do LLMs “think”? The definition of “think” is wooly enough and we understand them little enough that it’s quite an assertion to say that they definitely don’t.
It has no memory, for one. What makes you think that it does know its in a conversation?
It has very short term memory in the form of it’s token context. Especially with something like Meta’s Coconut.
I don’t really. Yet. But I also don’t think that it is fundamentally impossible for LLMs to think, like you seem to. I also don’t think the definition of the word “think” is so narrow that it requires that level of self-awareness. Do you think a mouse is really aware it is a mouse? What about a spider?
How did you concluded that from theses 2 messages.
Since LLMs runs on CPUs with a lot of memory, do you agree that my calculator is thinking?
You think computation is thinking ?
I asked for your definition of thinking.
The OP talked about belief, then made a statement using a word that is not precisely defined.
If you think computation is thinking then by your definition the LLM is thinking.
But that’s your definition of thinking.
This argument makes no more sense than trying to say that a plant is thinking because brains are made of cells and so are plants.
’ Please succinctly answer a question of philosophy that has plagued mankind for thousands of years. can’t? <crosses arms with a superior smirk> I win’
Claiming LLMs can’t think with the current informations available, and calling that not a belief, is claiming to have a response to this philosophy question.
The only sensible answer is saying you don’t know, or being aware and communicating that your statement is a belief.
I don’t think the current common implementation of AI systems are “thinking” and I’ll base my argument on Oxford’s definitions of words. Thinking is defined as “the process of using one’s mind to consider or reason about something”. I’ll ignore the word “mind” and focus on the word “reason”. I don’t think what AIs are doing counts as reasoning as defined by Oxford. Let’s go to that definition: “the power of the mind to think, understand, and form judgments by a process of logic”. I take issue with the assertion that they form judgments. For completeness, but I don’t think it’s definition is particularly relevant here, a judgment is: “the ability to make considered decisions or come to sensible conclusions”.
I think when you ask an LLM how many 'r’s there are in Strawberry and questions along this line you can see they can’t form judgments. These basic but obscure questions are where you see that the ability to form judgements isn’t there. I would also add that if you “form judgments” you probably don’t need to be reminded you formed a judgment immediately after forming one. Like if I ask an LLM a question, and it provides an answer, I can convince it that it was wrong whether or not I’m making junk up or not. I can tell it it made a mistake and it will blindly change it’s answer whether it made a mistake or not. That also doesn’t feel like it’s able to reason or make judgments.
This is where all the hype falls flat for me. It feels like sometimes it looks like a concrete wall, but occasionally that concrete wall is made of wet paper. You can see how impressive the tool is and how paper thin it is at the same time. It’s cool, it’s useful, it’s fake, and that’s ok. Just be aware of what the tool is.
Like a LLMs you are making the wrong affirmation based lacking knowledge.
Current LLMs input, and output tokens, they dont ever see the individual letters, they see tokens, for straberry, they see 3 tokens:
<img alt="" src="https://lemmy.world/pictrs/image/33f68b38-9f9e-412c-8126-1363e762dd92.png">
They dont have any information on what characters are in this tokens. So they come up with something. If you learned a language only by speaking, you’ll be unable to write it down correctly (except purely phonetical systems), instead you’ll come up with what you think the word should be written.
You come up with the judgment before you are aware of it: unsw.edu.au/…/our-brains-reveal-our-choices-befor…
That’s also how the brain can works, it come up with a plausible explanation after having the result.
See the experience which are spoken about here: www.youtube.com/watch?v=wfYbgdo8e-8
I showed the same behavior in humans of some behavior you observed in LLMs, does this means that by your definition, humans doesnt think ?
If the LLM could reason, shouldn’t it be able to say “my token training prevents me from understanding the question as asked. I don’t know how many 'r’s there are in Strawberry, and I don’t have a means of finding that answer”? Or at least something similar right? If I asked you what some word in a language you didn’t know, you should be able to say “I don’t know that word or language”. You may be able to give me all sorts of reasons why you don’t know it, and that’s all fine. But you would be aware that you don’t know and would be able to say “I don’t know”.
If I understand you correctly, you’re saying the LLM gets it wrong because it doesn’t know or understand that words are built from letters because all it knows are tokens. I’m saying that’s fine, but it should be able to reason that it doesn’t know the answer, and say that. I assert that it doesn’t know that it doesn’t know what letters are, because it is incapable of coming to that judgement about its own knowledge and limitations.
Being able to say what you know and what you don’t know are critical to being able to solve logic problems. Knowing which information is missing and can be derived from known things, and which cannot be derived is key to problem solving based on reason. I still assert that LLMs cannot reason.
That is of course a big problem. They try to guess too much stuff, but it’s also why it kinda works. Symbolics AI have the opposite problem, they are rarely useful, because they can’t guess stuff, they are rooted in hard logic, and cannot come up with a reasonable guess.
Now humans also try to guess stuff and sometimes get it wrong, it’s required in order to produce results from our thinking and not be stuck in a state where we don’t have enough data to do anything, like a symbolic AI.
Now, this is becoming a spectrum, humans are somewhere in the middle of LLMs and symbolics AI.
LLMs are not completely unable to say what they know and doesnt know, they are just extremely bad at it from our POV.
The probleme with “does it think” is that it doesn’t give any quantity or quality.
Is the argument that LLMs are thinking because they make guesses when they don’t know things combined with no provided quantity or quality to describe thinking?
If so, I would suggest that the word “guessing” is doing a lot of heavy lifting here. The real question would be “is statistics guessing”? I would say guessing and statistics are not the same thing, and Oxford would agree. An LLM just grabs tokens based on training data on what word or token most likely comes next, it will just be using what the statistically most likely next token or word is. I don’t think grabbing the highest likely next token counts as guessing. That feels very algorithmic and statistical to me. It is also possible I’m missing the argument still.
No, it’s that you can’t root the argument that they don’t think over the fact they make stuff up, because humans too. You could root it in the amount of things it guess wrong, but it’s extremely hard to measure.
Again, I’m not claiming that they think, but that we don’t know until one or the other is proven.
Right now, thinking one, or the other is true, is belief.
I think you can make a strong argument that they don’t think rooted in words should mean something and that statistics and thinking don’t mean the same thing. To me, that feels like a fairly valid argument.
So you think you need words to be able to think ? Monkeys, birds, human babies are unable to think then ?
My apologies, I was too vague. I’m saying “thinking” by definition is not “statistics”. Where Monkeys, birds, and human babies all “think”, LLMs use algorithms and “statistics”. I also think that “statistics” not meaning the same thing that “thinking” is a valid argument. I would go farther and say it’s important that words have meaning. That is what I was attempting to convey. I’m happy to clear up anything I was unclear about.
You are mistaking how LLMs are trained to how they work.
It’s not because it’s been trained with statistics, that they compute, or think using statistics.
For example, to do additions, internally LLMs do trignonometry: arxiv.org/abs/2502.00873
They do probably use statistics for tons of stuff internally, but humans do too: guessing, bias, tendency, preferences.
Anthropics researcher found that their LLMs have “features” for concepts.
I don’t think you can disconnect how an LLM was trained from how it operates. If you train an LLM to use trigonometry to solve addition problems, I think you will find the LLM will do trigonometry to solve addition problems. If you train an LLM in only Russian, it will speak Russian. I would suggest that regardless of what you train it on it will choose the statistically most likely next token based on its training data.
I would also suggest we don’t know the exact training data being used on most LLMs, so as outsiders we can’t say one way or another on how the LLM is being trained to do anything. We can try to extrapolate from posts like the one that you linked to how the LLM was trained though. In general if that is how the LLM is coming to its next token, then the training data must be really heavily weighted in that manner.
You can, heck the example I gave show exaclty this:
It was not trained to do trigonometry to solve addition problem, it was trained to respond to additions, trigonometry is how the statiscal part, the backpropagation, found a way to make the neurons solve additions.
You are mixing up stuff, the way LLM are trained does not impose anything about how the neurons gets organised to get better score at inferrence.
I would point out I think you might be overly confident in the manner in which it was trained addition. I’m open to being wrong here, but when you say “It was not trained to do trigonometry to solve addition problem”, that suggests to me either you know how it was trained, or are making assumptions about how it was trained. I would suggest unless you work at one of these companies, you probably are not privy to their training data. This is not an accusation, I think that is probably a trade secret at this point. And if the idea that there would be nobody training an LLM to do addition in this manner, I invite you to glance the Wikipedia article on addition. Really glance at literally any math topic on Wikipedia. I didn’t notice any trigonometry in this entry but I did find the discussion around finding the limits of logarithmic equations in the “Related Operations” section: en.m.wikipedia.org/wiki/Addition. They also cite convolution as another way to add in which they jump straight to calculus: en.m.wikipedia.org/wiki/Convolution.
This is all to say, I would suggest that we don’t know how they’re training LLMs. We don’t know what that training data is or how it is being used exactly. What we do know is that LLMs work on tokens and weights. The weights and statistical relevance to each of the other tokens depends on the training data, which we don’t have access to.
I know this is not the point, but up until this point I’ve been fairly pedantic and tried to use the correct terminology, so I would point out that technically LLMs have “tensors” not “neurons”. I get that tensors are designed to behave like neurons, and this is just me being pedantic. I know what you mean when you say neurons, just wanted to clarify and be consistent. No shade intended.
The general way it is trained is known, specifics and technics are not known, but the public do know how one of the flagship model was trained, the training process of deepseek r1 was documented in their research paper: arxiv.org/pdf/2501.12948
Which I did read a chunk when it was released.
The LLMs have multiple way to do additions, i’ll showcase two as an example, I asked ChatGPT 4.1 to solve a big addition. Here it’s output:
I need to add these two large numbers carefully. First, I'll write them one under the other, aligned at the right: 21,500,996,531,314,339 + 96,357,764,557,683,165,397,721 ---------------------------------- Working from right to left: Units place: 9 + 1 = 10. Write 0, carry 1. Tens place: 3 + 2 + 1(carry) = 6. Write 6, no carry. Hundreds place: 3 + 7 = 10. Write 0, carry 1. Thousands place: 4 + 7 + 1(carry) = 12. Write 2, carry 1. Ten thousands place: 1 + 9 + 1(carry) = 11. Write 1, carry 1. Hundred thousands place: 3 + 3 + 1(carry) = 7. Write 7, no carry. Millions place: 1 + 5 = 6. Write 6, no carry. Ten millions place: 5 + 6 = 11. Write 1, carry 1. Hundred millions place: 3 + 1 + 1(carry) = 5. Write 5, no carry. Billions place: 5 + 3 = 8. Write 8, no carry. Ten billions place: 9 + 8 = 17. Write 7, carry 1. Hundred billions place: 9 + 6 + 1(carry) = 16. Write 6, carry 1. Trillions place: 0 + 7 + 1(carry) = 8. Write 8, no carry. Ten trillions place: 0 + 5 = 5. Write 5, no carry. Hundred trillions place: 5 + 5 = 10. Write 0, carry 1. Quadrillions place: 1 + 4 + 1(carry) = 6. Write 6, no carry. Ten quadrillions place: 2 + 6 = 8. Write 8, no carry. For the remaining places, I just copy the digits from the second number: 7, 7, 5, 3, 6, 9 Therefore, the answer is: 96,357,786,056,867,851,761,260You can notice, the whole reasoning is correct, but it wrote the wrong response, I can expand more on this if you want (I do some research on it on my free time)
This reasoning of decomposing the addition was of course learned from training data.
Now, the trigonometry used to calculate additions that i talked earlier, is not for writing a “reasoning” but when it try to write the correct response. It was created by the backpropagation trying to find a local minimum that can solve additions in order to more accuratly predict the next token.
Artificial neurons were made to behave like neurons: en.wikipedia.org/wiki/Artificial_neuron
And the terminology used, is neurons, cf the paper i sent earlier about how they do additions: arxiv.org/pdf/2502.00873
I don’t doubt that it can perform addition in multiple ways. I would go as far as saying it can probably attempt to perform addition in more ways than the average person as it has probably been trained on a bunch of math. Can it perform it correctly? Sometimes. That’s ok, people make mistakes all the time too. I don’t take away from LLMs just because they make mistakes. The ability to do math in multiple ways is not evidence of thinking though. That is evidence that it’s been trained on at least a fair bit of math. I would say if you train it on a lot of math, it will attempt to do a lot of math. That’s not thinking, that’s just increasing the weighting on tokens related to math. If you were to train an LLM on nothing but math and texts about math, then asked it an art question, it would respond somewhat nonsensically with math. That’s not thinking, that’s just choosing the statistically most likely next token.
I had no idea about artificial neurons, TIL. I suppose that makes “neural networks” make more sense. In my readings on ML they always seemed to go straight to the tensor and overlook the neuron. They would go over the functions to help populate the weights but never used that term. Now I know.
I’ve been re reading my response and my bad, I meant “artificial neurons were inspired from neurons”, not to behave like, they have little in common.
If you asked an human that speak german and nothing else, a question in english, it would also respond in german (that they cant understand you).
LLMs sometimes (not often enough) do respond they don’t know.
Ah still rolling out the old “stochastic parrot” nonsense I see.
Anyway on to the actual article… I was hoping it wouldn’t make these basic mistakes:
Yes we do. Frankly if you’ve used it it’s so obviously better than regular JavaScript you probably don’t need more evidence (it’s like looking for “evidence” that film stars are more attractive than average people). But anyway we do have great papers like this one.
Anyway that’s slightly beside the point. I think the article is right that smart people are not invulnerable to manipulation or falling for “obviously” stupid ideas. I know plenty of very smart religious people for example.
However I think using this to dismiss LLMs is dumb, in the same way that his dismissal of Typescript is. LLMs aren’t homeopathy or religion.
I have used LLMs to get some work done and… guess what, it did the work! Do I trust it to do everything? Obviously not. But sometimes I don’t need perfect code. For example recently I asked it to create an example SystemVerilog file for me utilising as many syntax features as possible (testing an auto-formatter). It did a pretty good job. Saved some time. What psychological hazard have I fallen for exactly?
Overall, B-. Interesting ideas but flawed logic.
Amen
And to add that smart people fall for dumb biases, we just need to look at the object oriented mania of the 2000s to late 2010s to see us shoehorn in one paradigm into everything without critically considering whether it made sense over other models.
Can an LLM do everything I need yet? No.
But is a stochastic parrot good enough to help me complete a function and help me restructure code? Yes definitely.
Claude is good enough for so much of the low value code I write that is actually a useful tool. I have to review the code but it’s useable.
I use AI search to lookup functions that I don’t need detailed docs for, or to help me debug arcane library specific errors (just had one earlier today where in polars the list and array types are very much not interchangeable and the explode method was failing).
I still read the docs on things that are critical, and I write the critical paths and dictate structure and understand the problem im solving well.
It’s really amazing the number of people trying to argue that LLMs are useless, while simultaneously so many people are using them successfully. Makes me wonder if they’ve even tried them.
Ah still rolling out the old “computers think” pseudo-science.
Ah yes the old pointless vague anecdote.
Promoting pseudo-science.
Overall D. Neither interesting nor new nor useful.
If your argument is “LLMs can’t do useful work”, and then I say “no, I’ve used them to do useful work many times” how is that a pointless vague anecdote? It’s a direct proof that you’re wrong.
Sorry what? This is bizarre.
It is a bunch of stochastic parrots. It just happens frequently that the words they are parroting were orginally written by a bunch of intelligent people which were knowledgeable in their fields.
Note this doesn’t makes the parrots intelligent - in the same way that a book written by Einstein to explain special relativity has any own intelligence. Einstein was intelligent, his words transport his intelligent ideas, but the book conveying them to other people (as, the printed pages with cardboard cover) is as dumb as a stone. You would not ask a piece of cardboard so solve a math problem, would you?
Your comment doesn’t account for the fact that LLMs can generalise. Often not very well but they can produce outputs for inputs not seen in their training sets. Otherwise what would be the point?
Uhhh you know LLMs can solve quite complex maths problems? Including novel ones.
I fear this is a problem that may never be solved. I mean that people of any intelligence fall for the mind’s biases.
There’s just too little to be gained feelings-wise. Yeah, you make better decisions, but you’re also sacrificing “going with the flow”, acting like our nature wants us to act. Going against your own nature is hard and sometimes painful.
Making wrong decisions is objectively worse, leading to worse outcomes, but if it doesn’t feel worse (because you’re not attributing the effects of the wrong decisions to the right cause, i.e. acting irrationally), then why should a person do it. If you follow the mind’s bias towards attributing your problems away from irrationality, it’s basically a self-fulfilling prophecy.
Great article.
Another similar article that’s really good:
The LLMentalist Effect: how chat-based Large Language Models replicate the mechanisms of a psychic’s con
Reponding to another comment in opensource@lemmy.ml:
What you confuse here is doing something that can benefit from applying logical thinking with doing science. For exanple, mathematical arithmetic is part of math and math is science. But summing numbers is not necessarily doing science. And if you roll, say, octal dice to see if the result happens to match an addition task, it is certainly not doing science, and no, the dice still can’t think logically and certainly don’t do math even if the result sometimes happens to be correct.
For the dynamic vs static typing debate, see the article by Dan Luu:
danluu.com/empirical-pl/
But this is not the central point of the above blog post. The central point of it is that, by the very nature of LLMs to produce statistically plausible output, self-experimenting with them subjects one to very strong psychological biases because of the Barnum effect and therefore it is, first, not even possible to assess their usefulness for programming by self-experimentation(!) , and second, it is even harmful because these effects lead to self-reinforcing and harmful beliefs.
And the quibbling about what “thinking” means is just showing that the arguments pro-AI has degraded into a debate about belief - the argument has become “but it seems to be thinking to me” even if it is technically not possible and also not in reality observed that LLMs apply logical rules, cannot derive logical facts, can not explain output by reasoning , are not aware about what they ‘know’ and don’t ‘know’, or can not optimize decisions for multiple complex and sometimes contradictory objectives (which is absolutely critical to any sane software architecture).
What would be needed here are objective controlled experiments whether developers equipped with LLMs can produce working and maintainable code any faster than ones not using them.
And the very likely result is that the code which they produce using LLMs is never better than the code they write themselves.
What’s the difference between copying a function from stack overflow and copying a function from a llm that has copied it from SO?
LLM are sort of a search engine with advanced language substitution features nothing more nothing less.
LLM are poor snapshots of a search engine with no way to fix any erroneous data. If you search something on Stack you get the page with several people providing snippets and debating the best approach. The LLM does not give you this. Furthermore if the author goes back and fixes an error in their code the search will find it whereas the LLM will give you the buggy code with no way to reasonably update it
LLM have major issues and even bigger limitations. Pretending they are some panacea is going to disappoint.
LLM also does not bully you for asking. Nor it says “duplicated question” for non duplicated questions… There’s a reason people prefer LLM to SO nowadays.
It’s not panacea. But it’s not the doom world destroying useless machine that some people like to tell it is.
It’s a useful tool for some task if you know how to use it. Everyone who actively use it is because we have find put that it works for us better than other tools for that task, of not we would not use it.
Giving my own personal experience, I tend to ask first to an LLM rather that what I used to do digging in old SO answers because I get the answer quicker and a lot of the times just better. It’s not perfect by any stretch of the imagination, but it serves me a purpose.
For instance last week I needed a PowerShell command to open an app compatibility menu from the command line. I asked and got this as a response:
(New-Object -ComObject Shell.Application).Namespace((Split-Path “C:\Ruta\A\TuPrograma.exe”)).ParseName((Split-Path “C:\Ruta\A\TuPrograma.exe” -Leaf)).InvokeVerb(“P&roperties”)
Worked at first try, exactly as I wanted.
You are free to try a search engine with the query “PowerShell command to open an app compatibility menu from the command line” and check for yourself how little help the firsts results get you.
It’s a tool, as many others. The magic lies in knowing when and how to use it. For other things I may not use it, but after a couple of years using it I’m developing a good sense of which questions does it handle well and which questions is better not even to try.
It takes an enormous amount of energy and processing power to create these shitty snapshots so in many ways it is doom considering it will dramatically increase our energy usage.
I get it, you are an AI supporter but you fail to critically analyze it or even understand it. What tool would you use that you can’t correct errors to or even determine how it works. You are really operating on faith here that the black box your getting an answer from is giving you the correct answer.
Perhaps a code snippet works, but after this is where it all falls apart. What if the snippet does not work or causes a problem. The LLM has nothing to offer you here.
Not really.
I self host my own LLM. Energy consumption for queries is lower than gaming according to my own measures. And the models are not made so frequently (I use models made last year still). And once the model is done is infinitely reusable by anyone.
I get that you are starting by the axiom “AI is bad” and then create the arguments needed to support that axiom. Instead of going the other way around with an open mind.
I told you my own personal experience with it. Take it as you want. For me, my situation will be the same. I would keep using same as I use any other tool that works for me, and will stop using it when there’s something better same as I’ve done countless times. I’m not easy to peer pressure into any particular stance, so I can form my own opinions based on what I test for myself. I really think a lot of arguments against AI boil down to some sort of political stance. AI hurt a series of small artists which had a very big voice in some spaces, and thus an anti-AI political movement was created. My own copyleft morals made me undisturbed by this original complains about generative AI, and the rest of arguments have been very unconvincing, straight up fake, logical fallacies, or just didn’t really check out with the reality I was able to test by myself.
For instance I saw other post today saying how 3 watts hour per query was an absolute energy waste for a household. When that’s absolutely nothing compared to the 30.000 watts hour a typical household spend each day, even with quite and amount of queries. Sincerely I spent last few months with one of these devices to measure energy consumption attached to my computer and AI energy usage was really underwhelming compared with what people told me it was going to be. AAA gaming is consistently more energy hungry.
I know you are willfully being ignorant here as AI data centers are projected to use more electricity than the entire nation of Japan by 2030.
Your own hosted LLM is not the problem nor the issue we are even discussing and quite frankly a little insulting you bring it up
I am not anymore anti-AI than any tool that you can’t determine is accurate nor correct if there is an issue with it. LLM have a long way to go before they are even a fraction of what they claim to be.
Another problem is they do not cite where they get their answers from. Without the ability to audit the answers you are given you won’t know how accurate they are.
I have listed several legitimate gripes about LLM. I find your fanboism misplaced and I think you are just playing devil’s advocate at this point. AI is a hype train and I am sick of it already.
I will just copy my other response about datacenters energy usage, ignore the parts not related to our conversation:
Google is not related with chatgpt. Chatgpt parent company is openAI which is a competitor with google.
A more rational explanation is that technology and digital services on general have been growing and are on the rise. Both because more and more complex services are being offered, and more importantly more people are requesting those services. Whole continents that used not to be cover by digital services are now covered. Generative AI is just a very small part of all that.
The best approach to reduce CO2 emissions is to ask for a reduction in human population. From my point of view is the only rational approach, as with a growing population there’s only two solutions, pollute until we die, or reduce quality of life until life is not worth living. Reducing population allows for fewer people to live better loves without destroying the planet.
It also arises the question on why am I responsible if a big tech company decided to make an llm query of every search or overuse the technology, when I am talking about a completely different usage of that technology, that doesn’t even reach a 20-30 queries a day which would have a power usage of less than a few hundreds wh at most, which os negligible in the scheme of global warming and my total energetic footprint.
How it’s being a fanboy saying that “It works for me in some particular cases and not others, it’s a tool that can be used”.
Please, read again this conversation and do a second guess on who is a radical extremist here.
In the case we were talking, writing code, I am the auditor of the answers. I do not ““vibe code”” I read the code that’s proposed, understand it, and if it’s code that I would have written I copy it, if not I change it. “Vibe coding” is an example of bad usage of the tool that would lead to problems. All code not written by yourself and copied from other source should be reviewed. Once it pass my review is as good as my own code. If it fail it would fail the same as any other code witten by me, as it’s something that I was clearly unable to see.
For instance a couple of months ago I wrote a small API service that worked fine at first and suddenly stopped working a few weeks in production. It was a stupid mistake I made, and I needed no LLM to do that mistake. The service was so simple that I didn’t really even used LLM there. But I made a mistake regardless. I could have use AI and get the same bad function that caused the issue. And the blame would still be mine for not seeing the problem.
Once again is a tool. If some jackass decide to vibe code an app and it’s a shit app, is a bad use of the tool. But some other people can de proper reviews and analysis of the generated code and assume full responsibility of any failures of that code.
That is actually missing an important issue, hallucinations.
Copying from SO means you are copying from a human who might be stupid or lie but rarely spews out plausible sounding hot garbage (not never though) and because of other users voting and reputation etc etc, you actually do endup with a decently reliable source.
With an LLM you could get something made up based on nothing related to the real world. The LLM might find your question to be outside of it’s knowledge but instead of realizing it it would just make up what it thinks sounds convincing.
It would be like if you asked me how that animal that is half horse and half donkey is called and instead of saying “shit i’m blanking” I would say “Oh, that is called a Drog” and I couldn’t even tell you that I just made up that word because I will now be convinced that this is factual. Btw it’s “mule”
So there is a real difference until we solve hallucinations, which right now doesn’t seem solvable but at best reduced to insignificance (maybe)
That’s why you meed to know the cavieats of the tool you are using.
LLM hallucinate. People willing to use them need to know, where is more prone to hallucinate. Which is where the data about the topic you are requesting is more fuzzy. If you ask for the capital of France is highly unlikely you will get an hallucination, if you as for the color of the hair of the second spouse of the fourth president of the third French republic, you probably will get an hallucination.
And you need to know what are you using it for. If it’s for roleplay, or any not critical matters you may not care about hallucinations. If you use them for important things you need to know that the output needs to be human reviewed before using it. For some things it may be worth the human review as it would be faster that writing from zero, for other instances it may not be worth it and then a LLM should not be used for that task.
As an example I just was writing some lsp library for an API and I tried the LLM to generate it from the source documentation. I had my doubts as the source documentation is quite bigger that my context size, I tried anyway but I quickly saw that hallucinations were all over the place and hard to fix, so I desisted and I’ve been doing it myself entirely. But before that I did ask the LLM how to even start writing such a thing as it is the first time I’ve done this, and the answer was quite on point, probably saving me several hours searching online trying to find out how to do it.
It’s all about knowing the tool you are using, same as anything in this world.
Because it’s not a plain copy but an Interpretation of SO.
With llm you just have one more layer between you and the information that can distort that information.
And?
The issue is that you should not blindly trust code. Being originally written by a human being is not, by any means, a quality certification.
You asked what’s the difference and I just told you.
Are you stupid or something?
Block and reported.
You should not insult people.
Genuine question.
If you have to use AI - maybe your work insists on it - always demand it cite its sources, hope they are relevant, and go read those instead.