vzq@lemmy.blahaj.zone
on 24 May 2024 20:21
nextcollapse
You should see 52% of the first version of my code.
It doesn’t have to be right to be useful.
restingboredface@sh.itjust.works
on 24 May 2024 20:24
nextcollapse
Yeah, but the non-tech savvy business leaders see they can generate code with AI and think ‘why do I need a developer if I have this AI?’ and have no idea whether the code it produces is right or not. This stat should be shared broadly so leaders don’t overestimate the capability and fire people they will desperately need.
scrubbles@poptalk.scrubbles.tech
on 24 May 2024 20:34
nextcollapse
Yeah management are all for this, the first few years here are rough with them immediately hitting the “fire the engineers we have ai now”. They won’t realize their fuckup until they’ve been promoted away from it
Boozilla@lemmy.world
on 24 May 2024 20:48
nextcollapse
Programming jobs will be safe for a while. They’ve been trying to eliminate those positions since at least the 90s. Because coders are expensive and often lack social skills.
But I do think the clock is ticking. We will see more and more sophisticated AI tools that are relatively idiot-proof and can do things like modify Salesforce, or create complex new Tableau reports with a few mouse clicks, and stuff like that. Jobs will be chiseled away like our unfortunate friends in graphic design.
BlameThePeacock@lemmy.ca
on 24 May 2024 21:10
collapse
You, along with most people, are still looking at automation wrong. It’s never been about removing people entirely, even AI, it’s about doing the same work with less cost.
If you can eliminate one programmers from your four person team by giving the other three AI to produce the same amount of work, congrats you’ve just automated one programming job.
Programming jobs aren’t going anywhere, but either the amount of code produced is about to skyrocket, or the number of employed programmers is going to drop (or most likely both of those things).
Tyrangle@lemmy.world
on 24 May 2024 21:59
nextcollapse
Right on. AI feels like a looming paradigm shift in our field that we can either scoff at for its flaws or start learning how to exploit for our benefit. As long as it ends up boosting productivity it’s probably something we’re going to have to learn to work with for job security.
BlameThePeacock@lemmy.ca
on 25 May 2024 01:31
collapse
It’s already boosting productivity in many roles. That’s just going to accelerate as the models get better, the processing gets cheaper, and (as you said) people learn to use it better.
myliltoehurts@lemm.ee
on 25 May 2024 00:17
nextcollapse
I wonder if this will also have a reverse tail end effect.
Company uses AI (with devs) to produce a large amount of code -> code is in prod for a few years with incremental changes -> dev roles rotate or get further reduced over time -> company now needs to modernize and change very large legacy codebase that nobody really understands well enough to even feed it Into the AI -> now hiring more devs than before to figure out how to manage a legacy codebase 5-10x the size of what the team could realistically handle.
Writing greenfield code is relatively easy, maintaining it over years and keeping it up to date and well understood while twisting it for all new requirements - now that’s hard.
BlameThePeacock@lemmy.ca
on 25 May 2024 01:29
collapse
AI will help with that too, it’s going to be able to process entire codebases at a time pretty shortly here.
Given the visual capabilities now emerging, it can likely also do human-equivalent testing.
One of the biggest AI tricks we haven’t started seeing much of yet in mainstream use is this kind of automated double-checking. Where it generates an answer, and then validates if the answer is valid before actually giving it to a human. Especially in coding bases, there really isn’t anything stopping it from coming up with an answer compiling, running into an error, re-generating, and repeating until the code passes all unit tests or even potentially visual inspection.
The big limit on this right now is sheer processing cost and context lengths for the models. However, costs for this are dropping faster than any new tech we’ve seen, and it will likely be trivial in just a few years.
MajorHavoc@programming.dev
on 25 May 2024 02:02
collapse
There are some areas I’m hoping get addressed by the coming skyrocket in programmer productivity:
Several phone apps aren’t utter garbage anymore. I’m not holding my breath on this one.
Online grocery websites aren’t shit-full-of-timing errors. If I get this, I’ll also wish for $1 million and buy a lottery ticket.
Municipalities and their allies (townships, city services, various local unions) will have barely passable specialized software support that actually fits their size, location and maybe even culture.
I think that last one stands to be strongly enabled by AI code assist tools. It might not be the sexiest or highest paying job, but it’ll be work that matters that largely isn’t even being done today.
piecat@lemmy.world
on 24 May 2024 20:49
nextcollapse
I say let it happen. If someone is dumb enough to fire all their workers… They deserve what will happen next
homesweethomeMrL@lemmy.world
on 24 May 2024 20:55
nextcollapse
Well the firing’s happening so, i guess let’s hope you’re right about the other part.
It won’t happen like that. Leadership will just under-hire and expect all their developers to be way more efficient. Working will be really stressful with increased deadlines and people questioning why you couldn’t meet them.
NuXCOM_90Percent@lemmy.zip
on 24 May 2024 21:14
nextcollapse
Mentioned it before but:
LLMs program at the level of a junior engineer or an intern. You already need code review and more senior engineers to fix that shit for them.
What they do is migrate that. Now that junior engineer has an intern they are trying to work with. Or… companies realize they don’t benefit from training up those newbie (or stupid) engineers when they are likely to leave in a year or two anyway.
Yeah cause my favorite thing to do when programming is debugging someone else’s broken code.
Supervisor194@lemmy.world
on 24 May 2024 22:18
nextcollapse
I think where it shines is in helping you write code you’ve never written before. I never touched Swift before and I made a fully functional iOS app in a week. Also, even with stuff I have done before, I can say “write me a function that does x” and it will and it usually works.
Like just yesterday I asked it to write me a function that would generate and serve up an .ics file based on a selected date and extrapolate the date of a recurring monthly meeting based on the day of the week picked and its position (1st week, 2nd week, etc) within the month and then make the .ics file reflect all that. I could have generated that code myself by hand but it would have probably taken me an hour or two. It did it in about five seconds and it worked perfectly.
Yeah, you have to know what you’re doing in general and there’s a lot of babysitting involved, but anyone who thinks it’s just useless is plain wrong. It’s fucking amazing.
Edit: lol the article is referring to a study that was using GPT 3.5, which is all but useless for coding. 4.0 has been out for a year blowing everybody’s minds. Clickbait trash.
thanks_shakey_snake@lemmy.ca
on 24 May 2024 23:35
collapse
3.5 is still reasonably useful for the same reasons you described, imo… Just less so.
Aurenkin@sh.itjust.works
on 25 May 2024 00:12
nextcollapse
Get it to debug itself then.
kjaeselrek@lemmy.ml
on 25 May 2024 03:05
nextcollapse
Yeah I’ve already got enough legacy code to deal with, I don’t need more of it faster.
vzq@lemmy.blahaj.zone
on 25 May 2024 08:54
collapse
To be fair, I’m starting to fear that all the fun bits of human jobs are the ones that are most easy to automate.
I dread the day I’m stuck playing project manager to a bunch of chat bots.
thehatfox@lemmy.world
on 24 May 2024 21:26
nextcollapse
Generally you want to the reference material used to improve that first version to be correct though. Otherwise it’s just swapping one problem for another.
I wouldn’t use a textbook that was 52% incorrect, the same should apply to a chatbot.
Bad take. Is the first version of your code the one that you deliver or push upstream?
LLMs can give great starting points, I use multiple LLMs each for various reasons. Usually to clean up something I wrote (too lazy or too busy/stressed to do manually), find a problem with the logic, or maybe even brainstorm ideas.
I rarely ever use it to generate blocks of code like asking it to generate “a method that takes X inputs and does Y operations, and returns Z value”. I find that those kinds of results are often vastly wrong or just done in a way that doesn’t fit with other things I’m doing.
brbposting@sh.itjust.works
on 25 May 2024 00:08
collapse
LLMs can give great starting points, I use multiple LLMs each for various reasons. Usually to clean up something I wrote (too lazy or too busy/stressed to do manually), find a problem with the logic, or maybe even brainstorm ideas.
Impressed some folks think LLMs are useless. Not sure if their lives/workflows/brains are that different from ours or they haven’t given at the college try.
I almost always have to use my head before a language model’s output is useful for a given purpose. The tool almost always saves me time, improves the end result, or both. Usually both, I would say.
It’s a very dangerous technology that is known to output utter garbage and make enormous mistakes. Still, it routinely blows my mind.
autotldr@lemmings.world
on 24 May 2024 20:25
nextcollapse
This is the best summary I could come up with:
In recent years, computer programmers have flocked to chatbots like OpenAI’s ChatGPT to help them code, dealing a blow to places like Stack Overflow, which had to lay off nearly 30 percent of its staff last year.
That’s a staggeringly large proportion for a program that people are relying on to be accurate and precise, underlining what other end users like writers and teachers are experiencing: AI platforms like ChatGPT often hallucinate totally incorrectly answers out of thin air.
For the study, the researchers looked over 517 questions in Stack Overflow and analyzed ChatGPT’s attempt to answer them.
The team also performed a linguistic analysis of 2,000 randomly selected ChatGPT answers and found they were “more formal and analytical” while portraying “less negative sentiment” — the sort of bland and cheery tone AI tends to produce.
The Purdue researchers polled 12 programmers — admittedly a small sample size — and found they preferred ChatGPT at a rate of 35 percent and didn’t catch AI-generated mistakes at 39 percent.
The study demonstrates that ChatGPT still has major flaws — but that’s cold comfort to people laid off from Stack Overflow or programmers who have to fix AI-generated mistakes in code.
The original article contains 340 words, the summary contains 199 words. Saved 41%. I’m a bot and I’m open source!
Boozilla@lemmy.world
on 24 May 2024 20:35
nextcollapse
It’s been a tremendous help to me as I relearn how to code on some personal projects. I have written 5 little apps that are very useful to me for my hobbies.
It’s also been helpful at work with some random database type stuff.
But it definitely gets stuff wrong. A lot of stuff.
The funny thing is, if you point out its mistakes, it often does better on subsequent attempts. It’s more like an iterative process of refinement than one prompt gives you the final answer.
mozz@mbin.grits.dev
on 24 May 2024 20:47
nextcollapse
It’s incredibly useful for learning. ChatGPT was what taught me to unlearn, essentially, writing C in every language, and how to write idiomatic Python and JavaScript.
It is very good for boilerplate code or fleshing out a big module without you having to do the typing. My experience was just like yours; once you’re past a certain (not real high) level of complexity you’re looking at multiple rounds of improvement or else just doing it yourself.
Boozilla@lemmy.world
on 24 May 2024 20:53
nextcollapse
Exactly. And for me, being in middle age, it’s a big help with recalling syntax. I generally know how to do stuff, but need a little refresher on the spelling, parameters, etc.
Personally I find all LLMs in general not that great at writing larger blocks of code. It’s fine for smaller stuff, but the more you expect out of it the more it’ll get wrong.
I find they work best with existing stuff that you provide. Like “make this block of code more efficient” or “rewrite this function to do X”.
Downcount@lemmy.world
on 24 May 2024 20:52
nextcollapse
The funny thing is, if you point out its mistakes, it often does better on subsequent attempts.
Or it get stuck in an endless loop of two different but wrong solutions.
Me: This is my system, version x. I want to achieve this.
ChatGpt: Here’s the solution.
Me: But this only works with Version y of given system, not x
ChatGpt: <Apology> Try this.
Me: This is using a method that never existed in the framework.
ChatGpt: <Apology> <Gives first solution again>
mozz@mbin.grits.dev
on 24 May 2024 21:00
nextcollapse
"Oh, I see the problem. In order to correct (what went wrong with the last implementation), we can (complete code re-implementation which also doesn't work)"
Goto 1
Boozilla@lemmy.world
on 24 May 2024 21:01
nextcollapse
But only sometimes. Not often enough that I don't still find it more useful than not.
UberMentch@lemmy.world
on 24 May 2024 22:28
nextcollapse
I used to have this issue more often as well. I’ve had good results recently by **not ** pointing out mistakes in replies, but by going back to the message before GPT’s response and saying “do not include y.”
brbposting@sh.itjust.works
on 25 May 2024 00:14
collapse
Agreed, I send my first prompt, review the output, smack my head “obviously it couldn’t read my mind on that missing requirement”, and go back and edit the first prompt as if I really was a competent and clear communicator all along.
It’s actually not a bad strategy because it can make some adept assumptions that may have seemed pertinent to include, so instead of typing out every requirement you can think of, you speech-to-text* a half-assed prompt and then know exactly what to fix a few seconds later.
*[ad] free Ecco Dictate on iOS, TypingMind’s built-in dictation… anything using OpenAI Whisper, godly accuracy. btw TypingMind is great - stick in GPT-4o & Claude 3 Opus API keys and boom
BrianTheeBiscuiteer@lemmy.world
on 25 May 2024 12:37
collapse
While explaining BTRFS I’ve seen ChatGPT contradict itself in the middle of a paragraph. Then when I call it out it apologizes and then contradicts itself again with slightly different verbiage.
WalnutLum@lemmy.ml
on 24 May 2024 22:45
nextcollapse
This is because all LLMs function primarily based on the token context you feed it.
The best way to use any LLM is to completely fill up it’s history with relevant context, then ask your question.
Boozilla@lemmy.world
on 24 May 2024 23:04
collapse
I worked on a creative writing thing with it and the more I added, the better its responses. And 4 is a noticeable improvement over 3.5.
I was recently asked to make a small Android app using flutter, which I had never touched before
I used chatgpt at first and it was so painful to get correct answers, but then made an agent or whatever it’s called where I gave it instructions saying it was a flutter Dev and gave it a bunch of specifics about what I was working on
Suddenly it became really useful…I could throw it chunks of code and it would just straight away tell me where the error was and what I needed to change
I could ask it to write me an example method for something that I could then easily adapt for my use
One thing I would do would be ask it to write a method to do X, while I was writing the part that would use that method.
This wasn’t a big project and the whole thing took less than 40 hours, but for me to pick up a new language, setup the development environment, and make a working app for a specific task in 40 hours was a huge deal to me… I think without chatgpt, just learning all the basics and debugging would have taken more than 40 hours alone
homesweethomeMrL@lemmy.world
on 24 May 2024 20:56
nextcollapse
AI Defenders! Assemble!
BlameThePeacock@lemmy.ca
on 24 May 2024 21:33
collapse
No need to defend it.
Either it’s value is sufficient that businesses can make money by implementing it and it gets used, or it isn’t.
I’m personally already using it to make money, so I suspect it’s going to stick around.
floofloof@lemmy.ca
on 24 May 2024 20:58
nextcollapse
What’s especially troubling is that many human programmers seem to prefer the ChatGPT answers. The Purdue researchers polled 12 programmers — admittedly a small sample size — and found they preferred ChatGPT at a rate of 35 percent and didn’t catch AI-generated mistakes at 39 percent.
Why is this happening? It might just be that ChatGPT is more polite than people online.
It’s probably more because you can ask it your exact question (not just search for something more or less similar) and it will at least give you a lead that you can use to discover the answer, even if it doesn’t give you a perfect answer.
Also, who does a survey of 12 people and publishes the results? Is that normal?
Even this Lemmy thread has more participants than the survey
brbposting@sh.itjust.works
on 25 May 2024 00:17
collapse
I have 13 friends who are researchers and they publish surveys like that all the time.
(You can trust this comment because I peer reviewed it.)
paddirn@lemmy.world
on 24 May 2024 21:00
nextcollapse
I wonder if the AI is using bad code pulled from threads where people are asking questions about why their code isn’t working, but ChatGPT can’t tell the difference and just assumes all code is good code.
sturlabragason@lemmy.world
on 24 May 2024 21:13
nextcollapse
For someone doing a study on LLM they don’t seem to know much about LLMs.
They don’t even mention which model was used…
Here’s the study used for this clickbait garbage :
Veraxus@lemmy.world
on 24 May 2024 21:17
nextcollapse
I’m surprised it scores that well.
Well, ok… that seems about right for languages like JavaScript or Python, but try it on languages with a reputation for being widely used to write terrible code, like Java or PHP (hence having been trained on terrible code), and it’s actively detrimental to even experienced developers.
eerongal@ttrpg.network
on 24 May 2024 21:22
nextcollapse
Worth noting this study was done on gpt 3.5, 4 is leagues better than 3.5. I’d be interested to see how this number has changed
zero_spelled_with_an_ecks@programming.dev
on 24 May 2024 23:46
nextcollapse
4 made up functions that didn’t exist last time I asked in a programming question.
FaceDeer@fedia.io
on 25 May 2024 01:11
nextcollapse
This is why I like Bing Chat for this kind of thing, it does a web search in the background and will often be working right from the API documentation.
eerongal@ttrpg.network
on 25 May 2024 02:23
collapse
sure, I’m not saying GPT4 is perfect, just that it’s known to be a lot better than 3.5. Kinda why I would be interested to see how much better it actually is.
There is huge gap between 3.5 and 4 especially in coding related questions. GPT3.5 does not have large enough token size to handle harder code related questions.
Melkath@kbin.social
on 24 May 2024 21:43
nextcollapse
Developing with ChatGPT feels bizzarely like when Tony Stark invented a new element with Jarvis' assistance.
It's a prolonged back and forth, and you need to point out the AIs mistakes and work through a ton of iterations to get something that is close enough that you can tweak it and use, but it's SO much faster than trawling through Stack Overflow or hoping someone who knows more than you can answer a post for you.
elgordio@kbin.social
on 24 May 2024 22:10
collapse
Yeah if you treat it is a junior engineer, with the ability to instantly research a topic, and are prepared to engage in a conversation to work toward a working answer, then it can work extremely well.
Some of the best outcomes I’ve had have needed 20+ prompts, but I still arrived at a solution faster than any other method.
In the end, there is this great fear of "the AI is going to fully replace us developers" and the reality is that while that may be a possibility one day, it wont be any day soon.
You still need people with deep technical knowledge to pilot the AI and drive it to an implemented solution.
AI isnt the end of the industry, it has just greatly sped up the industry.
haui_lemmy@lemmy.giftedmc.com
on 24 May 2024 21:52
nextcollapse
The interesting bit for me is that if you ask a rando some programming questions they will be 99% wrong on average I think.
Stack overflow still makes more sense though.
Epzillon@lemmy.ml
on 24 May 2024 21:58
nextcollapse
I worked for a year developing in Magento 2 (an open source e-commerce suite which was later bought up by Adobe, it is not well maintained and it just all around not nice to work with). I tried to ask some Magento 2 questions to ChatGPT to figure out some solutions to my problems but clearly the only data it was trained with was a lot of really bad solutions from forum posts.
The solutions did kinda work some of the times but the way it was suggesting it was absolutely horrifying. We’re talking opening so many vulnerabilites, breaking many parts of the suite as a whole or just editing database tables. If you do not know enough about the tools you are working with implementing solutions from ChatGPT can be disasterous, even if they end up working.
crossmr@kbin.social
on 24 May 2024 22:08
nextcollapse
The best method I've found for using it is to help you with languages you may have lost familiarity in and to walk it through what you need step by step. This lets you evaluate it's reasoning. When it gets stuck in a loop:
Try A!
Actually A doesn't work because that method doesn't exist.
Oh sorry Try B!
Yeah B doesn't work either.
You're right, so sorry about that, Try A!
Yeah.. we just did this.
at that point it's time to just close it down and try another AI.
And then agree with whatever you said, even if it was wrong.
gnuplusmatt@reddthat.com
on 24 May 2024 23:25
nextcollapse
I’ve used chatgpt and gemini to build some simple powershell scripts for use in intune deployments. They’ve been fairly simple scripts. Very few have of them have been workable solutions out of the box, and they’ve often filled with hallucinated cmdlets that don’t exist or are part of a thirdparty module that it doesn’t tell me needs to be installed. It’s not useless tho, because I am a lousy programmer its been good to give me a skeleton for which I can build a working script off of and debug myself.
I reiterate that I am a lousy programmer, but it has sped up my deployments because I haven’t had to work from scratch. 5/10 its saved me a half hour here and there.
I'm a good programmer and I still find LLMs to be great for banging out python scripts to handle one-off tasks. I usually use Copilot, it seems best for that sort of thing. Often the first version of the script will have a bug or misunderstanding in it, but all you need to do is tell the LLM what it did wrong or paste the text of the exception into the chat and it'll usually fix its own mistakes quite well.
I could write those scripts myself by hand if I wanted to, but they'd take a lot longer and I'd be spending my time on boring stuff. Why not let a machine do the boring stuff? That's why we have technology.
I’m a 10 year pro, and I’ve changed my workflows completely to include both chatgpt and copilot. I have found that for the mundane, simple, common patterns copilot’s accuracy is close to 9/10 correct, especially in my well maintained repos.
It seems like the accuracy of simple answers is directly proportional to the precision of my function and variable names.
I haven’t typed a full for loop in a year thanks to copilot, I treat it like an intent autocomplete.
Chatgpt on the other hand is remarkably useful for super well laid out questions, again with extreme precision in the terms you lay out. It has helped me in greenfield development with unique and insightful methodologies to accomplish tasks that would normally require extensive documentation searching.
Anyone who claims llms are a nothingburger is frankly wrong, with the right guidance my output has increased dramatically and my error rate has dropped slightly. I used to be able to put out about 1000 quality lines of change in a day (a poor metric, but a useful one) and my output has expanded to at least double that using the tools we have today.
Are LLMs miraculous? No, but they are incredibly powerful tools in the right hands.
If you're careless with your prompting, sure. The "default style" of ChatGPT is widely known at this point. If you want it to sound different you'll need to provide some context to tell it what you want it to sound like.
Or just use one of the many other LLMs out there to mix things up a bit. When I'm brainstorming I usually use Chatbot Arena to bounce ideas around, it's a page where you can send a prompt to two randomly-selected LLMs and then by voting on which gave a better response you help rank them on a leaderboard. This way I get to run my prompts through a lot of variety.
raspberriesareyummy@lemmy.world
on 25 May 2024 01:03
nextcollapse
I’m a 10 year pro,
You wish. The sheer idea of calling yourself a “pro” disqualifies you. People who actually code and know what they are doing wouldn’t dream of giving themselves a label beyond “coder” / “programmer” / “SW Dev”.
Because they don’t have to. You are a muppet.
Gsus4@mander.xyz
on 25 May 2024 02:14
nextcollapse
elon?
TrickDacy@lemmy.world
on 25 May 2024 02:37
nextcollapse
A lot of rage for a small amount of confidence
chiisana@lemmy.chiisana.net
on 25 May 2024 02:44
nextcollapse
Here we observe a pro gatekeeper in their natural habitat…
Hey! So you may have noticed that you got downvoted into oblivion here. It is because of the unnecessary amount of negativity in your comment.
In communication, there are two parts - how it is delivered, and how it is received. In this interaction, you clearly stated your point: giving yourself the title of pro oftentimes means the person is not a pro.
What they received, however, is far different. They received: ugh this sweaty asshole is gatekeeping coding.
If your goal was to convince this person not to call themselves a pro going forward, this may have been a failed communication event.
raspberriesareyummy@lemmy.world
on 27 May 2024 10:26
collapse
while your measured response is appreciated, I hardly consider a few dozen downvotes relevant, nor do I care in this case. It’s telling that those who did respond to my comment seem to assume I would consider myself a “pro” when that’s 1) nothing I said and 2) it should be clear from my comment that I consider the expression cringy. Outside memeable content, only idiots call themselves a “pro”. If something is my profession, I could see someone calling themselves a “professional <whatever>” (not that I would use it), but professional has a profoundly distinct ring to it, because it also refers to a code of conduct / a way to conduct business.
“I’m a pro” and anything like it is just hot air coming from bullshitters who are mostly responsible for enshittification of any given technology.
MajorHavoc@programming.dev
on 25 May 2024 01:25
nextcollapse
As a fellow pro, who has no issues calling myself a pro, because I am…
You’re spot on.
The stuff most people think AI is going to do - it’s not.
But as an insanely convenient auto-complete, modern LLMs absolutely shine!
TrickDacy@lemmy.world
on 25 May 2024 02:41
nextcollapse
Refreshing to see a reasonable response to coding with AI. Never used chatgpt for it but my copilot experience mirrors yours.
I find it shocking how many developers seem to think so many negative thoughts about it programming with AI. Some guy recently said “everyone in my shop finds it useless”. Hard for me to believe they actually tried copilot if they think that
nephs@lemmygrad.ml
on 25 May 2024 08:51
nextcollapse
Omg, I feel sorry for the people cleaning up after those codebases later. Maintaing that kind of careless “quality” lines of code is going to be a job for actual veterans.
And when we’re all retired or dead, the whole world will be a pile of alien artifacts from a time when people were still able to figure stuff out, and llms will still be ridiculously inefficient for precise tasks, just like today.
sylver_dragon@lemmy.world
on 25 May 2024 10:52
nextcollapse
I think AI is good with giving answers to well defined problems. The issue is that companies keep trying to throw it at poorly defined problems and the results are less useful. I work in the cybersecurity space and you can’t swing a dead cat without hitting a vendor talking about AI in their products. It’s the new, big marketing buzzword. The problem is that finding the bad stuff on a network is not a well defined problem. So instead, you get the unsupervised models faffing about, generating tons and tons of false positives. The only useful implementations of AI I’ve seen in these tools actually mirrors you own: they can be scary good at generating data queries from natural language prompts. Which is, once again, a well defined problem.
Overall, AI is a tool and used in the right way, it’s useful. It gets a bad rap because companies keep using it in bad ways and the end result can be worse than not having it at all.
In fairness, it’s possible that if 100 companies try seemingly bad ideas, 1 of them will turn out to be extremely profitable.
Specal@lemmy.world
on 25 May 2024 12:04
nextcollapse
I’ve found that the better I’ve gotten at writing prompts and giving enough information for it to not hallucinate, the better answers I get. It has to be treated as what it is, a calculator that can talk, make sure it has all of the information and it will find the answer.
One thing I have found to be super helpful with GPT4o is the ability to give it full API pages so it can update and familiarise it’s self with what it’s working with.
Anyone who claims llms are a nothingburger is frankly wrong,
Exactly. When someone says that it either indicates to me that they ignorant (like they aren’t a programmer or haven’t used it) or they are a programmer who has used it, but are not good at all at integrating new tools into their development process.
Don’t throw out the baby with the bathwater.
Yup. The problem I see now is that every mistake an ai makes is parroted over and over here and held up as an example of why the tech is garbage. But it’s cherry picking. Yes, they make mistakes, I often scratch my head at the ai results from Google and know to double check it. But the number of times it has pointed me in the right direction way faster than search results has shown to me already how useful it is.
cupcakezealot@lemmy.blahaj.zone
on 25 May 2024 00:26
nextcollapse
ill use copilot in place of most of the times ive searched on stackoverflow or to do mundane things like generate repeated things but relying solely on it is the same as relying solely on stackoverflow.
Max_P@lemmy.max-p.me
on 25 May 2024 01:20
nextcollapse
I don’t even bother trying with AI, it’s not been helpful to me a single time despite multiple attempts. That’s a 0% success rate for me.
finestnothing@lemmy.world
on 25 May 2024 02:04
nextcollapse
I use chatgpt semi-often… For generating stuff in a repeating pattern. Any time I have used it to make code, I don’t save any time because I have to debug most of the generated code anyway. My main use case lately is making python dicts with empty keys (e.g. key1, key2… becomes “key1”: “”, “key2”: “”,…) or making a gold/prod level SQL view by passing in the backend names and frontend names (e.g. value_1, value_2… Value 1, Value 2,… Becomes value_1 as Value 1,…).
ramirezmike@programming.dev
on 25 May 2024 03:15
collapse
I know this is gonna sound annoying but I just use vim for stuff like this. Even notepad++ has a macro thing too, right? My coworkers keep saying how much of a productivity boost it is but all I see it do is mess up stuff like this that only takes a few seconds in vim to setup and I know it’ll be correct every time
finestnothing@lemmy.world
on 25 May 2024 14:18
collapse
I use vim keybinds (via doom emacs) for this sort of stuff if I’m doing it for personal projects, my professional work is all done in an online platform (no way around it) so it’s just faster and easier to throw the pattern and columns at the integrated chatgpt terminal rather than hop to a local editor and back
Thcdenton@lemmy.world
on 25 May 2024 03:13
nextcollapse
It was pretty good for a while! They lowered the power of it like immortan joe. Do not be come addicted to AI
dullbananas@lemmy.ca
on 25 May 2024 03:15
collapse
If you become addicted to ChatGPT then that makes you a cloud cyborg
SpicyLizards@reddthat.com
on 25 May 2024 06:42
nextcollapse
I would make some 1000 monkeys with typewriters comment, but I see what most actual contracted devs produce…
dgmib@lemmy.world
on 25 May 2024 07:03
nextcollapse
Sometimes ChatGPT/copilot’s code predictions are scary good. Sometimes they’re batshit crazy. If you have the experience to be able to tell the difference, it’s a great help.
EatATaco@lemm.ee
on 25 May 2024 12:19
nextcollapse
Due to confusing business domain terms, we often name variables the form of XY and YX.
One time copilot autogenerated about two hundred lines of a class that was like. XY; YX; XXY; XYX; XYXY; … XXYYXYXYYYXYXYYXY;
It was pretty hilarious.
But that being said, it’s a great tool that has definitely proven to worth the cost…but like with a co-op, you have to check it’s work.
Potatos_are_not_friends@lemmy.world
on 25 May 2024 13:31
nextcollapse
Pretty much this. Experienced developers see AI just as a next level lorem Ipsum.
fossilesque@mander.xyz
on 25 May 2024 17:17
nextcollapse
I find the mistakes it makes and trouble shooting them really good for learning. I’m self taught.
The amount of reference material it has is also a big influence. I’ve had to pick up PLC programming a while ago (codesys/structured text, which is kinda based on pascal). While chatgpt understands the syntax it has absolutely no clue about libraries and platform limitations so it keeps hallucinating those based on popular ones in other languages.
Still a great tool to have it fill out things like I/O mappings and the sorts. Just need to give it some examples to work with first.
Evotech@lemmy.world
on 25 May 2024 08:55
nextcollapse
Probably more than 52% of what programmers type is wrong too
lurch@sh.itjust.works
on 25 May 2024 13:16
collapse
The one time it was helpful at work was when I used it to thank and wish a person well that left a company we work with. I couldn’t come up with a good response and ChatGPT just spat real good stuff out in seconds. This is what it’s really good for.
grrgyle@slrpnk.net
on 25 May 2024 14:37
nextcollapse
Yeah things that follow a kind of lexical “script” that you don’t want to get creative with would be pretty easy to generate. Farewells, greetings, dear Johns, may he rest in peaces, etc etc
ChatGPT: I’m happy for you though, Or sorry that happened
bigkahuna1986@lemmy.ml
on 25 May 2024 22:11
collapse
ChatGPT just spat real good stuff out in seconds
There’s an entire episode of south park centered around this premise.
Samueru@lemmy.ml
on 25 May 2024 09:36
nextcollapse
I find it funny that thumbnail with a “fail” I’m actually surprised that it got 48% right.
ulterno@lemmy.kde.social
on 25 May 2024 11:24
nextcollapse
You forgot the “at least” before the 52%.
Crisps@lemmy.world
on 25 May 2024 13:05
nextcollapse
In the short term it really helps productivity, but in the end the reward for working faster is more work. Just doing the hard parts all day is going to burn developers out.
I program for a living and I think of it more as doing the interesting tasks all day, rather than the mundane and repetitive. Chat GPT and GitHub Copilot are great for getting something roughly right that you can tweak to work the way you want.
cows_are_underrated@discuss.tchncs.de
on 26 May 2024 15:06
collapse
I think we must change the way we see AI. A lot of people see it as the holy grail of everything that can do everything we can do, even tho it can’t. AI is a tool for humans to become more efficient in their work. It can do easy tasks for you and sometimes Assist you with harder stuff. It is the same as with Mathematicians and calculators. A good mathematician is able to calculate everytheverything he needs without a calculator, but the calculator makes him much more efficient at calculating stuff. The calculator didn’t replace mathematicians, because you still have to know how to do the stiff you’re doing.
sirico@feddit.uk
on 25 May 2024 16:05
nextcollapse
It’s pretty fun, interesting times ahead. I wonder what kind of bullshit will take place and can’t wait to see that lol. Between all the climate, ai, warmongering future won’t be boring guys that is certain. Unpack your popcorn
Now non-coders can finally wield the foot-gun once reserved only for coders! /s
Truth be told, computer engineering should really be something that one needs a licence to do commercially, just like regular engineering. In this modern era where software can be ruinous to someone’s life just like shoddy engineering, why is it not like this already.
Look, nothing will blow up if I mess up my proxy setup on my machine. I just won’t have internet until I revert my change. Why would that be different if I were getting paid for it?
I don’t see how this supports your point then. If “setting up proxy” means “packaging it to run on thousands user machines” then isn’t there obvious and huge potential for a disastrous fuckup?
No, because it either runs the program successfully, or it fails to launch. I don’t mess with the protocol. It runs as root because it needs to set the iptables when turned on to be a “global” proxy
cows_are_underrated@discuss.tchncs.de
on 26 May 2024 15:01
collapse
Nothing happens if you fuck up your proxy, but if you develop an app that gets very popular and don’t care about safety, so hackers are able to take control over your whole Server they can do a lot of damage. If you develop software for critical infrastructure it can actually cost human lives if you fuck up your security systems.
Is it, though? A lot of self-taught programmers do great work. I’m not sure this is true
InvaderDJ@lemmy.world
on 26 May 2024 17:00
nextcollapse
You can also play with it to try and get closer to correct. I had problems with getting an Excel macro working and getting unattended-updates working on my pihole. GPT was wrong at first, but got me partly there and I could massage the question and Google and get closer to the right answer. Without it, I wouldn’t have been able to get any of it, especially with the macro.
Exactly, I also find that it tends to do a pretty good job pointing you in the right direction. It’s way faster than googling or going through sites like stackoverflow because the answers are contextual. You can ask about a specific thing you want to do, and and an answer that gives you a general idea of what to do. For example, I’ve found it to be great for crafting complex sql queries. I don’t really care if the answer is perfect, as long as it gives me an idea of what I need to do.
slacktoid@lemmy.ml
on 25 May 2024 21:04
nextcollapse
We need a comparison against an average coder. Some fucking baseline ffs.
hayes_@sh.itjust.works
on 25 May 2024 21:18
nextcollapse
Why would we compare it against an average coder?
ChatGPT wants to be a coding aid/reference material. A better baseline would be the top rated answer for the question on stackoverflow or whether the answer exists on the first 3 Google search results.
BleatingZombie@lemmy.world
on 25 May 2024 21:53
nextcollapse
threaded - newest
You should see 52% of the first version of my code.
It doesn’t have to be right to be useful.
Yeah, but the non-tech savvy business leaders see they can generate code with AI and think ‘why do I need a developer if I have this AI?’ and have no idea whether the code it produces is right or not. This stat should be shared broadly so leaders don’t overestimate the capability and fire people they will desperately need.
Yeah management are all for this, the first few years here are rough with them immediately hitting the “fire the engineers we have ai now”. They won’t realize their fuckup until they’ve been promoted away from it
Programming jobs will be safe for a while. They’ve been trying to eliminate those positions since at least the 90s. Because coders are expensive and often lack social skills.
But I do think the clock is ticking. We will see more and more sophisticated AI tools that are relatively idiot-proof and can do things like modify Salesforce, or create complex new Tableau reports with a few mouse clicks, and stuff like that. Jobs will be chiseled away like our unfortunate friends in graphic design.
You, along with most people, are still looking at automation wrong. It’s never been about removing people entirely, even AI, it’s about doing the same work with less cost.
If you can eliminate one programmers from your four person team by giving the other three AI to produce the same amount of work, congrats you’ve just automated one programming job.
Programming jobs aren’t going anywhere, but either the amount of code produced is about to skyrocket, or the number of employed programmers is going to drop (or most likely both of those things).
Right on. AI feels like a looming paradigm shift in our field that we can either scoff at for its flaws or start learning how to exploit for our benefit. As long as it ends up boosting productivity it’s probably something we’re going to have to learn to work with for job security.
It’s already boosting productivity in many roles. That’s just going to accelerate as the models get better, the processing gets cheaper, and (as you said) people learn to use it better.
I wonder if this will also have a reverse tail end effect.
Company uses AI (with devs) to produce a large amount of code -> code is in prod for a few years with incremental changes -> dev roles rotate or get further reduced over time -> company now needs to modernize and change very large legacy codebase that nobody really understands well enough to even feed it Into the AI -> now hiring more devs than before to figure out how to manage a legacy codebase 5-10x the size of what the team could realistically handle.
Writing greenfield code is relatively easy, maintaining it over years and keeping it up to date and well understood while twisting it for all new requirements - now that’s hard.
AI will help with that too, it’s going to be able to process entire codebases at a time pretty shortly here.
Given the visual capabilities now emerging, it can likely also do human-equivalent testing.
One of the biggest AI tricks we haven’t started seeing much of yet in mainstream use is this kind of automated double-checking. Where it generates an answer, and then validates if the answer is valid before actually giving it to a human. Especially in coding bases, there really isn’t anything stopping it from coming up with an answer compiling, running into an error, re-generating, and repeating until the code passes all unit tests or even potentially visual inspection.
The big limit on this right now is sheer processing cost and context lengths for the models. However, costs for this are dropping faster than any new tech we’ve seen, and it will likely be trivial in just a few years.
There are some areas I’m hoping get addressed by the coming skyrocket in programmer productivity:
I think that last one stands to be strongly enabled by AI code assist tools. It might not be the sexiest or highest paying job, but it’ll be work that matters that largely isn’t even being done today.
I say let it happen. If someone is dumb enough to fire all their workers… They deserve what will happen next
Well the firing’s happening so, i guess let’s hope you’re right about the other part.
It won’t happen like that. Leadership will just under-hire and expect all their developers to be way more efficient. Working will be really stressful with increased deadlines and people questioning why you couldn’t meet them.
Mentioned it before but:
LLMs program at the level of a junior engineer or an intern. You already need code review and more senior engineers to fix that shit for them.
What they do is migrate that. Now that junior engineer has an intern they are trying to work with. Or… companies realize they don’t benefit from training up those newbie (or stupid) engineers when they are likely to leave in a year or two anyway.
And they’ll find out very soon that they need devs when they actually try to test something and nothing works.
Yeah cause my favorite thing to do when programming is debugging someone else’s broken code.
I think where it shines is in helping you write code you’ve never written before. I never touched Swift before and I made a fully functional iOS app in a week. Also, even with stuff I have done before, I can say “write me a function that does x” and it will and it usually works.
Like just yesterday I asked it to write me a function that would generate and serve up an .ics file based on a selected date and extrapolate the date of a recurring monthly meeting based on the day of the week picked and its position (1st week, 2nd week, etc) within the month and then make the .ics file reflect all that. I could have generated that code myself by hand but it would have probably taken me an hour or two. It did it in about five seconds and it worked perfectly.
Yeah, you have to know what you’re doing in general and there’s a lot of babysitting involved, but anyone who thinks it’s just useless is plain wrong. It’s fucking amazing.
Edit: lol the article is referring to a study that was using GPT 3.5, which is all but useless for coding. 4.0 has been out for a year blowing everybody’s minds. Clickbait trash.
3.5 is still reasonably useful for the same reasons you described, imo… Just less so.
Get it to debug itself then.
Yeah I’ve already got enough legacy code to deal with, I don’t need more of it faster.
To be fair, I’m starting to fear that all the fun bits of human jobs are the ones that are most easy to automate.
I dread the day I’m stuck playing project manager to a bunch of chat bots.
Generally you want to the reference material used to improve that first version to be correct though. Otherwise it’s just swapping one problem for another.
I wouldn’t use a textbook that was 52% incorrect, the same should apply to a chatbot.
Bad take. Is the first version of your code the one that you deliver or push upstream?
LLMs can give great starting points, I use multiple LLMs each for various reasons. Usually to clean up something I wrote (too lazy or too busy/stressed to do manually), find a problem with the logic, or maybe even brainstorm ideas.
I rarely ever use it to generate blocks of code like asking it to generate “a method that takes X inputs and does Y operations, and returns Z value”. I find that those kinds of results are often vastly wrong or just done in a way that doesn’t fit with other things I’m doing.
Impressed some folks think LLMs are useless. Not sure if their lives/workflows/brains are that different from ours or they haven’t given at the college try.
I almost always have to use my head before a language model’s output is useful for a given purpose. The tool almost always saves me time, improves the end result, or both. Usually both, I would say.
It’s a very dangerous technology that is known to output utter garbage and make enormous mistakes. Still, it routinely blows my mind.
This is the best summary I could come up with:
In recent years, computer programmers have flocked to chatbots like OpenAI’s ChatGPT to help them code, dealing a blow to places like Stack Overflow, which had to lay off nearly 30 percent of its staff last year.
That’s a staggeringly large proportion for a program that people are relying on to be accurate and precise, underlining what other end users like writers and teachers are experiencing: AI platforms like ChatGPT often hallucinate totally incorrectly answers out of thin air.
For the study, the researchers looked over 517 questions in Stack Overflow and analyzed ChatGPT’s attempt to answer them.
The team also performed a linguistic analysis of 2,000 randomly selected ChatGPT answers and found they were “more formal and analytical” while portraying “less negative sentiment” — the sort of bland and cheery tone AI tends to produce.
The Purdue researchers polled 12 programmers — admittedly a small sample size — and found they preferred ChatGPT at a rate of 35 percent and didn’t catch AI-generated mistakes at 39 percent.
The study demonstrates that ChatGPT still has major flaws — but that’s cold comfort to people laid off from Stack Overflow or programmers who have to fix AI-generated mistakes in code.
The original article contains 340 words, the summary contains 199 words. Saved 41%. I’m a bot and I’m open source!
It’s been a tremendous help to me as I relearn how to code on some personal projects. I have written 5 little apps that are very useful to me for my hobbies.
It’s also been helpful at work with some random database type stuff.
But it definitely gets stuff wrong. A lot of stuff.
The funny thing is, if you point out its mistakes, it often does better on subsequent attempts. It’s more like an iterative process of refinement than one prompt gives you the final answer.
It’s incredibly useful for learning. ChatGPT was what taught me to unlearn, essentially, writing C in every language, and how to write idiomatic Python and JavaScript.
It is very good for boilerplate code or fleshing out a big module without you having to do the typing. My experience was just like yours; once you’re past a certain (not real high) level of complexity you’re looking at multiple rounds of improvement or else just doing it yourself.
Exactly. And for me, being in middle age, it’s a big help with recalling syntax. I generally know how to do stuff, but need a little refresher on the spelling, parameters, etc.
Personally I find all LLMs in general not that great at writing larger blocks of code. It’s fine for smaller stuff, but the more you expect out of it the more it’ll get wrong.
I find they work best with existing stuff that you provide. Like “make this block of code more efficient” or “rewrite this function to do X”.
Or it get stuck in an endless loop of two different but wrong solutions.
Me: This is my system, version x. I want to achieve this.
ChatGpt: Here’s the solution.
Me: But this only works with Version y of given system, not x
ChatGpt: <Apology> Try this.
Me: This is using a method that never existed in the framework.
ChatGpt: <Apology> <Gives first solution again>
Ha! That definitely happens sometimes, too.
But only sometimes. Not often enough that I don't still find it more useful than not.
I used to have this issue more often as well. I’ve had good results recently by **not ** pointing out mistakes in replies, but by going back to the message before GPT’s response and saying “do not include y.”
Agreed, I send my first prompt, review the output, smack my head “obviously it couldn’t read my mind on that missing requirement”, and go back and edit the first prompt as if I really was a competent and clear communicator all along.
It’s actually not a bad strategy because it can make some adept assumptions that may have seemed pertinent to include, so instead of typing out every requirement you can think of, you speech-to-text* a half-assed prompt and then know exactly what to fix a few seconds later.
*[ad] free Ecco Dictate on iOS, TypingMind’s built-in dictation… anything using OpenAI Whisper, godly accuracy. btw TypingMind is great - stick in GPT-4o & Claude 3 Opus API keys and boom
While explaining BTRFS I’ve seen ChatGPT contradict itself in the middle of a paragraph. Then when I call it out it apologizes and then contradicts itself again with slightly different verbiage.
This is because all LLMs function primarily based on the token context you feed it.
The best way to use any LLM is to completely fill up it’s history with relevant context, then ask your question.
I worked on a creative writing thing with it and the more I added, the better its responses. And 4 is a noticeable improvement over 3.5.
I was recently asked to make a small Android app using flutter, which I had never touched before
I used chatgpt at first and it was so painful to get correct answers, but then made an agent or whatever it’s called where I gave it instructions saying it was a flutter Dev and gave it a bunch of specifics about what I was working on
Suddenly it became really useful…I could throw it chunks of code and it would just straight away tell me where the error was and what I needed to change
I could ask it to write me an example method for something that I could then easily adapt for my use
One thing I would do would be ask it to write a method to do X, while I was writing the part that would use that method.
This wasn’t a big project and the whole thing took less than 40 hours, but for me to pick up a new language, setup the development environment, and make a working app for a specific task in 40 hours was a huge deal to me… I think without chatgpt, just learning all the basics and debugging would have taken more than 40 hours alone
AI Defenders! Assemble!
No need to defend it.
Either it’s value is sufficient that businesses can make money by implementing it and it gets used, or it isn’t.
I’m personally already using it to make money, so I suspect it’s going to stick around.
It’s probably more because you can ask it your exact question (not just search for something more or less similar) and it will at least give you a lead that you can use to discover the answer, even if it doesn’t give you a perfect answer.
Also, who does a survey of 12 people and publishes the results? Is that normal?
Even this Lemmy thread has more participants than the survey
I have 13 friends who are researchers and they publish surveys like that all the time.
(You can trust this comment because I peer reviewed it.)
I wonder if the AI is using bad code pulled from threads where people are asking questions about why their code isn’t working, but ChatGPT can’t tell the difference and just assumes all code is good code.
For someone doing a study on LLM they don’t seem to know much about LLMs.
They don’t even mention which model was used…
Here’s the study used for this clickbait garbage :
dl.acm.org/doi/pdf/10.1145/3613904.3642596
I’m surprised it scores that well.
Well, ok… that seems about right for languages like JavaScript or Python, but try it on languages with a reputation for being widely used to write terrible code, like Java or PHP (hence having been trained on terrible code), and it’s actively detrimental to even experienced developers.
Worth noting this study was done on gpt 3.5, 4 is leagues better than 3.5. I’d be interested to see how this number has changed
4 made up functions that didn’t exist last time I asked in a programming question.
This is why I like Bing Chat for this kind of thing, it does a web search in the background and will often be working right from the API documentation.
sure, I’m not saying GPT4 is perfect, just that it’s known to be a lot better than 3.5. Kinda why I would be interested to see how much better it actually is.
There is huge gap between 3.5 and 4 especially in coding related questions. GPT3.5 does not have large enough token size to handle harder code related questions.
Developing with ChatGPT feels bizzarely like when Tony Stark invented a new element with Jarvis' assistance.
It's a prolonged back and forth, and you need to point out the AIs mistakes and work through a ton of iterations to get something that is close enough that you can tweak it and use, but it's SO much faster than trawling through Stack Overflow or hoping someone who knows more than you can answer a post for you.
Yeah if you treat it is a junior engineer, with the ability to instantly research a topic, and are prepared to engage in a conversation to work toward a working answer, then it can work extremely well.
Some of the best outcomes I’ve had have needed 20+ prompts, but I still arrived at a solution faster than any other method.
In the end, there is this great fear of "the AI is going to fully replace us developers" and the reality is that while that may be a possibility one day, it wont be any day soon.
You still need people with deep technical knowledge to pilot the AI and drive it to an implemented solution.
AI isnt the end of the industry, it has just greatly sped up the industry.
The interesting bit for me is that if you ask a rando some programming questions they will be 99% wrong on average I think.
Stack overflow still makes more sense though.
I worked for a year developing in Magento 2 (an open source e-commerce suite which was later bought up by Adobe, it is not well maintained and it just all around not nice to work with). I tried to ask some Magento 2 questions to ChatGPT to figure out some solutions to my problems but clearly the only data it was trained with was a lot of really bad solutions from forum posts.
The solutions did kinda work some of the times but the way it was suggesting it was absolutely horrifying. We’re talking opening so many vulnerabilites, breaking many parts of the suite as a whole or just editing database tables. If you do not know enough about the tools you are working with implementing solutions from ChatGPT can be disasterous, even if they end up working.
The best method I've found for using it is to help you with languages you may have lost familiarity in and to walk it through what you need step by step. This lets you evaluate it's reasoning. When it gets stuck in a loop:
Try A!
Actually A doesn't work because that method doesn't exist.
Oh sorry Try B!
Yeah B doesn't work either.
You're right, so sorry about that, Try A!
Yeah.. we just did this.
at that point it's time to just close it down and try another AI.
For the upteenth time - an llm just puts words together, it isn’t a magic answer machine.
Yeah but it’s just going to get better at magicking. Soon all us wizards will be out of a job…
Just as soon as we no longer need to drive.
Self driving cars need to convince regulators that they’re safe enough, even if assuming they master the tech.
LLMs has already convinced our bosses that we are expendable, and can drastically reduce cost centres for their next earnings call.
A parrot blabbing the theory of relativity doesn’t make it Einstein.
Ask “are you sure?” and it will apologize right away.
And then agree with whatever you said, even if it was wrong.
I’ve used chatgpt and gemini to build some simple powershell scripts for use in intune deployments. They’ve been fairly simple scripts. Very few have of them have been workable solutions out of the box, and they’ve often filled with hallucinated cmdlets that don’t exist or are part of a thirdparty module that it doesn’t tell me needs to be installed. It’s not useless tho, because I am a lousy programmer its been good to give me a skeleton for which I can build a working script off of and debug myself.
I reiterate that I am a lousy programmer, but it has sped up my deployments because I haven’t had to work from scratch. 5/10 its saved me a half hour here and there.
I'm a good programmer and I still find LLMs to be great for banging out python scripts to handle one-off tasks. I usually use Copilot, it seems best for that sort of thing. Often the first version of the script will have a bug or misunderstanding in it, but all you need to do is tell the LLM what it did wrong or paste the text of the exception into the chat and it'll usually fix its own mistakes quite well.
I could write those scripts myself by hand if I wanted to, but they'd take a lot longer and I'd be spending my time on boring stuff. Why not let a machine do the boring stuff? That's why we have technology.
I’m a 10 year pro, and I’ve changed my workflows completely to include both chatgpt and copilot. I have found that for the mundane, simple, common patterns copilot’s accuracy is close to 9/10 correct, especially in my well maintained repos.
It seems like the accuracy of simple answers is directly proportional to the precision of my function and variable names.
I haven’t typed a full for loop in a year thanks to copilot, I treat it like an intent autocomplete.
Chatgpt on the other hand is remarkably useful for super well laid out questions, again with extreme precision in the terms you lay out. It has helped me in greenfield development with unique and insightful methodologies to accomplish tasks that would normally require extensive documentation searching.
Anyone who claims llms are a nothingburger is frankly wrong, with the right guidance my output has increased dramatically and my error rate has dropped slightly. I used to be able to put out about 1000 quality lines of change in a day (a poor metric, but a useful one) and my output has expanded to at least double that using the tools we have today.
Are LLMs miraculous? No, but they are incredibly powerful tools in the right hands.
Don’t throw out the baby with the bathwater.
On the other hand, using ChatGPT for your Lemmy comments sticks out like a sore thumb
If you're careless with your prompting, sure. The "default style" of ChatGPT is widely known at this point. If you want it to sound different you'll need to provide some context to tell it what you want it to sound like.
Or just use one of the many other LLMs out there to mix things up a bit. When I'm brainstorming I usually use Chatbot Arena to bounce ideas around, it's a page where you can send a prompt to two randomly-selected LLMs and then by voting on which gave a better response you help rank them on a leaderboard. This way I get to run my prompts through a lot of variety.
You wish. The sheer idea of calling yourself a “pro” disqualifies you. People who actually code and know what they are doing wouldn’t dream of giving themselves a label beyond “coder” / “programmer” / “SW Dev”. Because they don’t have to. You are a muppet.
elon?
A lot of rage for a small amount of confidence
Here we observe a pro gatekeeper in their natural habitat…
Hey! So you may have noticed that you got downvoted into oblivion here. It is because of the unnecessary amount of negativity in your comment.
In communication, there are two parts - how it is delivered, and how it is received. In this interaction, you clearly stated your point: giving yourself the title of pro oftentimes means the person is not a pro.
What they received, however, is far different. They received: ugh this sweaty asshole is gatekeeping coding.
If your goal was to convince this person not to call themselves a pro going forward, this may have been a failed communication event.
while your measured response is appreciated, I hardly consider a few dozen downvotes relevant, nor do I care in this case. It’s telling that those who did respond to my comment seem to assume I would consider myself a “pro” when that’s 1) nothing I said and 2) it should be clear from my comment that I consider the expression cringy. Outside memeable content, only idiots call themselves a “pro”. If something is my profession, I could see someone calling themselves a “professional <whatever>” (not that I would use it), but professional has a profoundly distinct ring to it, because it also refers to a code of conduct / a way to conduct business.
“I’m a pro” and anything like it is just hot air coming from bullshitters who are mostly responsible for enshittification of any given technology.
As a fellow pro, who has no issues calling myself a pro, because I am…
You’re spot on.
The stuff most people think AI is going to do - it’s not.
But as an insanely convenient auto-complete, modern LLMs absolutely shine!
Refreshing to see a reasonable response to coding with AI. Never used chatgpt for it but my copilot experience mirrors yours.
I find it shocking how many developers seem to think so many negative thoughts about it programming with AI. Some guy recently said “everyone in my shop finds it useless”. Hard for me to believe they actually tried copilot if they think that
Omg, I feel sorry for the people cleaning up after those codebases later. Maintaing that kind of careless “quality” lines of code is going to be a job for actual veterans.
And when we’re all retired or dead, the whole world will be a pile of alien artifacts from a time when people were still able to figure stuff out, and llms will still be ridiculously inefficient for precise tasks, just like today.
youtu.be/dDUC-LqVrPU
Here is an alternative Piped link(s):
https://piped.video/dDUC-LqVrPU
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I’m open-source; check me out at GitHub.
I think AI is good with giving answers to well defined problems. The issue is that companies keep trying to throw it at poorly defined problems and the results are less useful. I work in the cybersecurity space and you can’t swing a dead cat without hitting a vendor talking about AI in their products. It’s the new, big marketing buzzword. The problem is that finding the bad stuff on a network is not a well defined problem. So instead, you get the unsupervised models faffing about, generating tons and tons of false positives. The only useful implementations of AI I’ve seen in these tools actually mirrors you own: they can be scary good at generating data queries from natural language prompts. Which is, once again, a well defined problem.
Overall, AI is a tool and used in the right way, it’s useful. It gets a bad rap because companies keep using it in bad ways and the end result can be worse than not having it at all.
In fairness, it’s possible that if 100 companies try seemingly bad ideas, 1 of them will turn out to be extremely profitable.
I’ve found that the better I’ve gotten at writing prompts and giving enough information for it to not hallucinate, the better answers I get. It has to be treated as what it is, a calculator that can talk, make sure it has all of the information and it will find the answer.
One thing I have found to be super helpful with GPT4o is the ability to give it full API pages so it can update and familiarise it’s self with what it’s working with.
Exactly. When someone says that it either indicates to me that they ignorant (like they aren’t a programmer or haven’t used it) or they are a programmer who has used it, but are not good at all at integrating new tools into their development process.
Yup. The problem I see now is that every mistake an ai makes is parroted over and over here and held up as an example of why the tech is garbage. But it’s cherry picking. Yes, they make mistakes, I often scratch my head at the ai results from Google and know to double check it. But the number of times it has pointed me in the right direction way faster than search results has shown to me already how useful it is.
ill use copilot in place of most of the times ive searched on stackoverflow or to do mundane things like generate repeated things but relying solely on it is the same as relying solely on stackoverflow.
I don’t even bother trying with AI, it’s not been helpful to me a single time despite multiple attempts. That’s a 0% success rate for me.
I use chatgpt semi-often… For generating stuff in a repeating pattern. Any time I have used it to make code, I don’t save any time because I have to debug most of the generated code anyway. My main use case lately is making python dicts with empty keys (e.g. key1, key2… becomes “key1”: “”, “key2”: “”,…) or making a gold/prod level SQL view by passing in the backend names and frontend names (e.g. value_1, value_2…
Value 1,Value 2,… Becomes value_1 asValue 1,…).I know this is gonna sound annoying but I just use vim for stuff like this. Even notepad++ has a macro thing too, right? My coworkers keep saying how much of a productivity boost it is but all I see it do is mess up stuff like this that only takes a few seconds in vim to setup and I know it’ll be correct every time
I use vim keybinds (via doom emacs) for this sort of stuff if I’m doing it for personal projects, my professional work is all done in an online platform (no way around it) so it’s just faster and easier to throw the pattern and columns at the integrated chatgpt terminal rather than hop to a local editor and back
It was pretty good for a while! They lowered the power of it like immortan joe. Do not be come addicted to AI
If you become addicted to ChatGPT then that makes you a cloud cyborg
I would make some 1000 monkeys with typewriters comment, but I see what most actual contracted devs produce…
Sometimes ChatGPT/copilot’s code predictions are scary good. Sometimes they’re batshit crazy. If you have the experience to be able to tell the difference, it’s a great help.
Due to confusing business domain terms, we often name variables the form of XY and YX.
One time copilot autogenerated about two hundred lines of a class that was like. XY; YX; XXY; XYX; XYXY; … XXYYXYXYYYXYXYYXY;
It was pretty hilarious.
But that being said, it’s a great tool that has definitely proven to worth the cost…but like with a co-op, you have to check it’s work.
Pretty much this. Experienced developers see AI just as a next level lorem Ipsum.
I find the mistakes it makes and trouble shooting them really good for learning. I’m self taught.
The amount of reference material it has is also a big influence. I’ve had to pick up PLC programming a while ago (codesys/structured text, which is kinda based on pascal). While chatgpt understands the syntax it has absolutely no clue about libraries and platform limitations so it keeps hallucinating those based on popular ones in other languages.
Still a great tool to have it fill out things like I/O mappings and the sorts. Just need to give it some examples to work with first.
Probably more than 52% of what programmers type is wrong too
We mostly suck in emails.
The one time it was helpful at work was when I used it to thank and wish a person well that left a company we work with. I couldn’t come up with a good response and ChatGPT just spat real good stuff out in seconds. This is what it’s really good for.
Yeah things that follow a kind of lexical “script” that you don’t want to get creative with would be pretty easy to generate. Farewells, greetings, dear Johns, may he rest in peaces, etc etc
ChatGPT: I’m happy for you though, Or sorry that happened
There’s an entire episode of south park centered around this premise.
I find it funny that thumbnail with a “fail” I’m actually surprised that it got 48% right.
You forgot the “at least” before the 52%.
In the short term it really helps productivity, but in the end the reward for working faster is more work. Just doing the hard parts all day is going to burn developers out.
I program for a living and I think of it more as doing the interesting tasks all day, rather than the mundane and repetitive. Chat GPT and GitHub Copilot are great for getting something roughly right that you can tweak to work the way you want.
I think we must change the way we see AI. A lot of people see it as the holy grail of everything that can do everything we can do, even tho it can’t. AI is a tool for humans to become more efficient in their work. It can do easy tasks for you and sometimes Assist you with harder stuff. It is the same as with Mathematicians and calculators. A good mathematician is able to calculate everytheverything he needs without a calculator, but the calculator makes him much more efficient at calculating stuff. The calculator didn’t replace mathematicians, because you still have to know how to do the stiff you’re doing.
It’s programming spell check
Sure, but by randomly guessing code you’d get 0%. Getting 48% right is actually very impressive for an LLM compared to just a few years ago.
Just useful enough to become incredibly dangerous to anyone who doesn’t know what they’re doing. Isn’t it great?
It’s pretty fun, interesting times ahead. I wonder what kind of bullshit will take place and can’t wait to see that lol. Between all the climate, ai, warmongering future won’t be boring guys that is certain. Unpack your popcorn
Now non-coders can finally wield the foot-gun once reserved only for coders! /s
Truth be told, computer engineering should really be something that one needs a licence to do commercially, just like regular engineering. In this modern era where software can be ruinous to someone’s life just like shoddy engineering, why is it not like this already.
Look, nothing will blow up if I mess up my proxy setup on my machine. I just won’t have internet until I revert my change. Why would that be different if I were getting paid for it?
Setting up proxy is not engineering.
I have to actually modify the code to properly package it for my distro, so it’s engineering because I have to make decisions for how things work
I don’t see how this supports your point then. If “setting up proxy” means “packaging it to run on thousands user machines” then isn’t there obvious and huge potential for a disastrous fuckup?
No, because it either runs the program successfully, or it fails to launch. I don’t mess with the protocol. It runs as root because it needs to set the iptables when turned on to be a “global” proxy
Nothing happens if you fuck up your proxy, but if you develop an app that gets very popular and don’t care about safety, so hackers are able to take control over your whole Server they can do a lot of damage. If you develop software for critical infrastructure it can actually cost human lives if you fuck up your security systems.
Yes, but people with master’s degrees also fuck this up, so it’s not like some accreditation system will solve the issue of people making mistakes
Yeah, but its probably more likely that the untaught might fuck up some stuff.
Is it, though? A lot of self-taught programmers do great work. I’m not sure this is true
You can also play with it to try and get closer to correct. I had problems with getting an Excel macro working and getting unattended-updates working on my pihole. GPT was wrong at first, but got me partly there and I could massage the question and Google and get closer to the right answer. Without it, I wouldn’t have been able to get any of it, especially with the macro.
Exactly, I also find that it tends to do a pretty good job pointing you in the right direction. It’s way faster than googling or going through sites like stackoverflow because the answers are contextual. You can ask about a specific thing you want to do, and and an answer that gives you a general idea of what to do. For example, I’ve found it to be great for crafting complex sql queries. I don’t really care if the answer is perfect, as long as it gives me an idea of what I need to do.
We need a comparison against an average coder. Some fucking baseline ffs.
Why would we compare it against an average coder?
ChatGPT wants to be a coding aid/reference material. A better baseline would be the top rated answer for the question on stackoverflow or whether the answer exists on the first 3 Google search results.
Or a textbook’s explanation
Cause people arent looking at chatgpt as an accurate simple code generator, rather a junior dev.
“Self driving cars will make the roads safer. They won’t be drunk or tired or make a mistake.”
Self driving cars start killing people.
“Yeah but how do they compare to the average human driver?”
Goal post moving.
The issue is the powers that be arent comparing chatgpt to a reference manual. Rather another human being.
We need a comparison against an average coder. Some fucking baseline ffs.