For the love of God, if you’re a junior programmer you’re overestimating your understanding if you keep relying on chatGPT thinking ‘of course I’ll spot the errors’. You will until you won’t and you end up dropping the company database or deleting everything in root.
All ChatGPT is doing is guessing the next word. And it’s trained on a bunch of bullshit coding blogs that litter the internet, half of which are now chatGPT written (without any validation of course).
If you can’t take 10 - 30 minutes to search for, read, and comprehend information on stack overflow or docs then programming (or problem solving) just isn’t for you. The junior end of this feel is really getting clogged with people who want to get rich quick without doing any of the legwork behind learning how to be good at this job, and ChatGPT is really exarcebating the problem.
If you can’t take 10 - 30 minutes to search for, read, and comprehend information on stack overflow or docs
A lot of the time this is just looking for syntax though; you know what you want to do, and it’s simple, but it is gated behind busywork. This is to me the most useful part about ChatGPT, it knows all the syntax and will write it out for you and answer clarifying questions so you can remain in a mental state of thinking about the actual problem instead of digging through piles of junk for a bit of information.
Somehow you hit an unpopular opinion landmine with the greybeard devs.
For the greybeard devs:
Try asking ChatGPT to write you some Arduino code to do a specific task. Even if you don’t know how to write code for an Arduino, ChatGPT will get you 95% of the way there with the proper libraries and syntax.
No way in hell I’m digging through forums and code repos for hours to blink an led and send out a notification through a web hook when a sensor gets triggered if AI can do it for me in 30 seconds.
AI obviously can’t do everything for you if you’ve never coded anything before, but it can do a damn good job of translating your knowledge of one programming language into every other programming language available.
It’s great for jumping into something you’re very unfamiliar with. Unfortunately, if you often find yourself very unfamiliar with day to day tasks, you’re probably incompetent. (Or maybe a butterfly who gets paid to learn new things every day.)
Did ChatGPT just hallucinate it? Does it exist but it isn’t used like ChatGPT says? Does it exist but it doesn’t do what ChatGPT thinks it does?
I use ChatGPT sometimes to help out with stuff at home (I’ve tried it for work stuff but the stuff I work on is niche enough that it purely hallucinates), and I’ve ended up running in circles for hours because the answer I got ended up in this uncanny valley: Correct enough that it isn’t immediately obviously wrong, but incorrect enough that it won’t work, it can’t work, and you’re going to really have to put a lot of work in to figure that out.
The more you grow in experience the more you’re going to realize that syntax and organization is the majority of programming work.
When you first start out, it feels like the hardest part is figuring out how to get from a to b on a conceptual level. Eventually that will become far easier.
You break the big problem down into discrete steps, then figure out the besy way to do each step. It takes little skill to say “the computer just needs to do this”. The trick is knowing how to speak to the computer in a way that can make sense to the computer, to you, and to the others who will eventually have to work with your code.
You’re doing the equivalent of a painter saying “I’ve done the hard part of envisioning it in my head! I’m just going to pay some guy on fiver to move the brush for me”
This is difficult to put into words, as it’s also not about memorization of every language specific syntax pattern. But there’s a difference between looking up documentation or at previous code for syntax, and trying to have chatGPT turn your psuedocode notes into working code.
I’m a pretty senior dev and have chat gpt open for quick searches. It’s great for helping me figure out what to Google in the cases where I can’t think of the name of a pattern or type I’m looking for. It also helps quite a bit with learning about obscure functions and keywords in SQL that I can do more research on
Hell, I use Copilot daily. Its auto complete is top-tier
Copilot is good for tedious stuff like writing enums. But otherwise I more often than not need to only accept tne suggested line or particular words, since in multiline snippets it can do stupid things, like exiting outside of main() or skipping error checks.
I’ve been programming for decades, I’m not actually a beginner. A mistake I made early on was thinking that everything I learn will be worth the time to learn it, and will always increase my overall skill level. But (particularly as relates to syntax) it’s not and it doesn’t; something I only use once or rarely, something that isn’t closely connected with the rest of what I often do, I’ll just forget it after a while. I greatly prefer being broadly capable of making things happen to having a finely honed specialization, so I run into that sort of thing a lot, there is an ocean of information out there and many very different things a programmer can be doing.
I think it is an important and valuable lesson to know when to get over yourself and take shortcuts. There are situations where you absolutely should never do that, but they are rare. There are many situations where not taking shortcuts is a huge mistake and will result in piles of abandoned code and not finishing what you set out to do. AI is an incredibly powerful source of shortcuts.
You’re doing the equivalent of a painter saying “I’ve done the hard part of envisioning it in my head! I’m just going to pay some guy on fiver to move the brush for me”
More like you’ve coded the functionality for a webapp, have a visual mockup, and pay some guy on fiver to write the CSS for you, because doing it yourself is an inefficient use of your time and you don’t specialize in CSS.
As for the issue of a new programmer ending up with problems because they rely too much on AI and somehow fail to learn how to model the structure of programs in their head, that’s probably real, but I can’t imagine how that will go because all I had to go on when I was learning was google and IRC and it’s totally different. Hope it works out for them.
TBF that’s how many master artists worked in the past. The big art producers had one master painter guiding a bunch of apprentices who did the actual legwork.
That the exact opposite problem for most people though. Syntax is hard at first because you are unfamiliar and gets more natural to you overtime. Algorithms are easier to think about conceptually as a person with no programming experience as they are not programming specific. If you are an experienced developer struggling over syntax yet breezing through difficult data structure and algorithm problems (Eg. Thinking about the most efficient way to upload constantly updating vertex data to the gpu) you are definitely the anomaly.
There’s a 5 hour interview with John Carmack on YouTube where he talks about transitioning from really caring deeply about algorithms and the like to deeply caring about how to make a sustainable and maintainable codebase you can have an entire team work on.
Often, a solution that is completely correct if all you’re doing is solving that problem is completely incorrect in the greater context of the codebase you’re working within, like if you wanted to add a dog to the Mona Lisa, you can’t just draw a detailed line art dog or a cartoon dog and expect it to work – you’d need to find someone who can paint a dog similar to the art style of the piece and properly get it to mesh with the painting.
Never ask ChatGPT to write code that you plan to actually use, and never take it as a source of truth. I use it to put me on a possible right path when I’m totally lost and lack the vocabulary to accurately describe what I need. Sometimes I’ll ask it for an example of how sometimes works so that I can learn it myself. It’s an incredibly useful tool, but you’re out of your damn mind if you’re just regularly copying code it spits out. You need to error check everything it does, and if you don’t know the syntax well enough to write it yourself, how the hell do you plan to reliably error check it?
You absolutely can ask it for code you plan to use as long as you treat chatgpt like a beginner dev. Give it a small, very simple, self contained task and test it thoroughly.
Also, you can write unit tests while being quite unfamiliar with the syntax. For example, you could write a unit test for a function which utilizes a switch statement, without using a switch statement to test it. There’s a whole sect of “test driven development” where this kind of development would probably work pretty well.
I’ll agree that if you can’t test a piece of code, you have no business writing in the language in a professional capacity.
I write a lot of bash and I still have to check syntax every day, but the answer to that is not chatGPT but a proper linter like shell check that you can trust because it’s based on a rigid set of rules, not the black box of a LLM.
I can understand the syntax justification for obscure languages that don’t have a well written linter, but if anything that gives me less confidence about CHATGPT because it’s training material for an obscure language is likely smaller.
Less checking syntax and more like “what was this function name again ?”
“Which library has that ?”
“Do I need to instance this or is it static ?”
All of theses can be answered by documentation, but who want to read the docs when you can ask chatgpt. (Copilot is better in my experience btw)
ChatGPT cannot explain, because it doesn’t understand. It will simply string together a likely sequence of characters. I’ve tried to use it multiple times for programming tasks and found each time that it doesn’t save much time, compared to an IDE. ChatGPT regularly makes up methods or entire libraries. I do like it for creating longer texts that I then manually polish, but any LLM is awful for factual information.
ChatGPT regularly makes up methods or entire libraries
I think that when it is doing that, it is normally a sign that what you are asking for does not exist and you are on the wrong track.
ChatGPT cannot explain, because it doesn’t understand
I often get good explanations that seem to reflect understanding, which often would be difficult to look up otherwise. For example when I asked about the code generated, {myVariable} , and how it could be a valid function parameter in javascript, it responded that it is the equivalent of {"myVariable":myVariable}, and “When using object literal property value shorthand, if you’re setting a property value to a variable of the same name, you can simply use the variable name.”
If ChatGPT gives you correct information you’re either lucky or just didn’t realize it was making shit up. That’s a simple fact. LLMs absolutely have their uses, but facts ain’t one of them.
you can remain in a mental state of thinking about the actual problem
more like you’ll end up wasting a significant amount of time debugging not only the problem, but also chatGPT, trying to correct the bullshit it spews out, often ignoring parts of your prompt
It might be wrong even if you provide extensive context to make it more accurate in it’s heuristics. And providing extensive context is pretty time consuming at times.
I mean it might be good at helping you when you’re stuck, but sometimes it misses simple issues such as typos and for one issue resolved, it might introduce another if you’re not careful.
You are saying that as if it’s a small feat. Accurately guessing the next word requires understanding of what the words and sentences mean in a specific context.
Don’t get me wrong, it’s incredible. But it’s still a variation of the Chinese room experiment, it’s not a real intelligence, but really good at pretending to be one. I might trust it more if there were variants based on strictly controlled datasets.
I have read more than is probably healthy about the Chinese room and variants since it was first published. I’ve gone back and forth on several ideas:
There is no understanding
The person in the room doesn’t understand, but the system does
We are all just Chinese rooms without knowing it (where either of the first 2 points might apply)
Since the advent of ChatGPT, or, more properly, my awareness of it, the confusion has only increased. My current thinking, which is by no means robust, is that humans may be little more than “meatGPT” systems. Admittedly, that is probably a cynical reaction to my sense that a lot of people seem to be running on automatic a lot of the time combined with an awareness that nearly everything new is built on top of or a variation on what came before.
I don’t use ChatGPT for anything (yet) for the same reasons I don’t depend too heavily on advice from others:
I suspect that most people know a whole lot less than they think they do
I very likely know little enough myself
I definitely don’t know enough to reliably distinguish between someone truly knowledgeable and a bullshitter.
I’ve not yet seen anything to suggest that ChatGPT is reliably any better than a bullshitter. Which is not nothing, I guess, but is at least a little dangerous.
So theoretically could you program an AI using strictly verified programming textbooks/research etc, is it currently possible to make an AI that would do far better at programming? I love the concepts around AI but I know fuckall about ML and the actual intricacies of it. So sorry if it’s a dumb question.
Yeah, this is the approach people are trying to take more now, the problem is generally amount of that data needed and verifying it’s high quality in the first place, but these systems are positive feedback loops both in training and in use. If you train on higher quality code, it will write higher quality code, but be less able to handle edge cases or potentially complete code in a salient way that wasn’t at the same quality bar or style as the training code.
On the use side, if you provide higher quality code as input when prompting, it is more likely to predict higher quality code because it’s continuing what was written. Using standard approaches, documenting, just generally following good practice with code before sending it to the LLM will majorly improve results.
The Chinese room thought experiment doesn’t prove anything and probably confuses the discussion more than it clarifies.
In order for the Chinese room to convince an outside observer of knowing Chinese like a person the room as a whole basically needs to be sentient and understand Chinese. The person in the room doesn’t need to understand Chinese. “The room” understands Chinese.
The confounding part is the book, pen and paper. It suggests that the room is “dumb”. But to behave like a person the person-not-knowing-Chinese plus book and paper needs to be able to memorize and reason about very complex concepts. You can do that with pen and paper and not-understanding-Chinese person it just takes an awful amount of time and complex set of continuously changing rules in said book.
For the love of God, if you’re a junior programmer you’re overestimating your understanding if you keep relying on chatGPT thinking ‘of course I’ll spot the errors’. You will until you won’t and you end up dropping the company database or deleting everything in root.
All ChatGPT is doing is guessing the next word. And it’s trained on a bunch of bullshit coding blogs that litter the internet, half of which are now chatGPT written (without any validation of course).
If you can’t take 10 - 30 minutes to search for, read, and comprehend information on stack overflow or docs then programming (or problem solving) just isn’t for you. The junior end of this feel is really getting clogged with people who want to get rich quick without doing any of the legwork behind learning how to be good at this job, and ChatGPT is really exarcebating the problem.
A lot of the time this is just looking for syntax though; you know what you want to do, and it’s simple, but it is gated behind busywork. This is to me the most useful part about ChatGPT, it knows all the syntax and will write it out for you and answer clarifying questions so you can remain in a mental state of thinking about the actual problem instead of digging through piles of junk for a bit of information.
Somehow you hit an unpopular opinion landmine with the greybeard devs.
For the greybeard devs: Try asking ChatGPT to write you some Arduino code to do a specific task. Even if you don’t know how to write code for an Arduino, ChatGPT will get you 95% of the way there with the proper libraries and syntax.
No way in hell I’m digging through forums and code repos for hours to blink an led and send out a notification through a web hook when a sensor gets triggered if AI can do it for me in 30 seconds. AI obviously can’t do everything for you if you’ve never coded anything before, but it can do a damn good job of translating your knowledge of one programming language into every other programming language available.
It’s great for jumping into something you’re very unfamiliar with. Unfortunately, if you often find yourself very unfamiliar with day to day tasks, you’re probably incompetent. (Or maybe a butterfly who gets paid to learn new things every day.)
Getting paid to learn new things everyday at work is a dream, I don’t see the issues
The issue is it’s a dream.
BIIIIG problem: The last 5%.
Did ChatGPT just hallucinate it? Does it exist but it isn’t used like ChatGPT says? Does it exist but it doesn’t do what ChatGPT thinks it does?
I use ChatGPT sometimes to help out with stuff at home (I’ve tried it for work stuff but the stuff I work on is niche enough that it purely hallucinates), and I’ve ended up running in circles for hours because the answer I got ended up in this uncanny valley: Correct enough that it isn’t immediately obviously wrong, but incorrect enough that it won’t work, it can’t work, and you’re going to really have to put a lot of work in to figure that out.
Just a few days ago I read an article on the newest features of Kotlin 1.9. Zero of it was true.
Internet is littered with stuff like this.
If model is correct, you are correct. If model is not correct, you are working on false assumptions.
No difference there, either way your information may be wrong or misleading. Running code and seeing what it does is the solution.
The more you grow in experience the more you’re going to realize that syntax and organization is the majority of programming work.
When you first start out, it feels like the hardest part is figuring out how to get from a to b on a conceptual level. Eventually that will become far easier.
You break the big problem down into discrete steps, then figure out the besy way to do each step. It takes little skill to say “the computer just needs to do this”. The trick is knowing how to speak to the computer in a way that can make sense to the computer, to you, and to the others who will eventually have to work with your code.
You’re doing the equivalent of a painter saying “I’ve done the hard part of envisioning it in my head! I’m just going to pay some guy on fiver to move the brush for me”
This is difficult to put into words, as it’s also not about memorization of every language specific syntax pattern. But there’s a difference between looking up documentation or at previous code for syntax, and trying to have chatGPT turn your psuedocode notes into working code.
organization, absolutely - but syntax? c’mon…
I’m a pretty senior dev and have chat gpt open for quick searches. It’s great for helping me figure out what to Google in the cases where I can’t think of the name of a pattern or type I’m looking for. It also helps quite a bit with learning about obscure functions and keywords in SQL that I can do more research on
Hell, I use Copilot daily. Its auto complete is top-tier
Copilot is good for tedious stuff like writing enums. But otherwise I more often than not need to only accept tne suggested line or particular words, since in multiline snippets it can do stupid things, like exiting outside of
main()or skipping error checks.I’ve been programming for decades, I’m not actually a beginner. A mistake I made early on was thinking that everything I learn will be worth the time to learn it, and will always increase my overall skill level. But (particularly as relates to syntax) it’s not and it doesn’t; something I only use once or rarely, something that isn’t closely connected with the rest of what I often do, I’ll just forget it after a while. I greatly prefer being broadly capable of making things happen to having a finely honed specialization, so I run into that sort of thing a lot, there is an ocean of information out there and many very different things a programmer can be doing.
I think it is an important and valuable lesson to know when to get over yourself and take shortcuts. There are situations where you absolutely should never do that, but they are rare. There are many situations where not taking shortcuts is a huge mistake and will result in piles of abandoned code and not finishing what you set out to do. AI is an incredibly powerful source of shortcuts.
More like you’ve coded the functionality for a webapp, have a visual mockup, and pay some guy on fiver to write the CSS for you, because doing it yourself is an inefficient use of your time and you don’t specialize in CSS.
As for the issue of a new programmer ending up with problems because they rely too much on AI and somehow fail to learn how to model the structure of programs in their head, that’s probably real, but I can’t imagine how that will go because all I had to go on when I was learning was google and IRC and it’s totally different. Hope it works out for them.
TBF that’s how many master artists worked in the past. The big art producers had one master painter guiding a bunch of apprentices who did the actual legwork.
And senior devs guide junior devs in the same way. The point is the masters already did their time in the trenches. That how they became masters.
That the exact opposite problem for most people though. Syntax is hard at first because you are unfamiliar and gets more natural to you overtime. Algorithms are easier to think about conceptually as a person with no programming experience as they are not programming specific. If you are an experienced developer struggling over syntax yet breezing through difficult data structure and algorithm problems (Eg. Thinking about the most efficient way to upload constantly updating vertex data to the gpu) you are definitely the anomaly.
There’s a 5 hour interview with John Carmack on YouTube where he talks about transitioning from really caring deeply about algorithms and the like to deeply caring about how to make a sustainable and maintainable codebase you can have an entire team work on.
Often, a solution that is completely correct if all you’re doing is solving that problem is completely incorrect in the greater context of the codebase you’re working within, like if you wanted to add a dog to the Mona Lisa, you can’t just draw a detailed line art dog or a cartoon dog and expect it to work – you’d need to find someone who can paint a dog similar to the art style of the piece and properly get it to mesh with the painting.
Never ask ChatGPT to write code that you plan to actually use, and never take it as a source of truth. I use it to put me on a possible right path when I’m totally lost and lack the vocabulary to accurately describe what I need. Sometimes I’ll ask it for an example of how sometimes works so that I can learn it myself. It’s an incredibly useful tool, but you’re out of your damn mind if you’re just regularly copying code it spits out. You need to error check everything it does, and if you don’t know the syntax well enough to write it yourself, how the hell do you plan to reliably error check it?
You absolutely can ask it for code you plan to use as long as you treat chatgpt like a beginner dev. Give it a small, very simple, self contained task and test it thoroughly.
Also, you can write unit tests while being quite unfamiliar with the syntax. For example, you could write a unit test for a function which utilizes a switch statement, without using a switch statement to test it. There’s a whole sect of “test driven development” where this kind of development would probably work pretty well.
I’ll agree that if you can’t test a piece of code, you have no business writing in the language in a professional capacity.
I write a lot of bash and I still have to check syntax every day, but the answer to that is not chatGPT but a proper linter like shell check that you can trust because it’s based on a rigid set of rules, not the black box of a LLM.
I can understand the syntax justification for obscure languages that don’t have a well written linter, but if anything that gives me less confidence about CHATGPT because it’s training material for an obscure language is likely smaller.
Less checking syntax and more like “what was this function name again ?”
“Which library has that ?”
“Do I need to instance this or is it static ?”
All of theses can be answered by documentation, but who want to read the docs when you can ask chatgpt. (Copilot is better in my experience btw)
ChatGPT cannot explain, because it doesn’t understand. It will simply string together a likely sequence of characters. I’ve tried to use it multiple times for programming tasks and found each time that it doesn’t save much time, compared to an IDE. ChatGPT regularly makes up methods or entire libraries. I do like it for creating longer texts that I then manually polish, but any LLM is awful for factual information.
I think that when it is doing that, it is normally a sign that what you are asking for does not exist and you are on the wrong track.
I often get good explanations that seem to reflect understanding, which often would be difficult to look up otherwise. For example when I asked about the code generated,
{myVariable}, and how it could be a valid function parameter in javascript, it responded that it is the equivalent of{"myVariable":myVariable}, and “When using object literal property value shorthand, if you’re setting a property value to a variable of the same name, you can simply use the variable name.”If ChatGPT gives you correct information you’re either lucky or just didn’t realize it was making shit up. That’s a simple fact. LLMs absolutely have their uses, but facts ain’t one of them.
more like you’ll end up wasting a significant amount of time debugging not only the problem, but also chatGPT, trying to correct the bullshit it spews out, often ignoring parts of your prompt
That hasn’t been my experience. How are you trying to use it?
It might be wrong even if you provide extensive context to make it more accurate in it’s heuristics. And providing extensive context is pretty time consuming at times.
I think it would help me organize my thoughts to write that all out anyway even without a LLM.
I mean it might be good at helping you when you’re stuck, but sometimes it misses simple issues such as typos and for one issue resolved, it might introduce another if you’re not careful.
You are saying that as if it’s a small feat. Accurately guessing the next word requires understanding of what the words and sentences mean in a specific context.
Don’t get me wrong, it’s incredible. But it’s still a variation of the Chinese room experiment, it’s not a real intelligence, but really good at pretending to be one. I might trust it more if there were variants based on strictly controlled datasets.
I have read more than is probably healthy about the Chinese room and variants since it was first published. I’ve gone back and forth on several ideas:
Since the advent of ChatGPT, or, more properly, my awareness of it, the confusion has only increased. My current thinking, which is by no means robust, is that humans may be little more than “meatGPT” systems. Admittedly, that is probably a cynical reaction to my sense that a lot of people seem to be running on automatic a lot of the time combined with an awareness that nearly everything new is built on top of or a variation on what came before.
I don’t use ChatGPT for anything (yet) for the same reasons I don’t depend too heavily on advice from others:
I’ve not yet seen anything to suggest that ChatGPT is reliably any better than a bullshitter. Which is not nothing, I guess, but is at least a little dangerous.
what often puts me of that people almost never fakt check me when i tell them something wich also tells me they wouldn’t do the same with chatgpt
So theoretically could you program an AI using strictly verified programming textbooks/research etc, is it currently possible to make an AI that would do far better at programming? I love the concepts around AI but I know fuckall about ML and the actual intricacies of it. So sorry if it’s a dumb question.
Yeah, this is the approach people are trying to take more now, the problem is generally amount of that data needed and verifying it’s high quality in the first place, but these systems are positive feedback loops both in training and in use. If you train on higher quality code, it will write higher quality code, but be less able to handle edge cases or potentially complete code in a salient way that wasn’t at the same quality bar or style as the training code.
On the use side, if you provide higher quality code as input when prompting, it is more likely to predict higher quality code because it’s continuing what was written. Using standard approaches, documenting, just generally following good practice with code before sending it to the LLM will majorly improve results.
Interesting, that makes sense. Thank you for such a thoughtful response.
The Chinese room thought experiment doesn’t prove anything and probably confuses the discussion more than it clarifies.
In order for the Chinese room to convince an outside observer of knowing Chinese like a person the room as a whole basically needs to be sentient and understand Chinese. The person in the room doesn’t need to understand Chinese. “The room” understands Chinese.
The confounding part is the book, pen and paper. It suggests that the room is “dumb”. But to behave like a person the person-not-knowing-Chinese plus book and paper needs to be able to memorize and reason about very complex concepts. You can do that with pen and paper and not-understanding-Chinese person it just takes an awful amount of time and complex set of continuously changing rules in said book.
Edit:
Yup. Accurately guessing the next thought (or action) is all brains need to do so I don’t see what the alleged “magic” is supposed to solve.
deleted by creator
It’s okay old man. There is a middle there where folks understand the task but aren’t familiar with the implementation.