The counter isn't that some hypothetical AI will come along and solve three problems with the current generation. That argument, and the response, is tired as dirt. No, the counter here is to point out that the prototype is fine for small values of production. I wouldn't take something Ai generated and serve it on the web, and have the world try and hack the site, but some small app that isn't exposed to the world? We can throw around claims of security and scalability, but Amy's (local) personal cookbook app doesn't care about either of those.
"Amy's (local) personal cookbook app" sounds great in theory, and that people will do this for the novelty, but my point is that this will not be a long-term trend. Largely because people will want more features in their apps beyond the prototype, they will find it hard and decide they'd rather be doing the cooking than cajoling, and go back to using the same app their peers do so they can be social and share, or even just send the recipe from their phone to the tablet on the kitchen counter
The problem is git's index let's you write a bunch of unconnected code, then commit it separately. To different branches, even! This works great for stacking diffs but is terribly confusing if you don't know what you're doing.
Who do you call? The Internet police? Anna's Archive is hosted in countries that don't give a shit about US copyright laws. Pirate bay is back up, and they've been at it for decades.
Sure, ultimately technical/knowledgeable people will be able to get around it. But preventing normies from accessing Anna's Archive is what they care about, because most people are normies.
preventing normies from accessing Anna's Archive is what they care about
Seriously?
Anna's Archive hosts ebooks and scholarly journal articles.
Not the kind of stuff your average Instagram Influencer (TM) is into.
I'd be utterly shocked if more than 1% of the population had ever used Anna's Archive. This isn't like Hollywood movie torrenting sites or IPTV sports streaming piracy. It's a long way from mainstream.
In this scenario your VPNs would still need to find a ISP that would let them route packets out of that country. This means that instead of a legit VPN company you have to deal with cyber criminals to get such a VPN.
I keep saying the real advancement by LLMs isn't for professional programmers, but for every job that is programming adjacent. Every biologist writing code to do analysis. Every test engineer interfacing with test results and graphing results. (eg all the instruments from cold weather testing) Anyone that's figured out you can glue Jira to a local LLM and then have voice command Jira. Etc.
I once wrote software that had to manage the traffic coming into a major shipping terminal- OCR, gate arms, signage, cameras for inspecting chassis and containers, SIP audio comms, RFID readers, all of which needed to be reasoned about in a state machine, none of which were reliable. It required a lot of on the ground testing and observation and tweaking along with human interventions when things went wrong. I’d guess LLMs would have been good at subsets of that project, but the entire thing would still require a team of humans to build again today.
I've had good luck when giving the AI its own feedback loop. On software projects, it's letting the AI take screenshots and read log files, so it can iterate on errors without human input. On hardware projects, it's a combination of solenoids, relays, a pi and pizerow, and a webcam. I'm not claiming that an AI could do the above mentioned project, just that (some) hardware projects can also get humans out of the loop.
Sir your experience is unique and thanks for answering this.
That being said, someone took the idea of you saying LLM's might be good at subsets of projects to consider we should use LLMs for that subset as well
But I digress because (I provided more in depth reasoning in other comment as well) because if there is an even minute bug which might slip up past LLM and code review for subset of that and for millions of cars travelling through points, we assume that one single bug in it somewhere might increase the traffic/fatality traffic rate by 1 person per year. Firstly it shouldn't be used because of the inherent value of human life itself but even from monetary sense as well so there's really not much reason I can see in using it
That alone over a span of 10 years would cost 75 million-130Million$ (the value of life in US for a normal perosn ranges from 7.5 million - 13 million$)
Sir I just feel like if the point of LLM is to have less humans or less giving them income, this feels so short sighted because I (if I were the state and I think everyone will agree after the cost analysis) would much rather pay a few hundred thousand dollars to even a few million$ right now to save 75-130 Million$ (on the smallest scale mind you, it can get exponentially more expensive)
I am not exactly sure how we can detect the rate of deaths due to LLM use itself (the 1 number) but I took the most conservative number.
And that is also the fact that we won't know if LLM's might save a life but I am 99.9% sure that might not be the case and once again it wouldn't be verifiable itself so we are shooting things in the dark
And we can have a much more sensitive job with better context (you know what you are working at and you know how valuable it is/can save lives and everything) whereas no amount of words can convey that danger to LLM's
To put it simply, the LLM might not know the difference between this life or death situation machine's code at times or a sloppy website created by it.
I just don't think its worth it especially in this context at all even a single % of LLM code might not be worth it here.
I had friend who was in crisis while the rest of us were asleep. Talking with ChatGPT kept her alive. So we know the number is at least one. If you go to the Dr ChatGPT thread, you'll find multiple reports of people who figured out debilitating medical conditions via ChatGPT in conjunction with a licensed human doctor, so we can be sure the numbers greater than zero. It doesn't make headlines the same way Adam's suicide does, and not just because OpenAI can't be the ones to say it.
Don’t you understand? That’s why all these AI companies are praying for humanoid robots to /just work/ - so we can replace humans mentally and physically ASAP!
I'm sure those will help. But that doesn't solve the problem the parent stated. Those robots can't solve those real world problems until they can reason, till they can hypothesize, till they can experiment, till they can abstract all on their own. The problem is you can't replace the humans (unilaterally) until you can create AGI. But that has problem of its own, as you now have to contend with previously creating a slave class of artificial life forms.
No worries - you’ve added useful context for those who may be misguided by these greedy corporations looking to replace us all. Maybe it helps them reconsider their point of view!
But you admit that fewer humans would be needed as “LLMs would have been good at subsets of that project”, so some impact already and these AI tools only get better.
If that is the only thing that you took out of that conversation, then I don't really believe that that job might've been suitable for you in the first place.
Now I don't know which language they used for the project (could be python or could be C/C++ or could be rust) but its like "python would have been good at subsets of that project", so some impact already and these python tools only get better
Did python remove the jobs? No. Each project has their own use case and in some LLM's might be useful, in others not.
In their project, LLM's might be useful for some parts but their majority of the work was doing completely new things with a human in feedback.
You are also forgetting trust factor, yes lets have your traffic lights system be written by a LLM, surely. Oops, the traffic lights glitched and all waymos (another AI) went beserk and oops accidents/crash happened which might cost millions.
Personally I wouldn't trust even a subset of LLM code and much rather have my country/state/city to pay to real developers that can be accountable & good quality control checks for such critical points to the point that no LLM in this context should be a must
For context, if LLM use can even impact 1 life every year. The value of 1 person is 7.5-13 million$
Over a period of 10 years in this really really small glitch of LLM, you end up in 10 years losing 75 million$
Yup go ahead save a few thousand dollars right now by not paying people enough in the first case to use LLM to then lose 75 million $ (on the least case scenario)
I doubt you have a clue regarding my suitability for any project, so I’ll ignore the passive l-aggressive ad hominem.
Anyway, it seems you are walking back your statement regarding LLM being useful for parts of your project, or ignoring the impact on personnel count. Not sure what you were trying to say then.
I went back because of course I could've just pointed out one picture but still wanted to give the whole picture.
my conclusion is rather the fact that this is a very high stakes project (both emotionally and mentally and economically) and AI are still black boxes with chances of being much more error prone (atleast in this context) and chances of it missing something to cause the -75 million and deaths of many is more likely and also that in such a high stakes project, LLM's shouldn't be used and having more engineers in the team might be worth it.
> I doubt you have a clue regarding my suitability for any project, so I’ll ignore the passive l-aggressive ad hominem.
Aside from the snark presented at me. I agree. And this is why you don't see me in a project regarding such high stakes project and neither should you see an LLM at any costs in this context. These should be reserved to the caliber of people who have both experience in the industry and are made of flesh.
Human beings are basically black boxes as far as the human brain is concerned. We don't blindly trust the code coming out of those black boxes, it seems illogical to do the same for LLMs.
Is your kitchen contractor an unthinking robot with no opinions or thoughts of their own that has never used a kitchen? Obviously if you want a specific cabinet to go in a specific place in the room, you're going to have to give the kitchen contractor specifics. But assuming your kitchen contractor isn't an utter moron, they can come up with something reasonable if they know it's supposed to be the kitchen. A sink, a stove, dishwasher, refrigerator. Plumbing and power for the above. Countertops, drawers, cabinets. If you're a control freak (which is your perogative, it's your kitchen after all), that's not going to work for you. Same too for generated code. If you absolutely must touch every line of code, code generation isn't going to suit you. If you just want a login screen with parameters you define, there are so many login pages the AI can crib from that nondeterminism isn't even a problem.
At least in case of the kitchen contractor, you can trust all the electrical equipment, plumbing etc. is going to be connected in such a way that disasters won't happen. And if it is not, at least you can sue the contractor.
The problem with LLMs is that it is not only the "irrelevant details" that are hallucinated. It is also "very relevant details" which either make the whole system inconsistent or full of security vulnerabilities.
The login page example was actually perfect for illustrating this. Meshing polygons? Centering a div? Go ahead and turn the LLM loose. If you miss any bugs you can just fix them when they get reported.
But if it's security critical? You'd better be touching every single line of code and you'd better fully understand what each one does, what could go wrong in the wild, how the approach taken compares to best practices, and how an attacker might go about trying to exploit what you've authored. Anything less is negligence on your part.
You kitchen contractor will never cook in your kitchen. If you leave the decisions to them, you'll get something that's quick and easy to build, but it for sure won't have all the details that make a great kitchen. It will be average.
Which seems like an apt analogy for software. I see people all the time who build systems and they don't care about the details. The results are always mediocre.
I think this is a major point people do not mention enough during these debates on "AI vs Developers": The business/stakeholder side is completely fine with average and mediocre solutions as long as those solutions are delivered quickly and priced competitively. They will gladly use a vibecoded solution if the solution kinda sorta mostly works. They don't care about security, performance or completeness... such things are to be handled when/if they reach the user/customer in significant numbers. So while we (the devs) are thinking back to all the instances we used gpt/grok/claude/.. and not seeing how the business could possibly arrive to our solutions just with AI and wihout us in the loop... the business doesn't know any of the details nor does it care. When it comes to anything IT related, your typical business doesn't know what it doesn't know, which makes it easy to fire employees/contractors for redundancy first (because we have AI now) and ask questions later (uhh... because we have AI now).
That still requires you to evaluate all the details in order to figure out which you care about. And if you haven't built a kitchen before you, won't know what the details even are ahead of time. Which means you need to be involved in the process, constantly evaluating whether what is currently happening and if you need to care about it.
Maybe they have a kitchen without dishwasher. So unless asked they won't include one. Or even make it possible to include one. Seems like a real possibility. Maybe eventually after building many kitchens they learn they should ask about that one.
Are you on a desert island with no access to the Internet? If you don't know docker, what's faster? Reading all of the documentation first and then figuring out the difference between, say, run and exec, or just copy and pasting a command from a tutorial until it sinks in and you gain a better understanding? This is the AI information age. If docker has eaten your hard drive, and again, you don't know docker, is it easier to have ChatGPT tell you, or muddle around with ps, rm, images, rmi and all of the various options.
If you have a command with a bunch of flags, static documentation like man pages are just such a poor interface compared to eg explainshell.com. This opinion obviously gets me thrown out of the Unix grey beards club, but I don't have a beard and it's not grey.
How do you know which command to copy and paste? Unless you're suggesting to just try them randomly until you get one that seems to do what you want.
There are plenty of commands where the documentation is nearly impenetrable (e.g. ffmpeg, or if it exists at all), but I think GP's point was that for docker it's fairly simple.
IMO except for the concrete examples for docker run/exec, this website looks more or less exactly like the CLI help output for docker.
What a terrible question. Why do you think speed is a good metric? Why is it better to copy-paste in 2 seconds than to read the manual for 20 minutes and learn the basics? What would have happened?
Quality QA folk are able to reason and develop an understanding of a system without ever seeing a line of code. As long as we're discussing fundamentals, being able to develop such an understanding will be a skill to develop that will pay off returns even after AI comes and goes. Even when given the code, rushing to throw print everywhere, or rushing to throw it at debugger both come behind someone that understands the system and is able to observe the bug, then sit and reason about it, and then in some cases, just fix the bug. I've worked with a couple of programmers that good, it's awesome to experience.
Point is though, you don't need to see the code to debug it, so the fact that the code was generated should not be the thing holding you back.
---
When given only three words, is the rewrite any good? When given to a human intern, would it be any good? Instead "refactor the foo that's repeated n bar, baz and qux into one reusable shared class/component/the appropriate thing for the given language" is good actual software engineering that a human would do, or ask an intern to do.
very cool. I may have to look a bit closer at the pipeline I used to create the art at ssh funky.nondeterministic.computer. The graphics could always be improved, however I will note that it needs color for best effect.
More to the point though, you should be using Agents in Claude Code to limit context pollution. Agents run with their own context, and then only return salient details. Eg, I have an Agent to run "make" and return the return status and just the first error message if there is one. This means the hundreds/thousands of lines of compilation don't pollute the main Claude Code context, letting me get more builds in before I run out of context there.
reply