Lots of respected limits and lines on government power are just being casually broken, so I don't think you're wrong. Whatever's going to happen next it probably won't have the stability of the past.
I don't see how asking AI to write some description of why it wrote this or that code would actually result in an explanation of why it wrote that code? It's not like it's thinking about it in that way, it's just generating both things. I guess they'd be in the same context so it might be somewhat correct.
If you ask it to document why it did something, then when it goes back later to update the code it has the why in its context. Otherwise, the AI just sees some code later and has no idea why it was written or what it does without reverse engineering it at the moment.
I'm not sure you understood the GP comment. LLMs don't know and can't tell you why they write certain things. You can't fix that by editing your prompt so it writes it on a comment instead of telling you. It will not put the "why" in the comment, and therefore the "why" won't be in the future LLM's context, because there is no way to make it output the "why".
It can output something that looks like the "why" and that's probably good enough in a large percentage of cases.
LLMs know why they are writing things in the moment, and they can justify decisions. Asking it to write those things down when it writes code works, or even asking them to design the code first and then generate/update code from the design also works. But yes, if things aren’t written down, “the LLM don’t know and can’t tell.” Don’t do that.
I'm going to second seanmcdirmid here, a quick trick is to have Claude write a "remaining.md" if you know you have to do something that will end the session.
Example from this morning, I have to recreate the EFI disk of one of my dev vm's, it means killing the session and rebooting the vm. I had Claude write itself a remaining.md to complement the overall build_guide.vm I'm using so I can pick up where I left off. It's surprisingly effective.
No, humans probably have tens of millions of token in memory of memory per PR. It includes not only what's in the code, but what all they searched, what all they tested and in which way, which order they worked on, the edge cases they faced etc. Claude just can't document all these, else it will run out of its working context pretty soon.
Ya, LLMs are not human level, they have smaller focus windows, but you can "remember" things with documentation, just like humans usually resort to when you realize that their tens of millions of token in memory per PR isn't reliable either.
The nice thing about LLMs, however, is that they don't grumble about writing extra documentation and tests like humans do. You just tell them to write lots of docs and they do it, they don't just do the fun coding part. I can empathize why human programmers feel threatened.
> It can output something that looks like the "why"
This feels like a distinction without difference. This is an extension of the common refrain that LLMs cannot “think”.
Rather than get overly philosophical, I would ask what the difference is in practical terms. If an LLM can write out a “why” and it is sufficient explanation for a human or a future LLM, how is that not a “why“?
If you're planning on throwing the code away, fine, but if you're not, eventually you're going to have to revisit it.
Say I'm chasing down some critical bug or a security issue. I run into something that looks overly complicated or unnecessary. Is it something a human did for a reason or did the LLM just randomly plop something in there?
I don't want a made up plausible answer, I need to know if this was a deliberate choice, forex "this is to work around an bug in XY library" or "this is here to guard against [security issue]" or if it's there because some dude on Stackoverflow wrote sample code in 2008.
If your concern is philosophical, and you are defining LLMs as not having a “why”, then of course they cannot write down “why” because it doesn’t exist. This is the philosophical discussion I am trying to avoid because I don’t think it’s fruitful.
If your concern is practical and you are worried that the “why” an LLM might produce is arbitrary, then my experience so far says this isn’t a problem. What I’m seeing LLMs record in commit messages and summaries of work is very much the concrete reasons they did things. I’ve yet to see a “why” that seemed like nonsense or arbitrary.
If you have engineers checking in overly complex blobs of code with no “why”, that’s a problem whether they use AI or not. AI tools do not replace engineers and I would not with in any code base where engineers were checking in vibe coded features without understanding them and vetting the results properly.
I don't care what text the LLM generates. If you wanna read robotext, knock yourself out. It's useless for what I'm talking about, which is "something is broken and I'm trying to figure out what"
In that context, I'm trying to do two things:
1. Fix the problem
2. Don't break anything else
If there's something weird in the code, I need to know if it's necessary. "Will I break something I don't know about if I change this" is something I can ask a person. Or a whole chain of people if I need to.
I can't ask the LLM, because "yes $BIG_CLIENT needs that behavior for stupid reasons" is not gonna be a part of its prompt or training data, and I need that information to fix it properly and not cause any regressions.
It may sound contrived but that sort of thing happens allllll the time.
> If there's something weird in the code, I need to know if it's necessary.
What does this have to do with LLMs?
I agree this sort of thing happens all the time. Today. With code written by humans. If you’re lucky you can go ask the human author, but in my experience if they didn’t bother to comment they usually can’t remember either. And very often the author has moved on anyway.
The fix for this is to write why this weird code is necessary in a comment or at least a commit message or PR summary. This is also the fix for LLM code. In the moment, when in the context for why this weird code was needed, record it.
You also should shame any engineer who checks in code they don’t understand, regardless of whether it came from an LLM or not. That’s just poor engineering and low standards.
Yeah. I know. The point is there is no Chesterson's Fence when it comes to LLMs. I can't even start from the assumption that this code is here for a reason.
And yes, of course people should understand the code. People should do a lot of things in theory. In practice, every codebase has bits that are duct taped together with a bunch of #FIXME comments lol. You deal with what you got.
The problem is that your starting point seems to be that LLMs can check in garbage to your code base with no human oversight.
If your engineering culture is such that an engineer could prompt an LLM to produce a bunch of code that contains a bunch of weird nonsense, and they can check that weird nonsense in with no comments and no will say “what the hell are you doing?”, then the LLM is not the problem. Your engineering culture is. There is no reason anyone should be checking in some obtuse code that solves BIG_CORP_PROBLEM without a comment to that effect, regardless of whether they used AI to generate the code or not.
Are you just arguing that LLM’s should not be allowed to check in code without human oversight? Because yeah, I one hundred percent agree and I think most people in favor of AI use for coding would also agree.
Yeah, and I'm explaining that the gap between theory and practice is greater in practice than it is in theory, and why LLMs make it worse.
It's easy to just say "just make the code better", but in reality I'm dealing with something that's an amalgam of the work of several hundred people, all the way back to the founders and whatever questionable choices they made lol.
The map is the territory here. Code is the result of our business processes and decisions and history.
You're treating this as a philosophical question like a LLM can't have actual reasons because it's not conscious. That's not the problem. No, the problem is mechanical. The processing path that would be needed to output actual reasons just doesn't exist.
LLMs only have one data path and that path basically computes what a human is most likely to write next. There's no way to make them not do this. If you ask it for a cake recipe it outputs what it thinks a human would say when asked for a fake recipe. If you ask it for a reason it called for 3 eggs, it outputs what it thinks a human would say when asked why they called for 3 eggs. It doesn't go backwards to the last checkpoint and do a variational analysis to see what factors actually caused it to write down 3 eggs. It just writes down some things that sound like reasons you'd use 3 eggs.
If you want to know the actual reasons it wrote 3 eggs, you can do that, but you need to write some special research software that metaphorically sticks the AI's brain full of electrodes. You can't do it by just asking the model because the model doesn't have access to that data.
Humans do the same thing by the way. We're terrible at knowing why we do things. Researchers stuck electrodes in our brains and discovered a signal that consistently appears about half a second before we're consciously aware we want to do something!
But this is exactly why it is philosophical. We’re having a discussion about why an LLM cannot really ever explain “why”. And then we turn around and say, but actually humans have the exact same problem. So it’s not an LLM problem at all. It’s a philosophical problem about whether it’s possible to identify a real “why”. In general it is not possible to distinguish between a “real why” and a post hoc rationalization so the distinction is meaningless for practical purposes.
It's absolutely not meaningless if you work on code that matters. It matters a lot.
I don't care about philosophical "knowing", I wanna make sure I'm not gonna cause an incident by ripping out or changing something or get paged because $BIG_CLIENT is furious that we broke their processes.
If I show you two "why" comments in a codebase, can you tell which one was written by an LLM and which was not?
Just like humans leave comments like this
// don't try to optimise this, it can't be done
// If you try, increment this number: 42
You can do the same for LLMs
// This is here because <reason> it cannot be optimised using <method>
It works, I've done it. (In the surface that code looks you can use a specific type of caching to speed it up, but it actually fails because of reasons - LLMs kept trying, I added a comment that stopped them).
Of course I can't tell the difference. That's not the point. And yes, humans can leave stupid comments too.
The difference is I can ping humans on Slack and get clarification.
I don't want reasons because I think comments are neat. If I'm tracking this sort of thing down, something is broken and I'm trying to fix it without breaking anything else.
It only takes screwing this up a couple times before you learn what a Chesterson's Fence is lol.
You are framing this as an AI problem, but from what I’m hearing, this is just an engineering culture problem.
You should not bet on the ability to ping humans on Slack long-term. Not because AI is going to replace human engineers, but because humans have fallible memories and leave jobs. To the extent that your processes require the ability to regularly ask other engineers “why the hell did you do this“, your processes are holding you back.
If anything, AI potentially makes this easier. Because it’s really easy to prompt the AI to record why the hell things are done the way they are, whether recording its own “thoughts” or recording the “why” it was given by an engineer.
It's not an engineering culture problem lol, I promise. I have over a decade in this career and I've worked at places with fantastic and rigorous processes and at places with awful ones. The better places slacked each other a lot.
I don't understand what's so hard to understand about "I need to understand the actual ramifications of my changes before I make them and no generated robotext is gonna tell me that"
StackOverflow is a tool. You could use it to look for a solution to a bug you're investigating. You could use it to learn new techniques. You could use it to guide you through tradeoffs in different options. You can also use it to copy/paste code you don't understand and break your production service. That's not a problem with StackOverflow.
> "I need to understand the actual ramifications of my changes before I make them and no generated robotext is gonna tell me that"
Who's checking in this robotext?
* Is it some rogue AI agent? Who gave it unfettered access to your codebase, and why?
* Is it you, using an LLM to try to fix a bug? Yeah, don't check it in if you don't understand what you got back or why.
* Is it your peers, checking in code they don't understand? Then you do have a culture problem.
An LLM gives you code. It doesn't free you of the responsibility to understand the code you check in. If the only way you can use an LLM is to blindly accept what it gives you, then yeah, I guess don't use an LLM. But then you also probably shouldn't use StackOverflow. Or anything else that might give you code you'd be tempted to check in blindly.
It does actually work incredibly well. It's even remarkably good at looking through existing stuff (written by AI or not) and reasoning about why it is the way it is. I agree it's not "thinking" in the same way a human might, but it gets to a more plausible explanation than many humans can a lot more often than I ever would have thought.
There's always been Arch linux based distros that come with more things set up and better (or just more specific) defaults. To my understanding Omarchy is just one of those, like Manjaro or etc in the past?
A before / after with the reference and output seems useful to me, and maybe a range from more generic to more recognizable / celebrity voice samples so people can kinda see how it tackles different ones?
(Prominent politician or actor or somebody with a distinct speaking tone?)
I suppose it‘s the latter + maybe some finetuning, it’s definitely not like DeepSeek where the answer of the model get‘s replaced when you are talking something uncomfortable for China
I've seen an ai generated image of the interaction already on Twitter, so basically right now. I'm sure someone will have a version that looks similar to the surveillance footage but speeds up the car or edits something by next week?
Just leave your phone at home and bring a plain old small digital camera, agree ahead of time with friends on when and where to meet up. It's interesting to me and i guess showing my age that this isn't self evident to everyone everywhere.
I suspect the old school stuff is generally less monitored. I think some of the cheap Baofeng radios support AES256 encryption. I think that's technically only legal with a business license from the FCC or some such, but I'd be a lot less worried about an FCC fine than having my phone tracked. There's probably some quick keypresses to clear the encryption config so it looks like it was on plaintext.
Do not use devices that can be trivially tracked through the cell network, or that can be surveilled by big tech. This means a device bought anonymously, a free/libre OS like Graphene, no Google/Facebook/Apple spyware apps, and an anonymous SIM paid for with cash or crypto. This should be done by everyone to avoid the possibility of mass surveillance, not only people who have something to hide from a three-letter agency. If you really have something to hide, then the cellular network shouldn't be used at all.
>Do not use devices that can be trivially tracked through the cell network, or that can be surveilled by big tech. This means a device bought anonymously, a free/libre OS like Graphene
GrapheneOS isn't magically exempt from cell tracking, and both android and ios phones can go into airplane mode and have location disabled, which provides similar privacy.
>and an anonymous SIM paid for with cash or crypto. This should be done by everyone to avoid the possibility of mass surveillance, not only people who have something to hide from a three-letter agency.
No, it's much harder than just "an anonymous SIM paid for with cash or crypto". You need to practice proper opsec. There's no point getting an anonymous sim when you then turn around and then use it as a 2fa number for your bank, or carry it around with you every day.
> GrapheneOS isn't magically exempt from cell tracking, and both android and ios phones can go into airplane mode and have location disabled, which provides similar privacy.
You practically can't do anything on a Googled Android device or iOS without a Google or Apple account, so no, they don't provide "similar privacy." The point of a FOSS system is that the user fully controls it, and can install apps privately from any source.
>You practically can't do anything on a Googled Android device or iOS without a Google or Apple account, so no, they don't provide "similar privacy."
If you're talking about not being able to install third party apps, aurora store doesn't require an account and works fine on stock android. Most other basic functionality works fine too, eg. camera, calls, browsing, maps.
> If you're talking about not being able to install third party apps, aurora store doesn't require an account and works fine on stock android. Most other basic functionality works fine too, eg. camera, calls, browsing, maps.
The Play Store is not the only issue with stock Android devices. Google dependencies run with high privileges and the device is constantly communicating with Google servers for one reason or another. You do not own a Google device for all intents an purposes. The main contribution of Graphene here is that it strips out the proprietary blobs and optionally provides an environment to run Google's libraries with unprivileged access.
The point about de-Google'd Android vs your insistence on GrapheneOS is that by the time you are using Google's libraries, like Maps, Play Services, or their notification service (Firebase, IIRC), you've already lost. GrapheneOS is not dramatically better than de-Google'd Android if you're still sending all your notifications through Google, as well as your location and things like contacts
The point is you have to leave Google with both for it to do much good
> insistence on GrapheneOS is that by the time you are using Google's libraries, like Maps, Play Services, or their notification service (Firebase, IIRC), you've already lost.
Graphene offers the option of sandboxing Google apps should you want them. The usual setup is a second user profile with all the Google stuff in it. My main profile only contain FOSS apps and nothing passes through Google's servers. I use the Google profile for the maps with a dedicated account maybe once a month when driving somewhere unfamiliar.
i think these already have you screwed. that anonymity is going to be superficial at best. you will be recorded making these purchases, and tracked to your identity
Then you're going to take it home for >8 hours per day, and to your job several hours per day, and likely call at least one or two of your important contacts. At which point that's the ball game - the pool of people that live in the immediate vicinity of your building, and work in the immediate vicinity of your job site, and call your partner / parents / kid, is made up of pretty well exclusively you
The true answer is: Hold your politicians accountable for this at every level, including at the "boring" local level and on all levels all the way up to the top.
This type of problem needs to be fixed on the society level.
See my other comment. At least in this particular case the databrokers are getting the data from apps themselves. If you don't grant location permissions to shady weather/transit/delivery apps, you should be safe.
Without more information about how the system works, a casual "eh if I don't grant location data access to shady apps I'm probably safe" seems very risky. What apps are "shady"? How does the real-time bidding system obtain and divulge location data?
I think that it is not a safe assumption that the only way corporations are obtaining people's location is via OS location APIs.
>Without more information about how the system works, a casual "eh if I don't grant location data access to shady apps I'm probably safe" seems very risky.
I don't think anyone who actually is at risk, or cares about risk, is going to be overconfident about their security because some HN commenter said "you're probably fine".
>What apps are "shady"?
Depends on your paranoia level. I'd say first party apps (eg. apple/google maps/weather) are probably fine. Google has the additional caveat that they record location history and therefore might be subject to geofence warrants. If you think iOS/Android is backdoored then all phones are off limits.
>How does the real-time bidding system obtain and divulge location data?
They're whatever ad SDKs can get their hands on. If the app has location permissions, it's that. Otherwise it's something like geoip. At the end of the day it's just third party code running in some app's sandbox. If the app can't get it, the SDK can't get it either.
>I think that it is not a safe assumption that the only way corporations are obtaining people's location is via OS location APIs.
What other plausible mechanism are there then? wifi/bluetooth scanning requires location permissions since forever ago.
One way to minimize the info they gather is by using a dumb phone. I have a flip phone running some RTOS that doesn't allow any kind of apps and doesn't have GPS, meaning the only trace it leaves is any cell activity
Not use any device that has GSM/LTE, or Bluetooth.
Alternatively, broadcast a hidden SSID WiFi AP via an enabled RPi and use only devices that's have WiFi. Hand them out to people for free to increase the spread.
Attach magnets to the RPi's and go rogue by sticking them to buses, cars and trains et cetera to increase range.
Are there decent wifi communicators on the market? I looked into some Lora projects for this but they never seem to actually ship or get past prorotypes
> Are there decent wifi communicators on the market? I looked into some Lora projects for this but they never seem to actually ship or get past prorotype
Yes, 100%. Meshtastic and Meshcore both do this, but I'd recommend Meshcore. Here in the Seattle area we have a network that fairly reliably delivers messages from Canada through the Seattle metro area all the way down to Portland. Fully encrypted with dual key cryptography. Meshcore uses a different strategy than Meshtastic, which enables Meshcore to work more reliably. To see what's happening in your area for Meshcore see https://analyzer.letsmesh.net/map
Is very fun to set up a repeater for under $50 and see a noticeable difference in the coverage area. Is a fun technical project that combines the best of hiking/walking/driving geocaching style, ham radio (but without a license requirement), antenna building, and more. I'm getting acquainted with people in my neighborhood too which is a bonus.
Figuring out what hardware to buy that'll actually work can be a challenge, to get started search amazon for "heltec v3" and make sure you get something that includes a battery, and you'll see 2-packs of radios for $60. There's a web flasher at the above link that'll put the software on the radios for you.
The BSSID is still visible, and is the unique identifier any trackers will be looking for anyway. Also making the SSID hidden just means the AP isn't broadcasting it, any listeners can still see the SSID whenever any client interacts with the AP.
Hidden SSIDs are generally much worse for privacy than non-hidden ones, since all stations (clients in 802.11 terminology) need to constantly go around yelling "hey, is SSID abc available?" while they're not connected to any SSID.
Open sourcing the server code would make getting your own instance of it way easier, and maybe opening the app code so people should change the controls?
Does asking for tailwind directly in the prompt not get it looking in that direction? I wonder if you could get a large enough context to include the css directly too
I was more hoping to use the Tailwind UI components (or tailwind plus or whatever they're calling it now) with the LLM output. I don't think they offer downloadable components or whatever so the LLM would need a way of knowing which were available to use and be able to pull them in for reference. At least that's my assumption.
Does it need to be hosted on your servers? Could you provide something to the customers where they host the data or their local doctors office does it?
Can you delete it after the shortest possible period of using it, potentially? Do you keep data after someone stops being a customer or stops actively using the tech?
Record retention is covered by a complex set of overlapping regulations and contracts. They are dependent on much more than date of service. M&A activity, interstate operations, subsequent changes in patient mental status, etc can all cause the horizon to change well after the last encounter.
As all the comments in this thread suggest the cost of having an extra record , even an extra breached record is low. The cost of failing to produce a required medical record is high.
Put this together with dropping storage prices, razor then margins, and IT estates made out of thousands of specialized point solutions cobbled together with every integration pattern ever invented and you get a de facto retention of infinity paired with a de jure obligation of could-be-anything-tomorrow.
I'm not trying to be rude, but it's clear you have idea what you're talking about. The medical world is heavily regulated and there are things we must do and thing's we can't do. If you go to your doctor with a problem, would you want your doctor to have the least amount of information possible or your entire medical history? The average person has no business hosting their sensitive data like banking and medical information. If you think fraud and hacks are bad now, what do you think would happen if your parents were forced to store their own data? Or if a doctor who can barely use an EMR was responsible for the security of your medical data? I would learn a lot more about the area before making suggestions.
Having seen this world up close, the absolute last place you ever want your medical data to be is on the Windows Server in the closet of your local doctors office. The public cloud account of a Silicon Valley type company that hires reasonably competent people is Fort Knox by comparison.
Yeah but the a local private practice is a fairly small target. No one is going to break into my house just to steal my medical records, for example.
This could also be drastically improved by the government spearheading a FOSS project for medical data management (archival, backup, etc). A single offering from the US federal government would have a massive return on investment in terms of impact per dollar spent.
Maybe the DOGE staff could finally be put to good use.
You seem to be confused about how this works. Attackers use automated scripts to locate vulnerable systems. Small local private practices are always targeted because everything is targeted. The notion of the US federal government offering an online data backup service is ludicrous, and wouldn't have even prevented the breach in this article.
> Attackers use automated scripts to locate vulnerable systems.
I'm aware. I thought we were talking about something a bit higher effort than that.
> online data backup service
That isn't what I said. I suggested federally backed FOSS tooling for the specific usecase. If nothing else that would ensure that low effort scanners came up empty by providing purpose built software hardened against the expected attack vectors. Since it seems we're worrying about the potential for broader system misconfiguration they could even provide a blessed OS image.
The breach in the article has nothing to do with what we're talking about. That was a case of shadow IT messing up. There's not much you can do about that.
I just registered CVEs in several platforms in a related industry, the founders of whom likely all asked themselves a similar question. And yet, it's the wrong question. The right one is, "Does this company need to exist?" I don't know you or your company. Maybe it's great. But many startups are born thinking there's a technological answer to a question that requires a social/political one. And instead of fixing the problem, the same founders use their newfound wealth to lobby to entrench the problem that justifies their company's existence, rather than resolves the need for it to exist in the first place. "How do you propose we service our customers without their medical data?" Fix your fucked healthcare system.
Otherwise it would suggest you think the problem is they didn't ask? When was the last time you saw a customer read a terms of service? Or better yet reject a product because of said terms once they hit that part of the customer journey?
The issue isn't about asking it's that for take your pick of reasons no one ever says no. The asking is thus pro forma and irrelevant.
reply