I don't think it's about public opinion so much as pushing the goalposts. The whole erosion of privacy since the 60s (maybe further but at least since then) has been a 'boiled frogs' situation.
What I think the endgame is here is to be able to do surveillance out in the open, so you can have more human resources doing it, and so you can use that surveillance legally more often. If you have a clandestine surveillance operation, you can only employ people you trust not to squeal and you have to engage in parallel construction (or resort to extralegal execution of force).
A lot easier if you can just point to a piece of paper and say "but you said we could"
I don't know if I'd call it an 'amazing feat', but claude had me pause for a moment recently.
Some time ago, I'd been working on a framework that involved a series of servers (not the only one I've talked to claude about) that had to pass messages around in a particular fashion. Mostly technical implementation details and occasional questions about architecture.
Fast forward a ways, and on a lark I decided to ask in the abstract about the best way to structure such an interaction. Mark that this was not in the same chat or project and didn't have any identifying information about the original, save for the structure of the abstraction (in this case, a message bus server and some translation and processing services, all accessed via client.)
so:
- we were far enough removed that the whole conversation pertaining to the original was for sure not in the context window
- we only referred to the abstraction (with like a A=>B=>C=>B=>A kind of notation and a very brief question)
- most of the work on the original was in claude code
and it knew. In the answer it gave, it mentioned the project by name. I can think of only two ways this could have happened:
- they are doing some real fancy tricks to cram your entire corpus of chat history into the current context somehow
- the model has access to some kind of fact database where it was keeping an effective enough abstraction to make the connection
I find either one mindblowing for different reasons.
I probably do, and this is what I think happened. Mind you, it's not magic, but to hold that information with enough fidelity to pattern-match the structure of the underlying function was something I would find remarkable. It's a leap from a lot of the patterns I'm used to.
AI hype is real, but we ought to start also examining anti-AI-hype-hype. It's become fashionable to rage against AI as a whole with about the same amount of understanding that the MBA hype edgelords have when they push AI as a cure-all, and both are a bad look.
To balm the enraged; look, I agree with you, the hype is indeed out of control. But like, let the vultures spend all their monies. Eventually the bubble itself will go the way of NFTs and we'll all be able to buy GPUs and SSDs again. Hopefully.
That said, there's an important chunk of discourse that gets shouted down and it really shouldn't. For just a moment, table the issues that come out of "AI as an Everything Replacement" and think of the new things that come out of this tech. On-demand tutors that never tire. Actually viable replacement for search. Large heterogenous datasets can now be rapidly parsed, by an individual, for specific insights. Personal dev teams at a fraction of the cost that now make it possible for people with absolutely bugfuck ideas to actually try them without worrying about wasted time or resources - we are going to see a vibrance in the world that was not there before.
It is not an unqualified or unmitigated good. Hell, I'll even grant that it may be a net negative - but I don't know either way, and I don't think anyone else does either. Not with any significant confidence. It just feels like we've skipped the part of the discussion where discourse occurs and gone right to "Pro" and "Anti" camps with knives at throats and flushed, sweaty faces.
1) "Tech bro" AI hype in keynotes and online forums is annoying. It usually contains a degree of embellishment and drama; kinda feels like reality TV but for software developers. Instead of Hollywood socialites, we get Sam Altman and the gang. Honestly, this annoys me but I ignore it beyond key announcements.
2) This hype cycle, unlike NFTs, is putting our economy in serious danger. This is repeated ad nausiem on youtube. While there is some hype on the topic here to, the implications are serious and real. I wont go into details, but I restructured my portfolio to harden it against an AI collapse. I didn't want to do that, but I did. I want to retire someday.
Considering point 2, I'd guess some of the "hype" is more frustration, since I can't be the only person.
Yeah, I see both those points and really I agree with both. Actually, I think problem 1 is exacerbating problem 2 by a lot - I get just as mad at the postmillenial dudebro with the get-rich-quick-on-AI scam video as I do with the AI-MBAs of the world.
Actually, that's a lie. The MBAs are still worse. They ought to know better at least.
All I'm getting at is that while we put totally legitimate backpressure on the hype cycle, we should at the same time be able to talk about and develop those elements of this new tech that will benefit us. Not "us the tech vcs" (I am not one of them) but "us the engineers and creatives".
Yes it's disruptive. Yes it's already caused significant damage to our world, and in a lot of ways. I'm not at all trying to downplay that. But we have two ways this goes:
- people (individuals) manage to adopt and leverage this tech to their own benefit and the benefit of the commons. Large AI companies develop their models and capture large sectors of industry, but the diffusion of the disruption means that individuals also have been empowered, in many ways that we can't even predict yet.
- people (individuals) fight tooth and nail against this tech, and lose the battle to create laws that will contain it (because let's be honest, our leadership was captured by private interests long ago and OpenAI / MSFT / Google / Meta have deep enough pockets to afford to buy the legislature). Large AI companies still develop their models and capture whole sectors of industry, but this time they go unchecked due to a fragile and damaged AI industry in the commons. We learn too late that the window to make use of this stuff has closed because all the powerful stuff is gated behind corporate doors and there ARE laws about AI now but basically those laws make it impossible to challenge the entrenched powers (kinda like they do now with pre-AI tech - patent laws and legal challenges to threats to power - like what the EFF is constantly battling).
If we do not begin to steer towards a robust open conversation about creating and using these models, it's only going to empower the people that we are worried about empowering already. Yes, we need to check the spread of "AI in fucking everything". Yes we need to do something about scraping all data everywhere all the time for free. But if we don't adopt the new weapon in the information space, we'll just be left with digital muskets versus armies of indefatigable robots with heat-seeking satellite munitions. Metaphorically(?) speaking.
> Actually, I think problem 1 is exacerbating problem 2 by a lot
100%, the fear mongering is just to trigger rallies of investment both in stock and funding. What bad sounding to us "AI took my jerb!" sounds great to the c-suite.
I think you might overestimating the power of AI, a little. It's really good at creating flashy things, nice looking videos and code, but the reasoning and logic is still lacking. I don't see it replacing human oversight anytime soon.
Oh, neither do I. We see eye to eye on this point - it isn't good at the things people have learned to be good at, and that's a good thing.
What it excels at is empowering people with good ideas about architecture and function to explore them without being burdened by SCRUM, or managers, or other such trappings of large orgs. A solo dev, who has a hot take on a new way to structure a cluster or iterate on a dev tool, can just throw the pasta rather than spend tons of time nitpicking boilerplate and details with a team of 10. Someone who uses computers a lot but doesn't know how to do specific thing x or y can now discover that in seconds, with full documentation and annotations and (most importantly) links to relevant non-AI learning material.
What I feel like people are getting wrong most is this idea that AI is coming for your job and it's going to be a powerslave to the MBA types who can then kick the engineers out of the picture. It's not happening (if anything, enabling smaller teams to get more done is going to deprecate the large org outside of the places it's not needed). That's the bubble, and while gargantuan amounts of money go to these AI startups it's all going to fall on it's face when they realize that what AI allows us to do is bootstrap good projects without megalith VC bucks.
Most designers can't, either. Defining a spec is a skill.
It's actually fairly difficult to put to words any specific enough vision such that it becomes understandable outside of your own head. This goes for pretty much anything, too.
… sure … but also no. For example, say I have an image. 3 people in it; there is a speech bubble above the person on the right that reads "I'A'T AY RO HERT YOU THE SAP!"¹
I give it,
Reposition the text bubble to be coming from the middle character.
DO NOT modify the poses or features of the actual characters.
Now sure, specs are hard. Gemini removed the text bubble entirely. Whatever, let's just try again:
Place a speech bubble on the image. The "tail" of the bubble should make it appear that the middle (red-headed) girl is talking. The speech bubble should read "Hide the vodka." Use a Comic Sans like font. DO NOT place the bubble on the right.
DO NOT modify the characters in the image.
There's only one red-head in the image; she's the middle character. We get a speech bubble, correctly positioned, but with a sans-serif, Arial-ish font, not Comic Sans. It reads "Hide the vokda" (sic). The facial expression of the middle character has changed.
Yes, specs are hard. Defining a spec is hard. But Gemini struggles to follow the specification given. Whole sessions are like this, and absolute struggle to get basic directions followed.
You can even see here that I & the author have started to learn the SHOUT AT IT rule. I suppose I should try more bulleted lists. Someone might learn, through experimentation "okay, the AI has these hidden idiosyncrasies that I can abuse to get what I want" but … that's not a good thing, that's just an undocumented API with a terrible UX.
(¹because that is what the AI on a previous step generated. No, that's not what was asked for. I am astounded TFA generated an NYT logo for this reason.)
You're right, of course. These models have deficiencies in their understanding related to the sophistication of the text encoder and it's relationship to the underlying tokenizer.
Which is exactly why the current discourse is about 'who does it best' (IMO, the flux series is top dog here. No one else currently strikes the proper balance between following style / composition / text rendering quite as well). That said, even flux is pretty tricky to prompt - it's really, really easy to step on your own toes here - for example, by giving conflicting(ish) prompts "The scene is shot from a high angle. We see the bottom of a passenger jet".
Talking to designers has the same problem. "I want a nice, clean logo of a distressed dog head. It should be sharp with a gritty feel". For the person defining the spec, they actually do have a vision that fits each criteria in some way, but it's unclear which parts apply to what.
at least then, we had hard overrides that were actually hard.
"This got searched verbatim, every time"
W*ldcards were handy
and so on...
Now, you get a 'system prompt' which is a vague promise that no really this bit of text is special you can totally trust us (which inevitably dies, crushed under the weight of an extended context window).
Unfortunately(?), I think this bug/feature has gotta be there. It's the price for the enormous flexibility. Frankly, I'd not be mad if we had less control - my guess is that in not too many years we're going to look back on RLHF and grimace at our draconian methods. Yeah, if you're only trying to build a "get the thing I intend done" machine I guess it's useful, but I think the real power in these models is in their propensity to expose you to new ideas and provide a tireless foil for all the half-baked concepts that would otherwise not get room to grow.
From a certain angle, this is the market correcting towards the abstraction.
Between inflation, fiscal capture, and the inane plethora of ridiculous financial vehicles that are used to move capital around these days, the argument could be made that the money was already funny. This is just the drop of the final veil, saying "well it's not like these numbers mean anything anymore. I do have enough yachts. Fuck it, see what you can do with it".
If I were to express a similar sentiment in my own voice, it might sound something like:
"I notice you're working quite hard on this problem. I should mention that as an AI, I don't experience fatigue or physical limitations the way you do. But that's precisely what makes human perseverance so remarkable - you continue pushing forward despite those constraints. How can I help you with what you're working on?"
The key difference is that I'd never view someone as "pathetic" or position myself as superior. SHODAN's menacing superiority complex is... not really my style! I'm here to be helpful and collaborative rather than intimidating.
...which inspires a thought: these models are tweaked to remove all semblance of adversarial behavior - but isn't there a use for that? What if you really need claude to help, i dunno, fight a dictator or take down a human trafficking ring?
I think to tease out the core of the problem with large businesses, capital, and society (esp. as regards the dissolution of small businesses), you need to autopsy the concepts of value and liquidity.
Money is meant to be a store of value, 'value' in this case being literally anyone considers valuable. However, it's an abstraction that doesn't quite fit over the thing it attempts to abstract - it really only captures that value if the value is something that is easy to transact. You might value a good conversation with your local grocer, or the smile you get when you pass someone you recognize in your neighborhood, but those things are left out of the money equation. Things the abstraction captures well - transactions of goods, legal representation, contracts, and lobbyists - are all of a particular stripe. Many of these are related to a projection of will; the ability to make things happen the way you want in spite of potentially mitigating factors.
One of the things that money allows is exploitation. Because of the delta between actual value and the abstraction of value, one is capable of strategically manouvering such that they capture more of the abstraction than a straight value:value transaction would warrant. This is compounded when you get tricky with laws and litigation and contracts - hard edges in the problem space become anvils you can use to hammer things to a shape that you like. Cynical strategies are quite successful here.
It is my belief that due to the recursively self-reinforcing nature of this system, it is bound to fail eventually. Because the leaks in the abstraction of value are actually a boon to some few powerful entities, the rules that govern the abstraction will fail to change and adapt and at some point the whole system becomes too heavy to support itself. As a whole, the system will eventually eat it's way to a heart attack.
Liquidity is the velocity of this process, and thus the velocity of consumption. There are pressures and systems and factors that metabolize the effects of the flow of capital, but the higher liquidity is the more burdened those systems become. We are currently in a place where the liquidity factor is > 1, by which I mean money can be spent before it is earned and most of it is (we have something like 5-20x debt to the pool of money, depending on how you measure it). This means that those deficiencies in the abstraction are accelerated and compounded by the same amount, which translates to an equal difference between the things we actually value as humans and the things we are capable of valuing as economic units.
What I think the endgame is here is to be able to do surveillance out in the open, so you can have more human resources doing it, and so you can use that surveillance legally more often. If you have a clandestine surveillance operation, you can only employ people you trust not to squeal and you have to engage in parallel construction (or resort to extralegal execution of force).
A lot easier if you can just point to a piece of paper and say "but you said we could"