Look up the Osborne 1, the first "portable" (i.e. luggable) computer. They went out of business not only because they lost money on each unit, but because of how many they sold. Then they pre-announced their next model, which killed all demand for the existing one, and they were toast.
It's a fascinating story, but is it really related?
IIRC, they were making decent-enough profits with the Osborne 1 at the beginning. It was never intended to be a loss-leader.
It was only after the Osborne 2 was announced (way too early) that existing orders got cancelled, and inventory was sold at fire-sale prices in sheer desperation to generate any value from the well they'd accidentally poisoned.
(For those who don't know, the company imploded before the Osborne 2 was finished.)
IMO Yes it is related. Anthropic and OAI, etc, announced "AGI" and "super intelligence" way too early.
Before the SaaS model teams can release Grand Opus and Really-This-Time-It's-AGI-Promise! GPT, local models and optimized hardware will obsolete them.
The lag from design of manufacturing adjustments needed to shipped hardware is 6-7 years. Still 3-5 more years for first gen hardware influenced by the post-GPT launch era. Right now we're just getting the early tweaks as models keep developing, providing feedback in a sense to where to take hardware.
Everyone will not see it coming as they fetishize every little thing to do with the circus intentionally erected around OAI and company to keep them relevant. We’ll have another All You Need Is Attention and ChatGPT release moment and SaaS models are hosed.
It's orders of magnitude more difficult than the era of the Osborne 2 so the timelines are longer but from inside the hardware industry, can say that is indeed the goal.
Right, that's what the various memory ordering constants are for in C++ atomics, and other languages will likely have an equivalent. On such architectures, those will emit special instructions or barriers.
That's what "lock-free" means. You still need to use the hardware mechanisms provided for atomicity.
The whole point of lock-free data structures and algorithms is that sometimes you can do better by using these atomic operations inside your own code, rather than using a one-size-fits-all mutex based on those same atomic operations.
(Note that I say "sometimes". Too many people believe that lock-free structures are always faster; as always, your mileage may vary. In this case it's a huge win, to the point where I would bet it almost always moves the bottleneck to the code actually using the ring buffer.)
My point is that the "huge win" is expressed in terms of a bogus and misleading baseline. The article moves immediately from the worst possible lock-based implementation to a pretty bad atomics-based implementation. The final punchline of the article is expressed as a ratio of the bad baseline. To make an honest conclusion, the article should also explore better ways of using the locks.
It's precisely the way we teach people how to build thread-safe systems. And we teach them to do it that way because we've learned from experience that letting them code up their own custom synchronization primitives leads to immense woe and suffering.
(and it's not slow because of the C++ mutex implementation, either - I tested a C/pthreads version, and it was the same speed as the C++ version)
The GNU libstdc++ STL mutex is nothing but pthread_lock, so that's not a surprise.
I really don't understand what you are saying about not using custom primitives. The whole article is "YOLO your own synchronization" and it fails to grapple with the subtleties. An example of the unaddressed complexity: use of acquire-release semantics for head_ and tail_ atomics imposes no ordering whatsoever between observations of head_ and tail_. The final solution has four atomics that use acquire-release and does not discuss the fact that threads may observe the values of these four things in very surprising order. The issue is so complex that I consider this 50-page academic paper to be the bare minimum survey of the problem that a programmer should thoroughly understand before they even consider using atomics.
"An example of the unaddressed complexity: use of acquire-release semantics for head_ and tail_ atomics imposes no ordering whatsoever between observations of head_ and tail_."
The Acquire / Release in version 4 looks right to me, but I'd like to know if I'm missing something.
Also, while your linked paper is good background for what the C++11 memory model is intended to abstract over, it's almost entirely its own thing with a mountain of complexity.
Somebody else in this comment section brought atomics knowledge to an Acquire/Release fight and it didn't go well.
As a starting introduction I'd probably recommend this:
I think he's complaining that because the head_ and tail_ loads in push/pop are relaxed, rather than also being acquire, they can be reordered relative to the acquire tail_ and head_ loads respectively. I don't believe this impacts the correctness of the logic, but I could be missing something.
You dismissed the standard lock-guarded data structure as a "bogus comparison", despite it being the way every programmer is taught to write multi-threaded code.
Now the more you write, the more you seem to make the case that (a) normal programmers shouldn't be writing code like this, and (b) there are significant speedups possible if someone who knows what they're doing *does* write a highly tuned lock-free library.
The easy speedup is to use 2 mutexes, one that protects head and tail_cached, and the other that protects tail and head_cached, and align so they don't interfere. In other words, take the RingBufferV5 from the article and define the class like this:
Then change the code to forget the atomics and just use the locks. On my system this is more than ten times faster than the baseline naïve thread-safe RingBufferV2. That's what I mean about using a bogus baseline.
I would point out that most products are useless, and either fail or replace other products which weren't any worse. None of which prevented me from cashing my paychecks for the first half of my career when I worked in private industry.
Most scientific research represents about the same amount of improvement over the state of the art as the shitty web app or whatever that you're working on right now. It's not zero, but very few are going to be groundbreaking. And since the rules are that we all have to publish papers[*], the scientific literature (at least in my field, CS) looks less like a carefully curated library of works by geniuses, and more like an Amazon or Etsy marketplace of ideas, where most are crappy.
[* just like software engineers have to write code, even if the product ends up being shitty or ultimately gets canceled]
Neither of us are going to be changing how the system works, so my advice is to deal with it.
Are there any factual allegations on that page? All I could find was "the method described in the paper is not the method the authors actually used", without any elaboration.
I'll add that the reaction of most of academia will be "It's in a management journal - of course it's nonsense."
90% of the power in our academic data center goes 13.8kV 3-phase -> 400v 3-phase, and then the machines run directly from one leg to neutral (230v). One transformer step, no UPS losses, and the server power supplies are more efficient at EU voltages.
But what about availability? If you ask most of our users whether they’d prefer 4 9s of availability or 10% more money to spend on CPUs, they choose the CPUs. We asked them.
There are a lot of availability-insensitive workloads in the commercial world, as well, like AI training. What matters in those cases is how much computing you get done by the end of the month, and for a fixed budget a UPS reduces this number.
> and then the machines run directly from one leg to neutral (230v)
And then every machine has a switching power supply to convert this to low-voltage DC, and then probably random point-of-load converters in various places (DC -> AC -> DC again) for stuff like the CPU / GPU core, RAM, etc. Each of these stages may be ~95% efficient with optimal load, but the losses add up, and get a lot worse outside a narrow envelope.
Yes, but it's not like any other layouts avoid those issues.
You could feed your servers off fat 12/24/48 volt supplies but with how much power a modern server can pull you're already converting in bulk even if you don't do that, limiting the potential advantages. For running CPU/GPU/RAM, there is no other option. When you need hundreds of amps at 1-2 volts, you convert that centimeters away if at all possible.
A datacenter using DC distribution is still using high voltages and stepping them down in layers. The hassle it avoids is in other aspects of power delivery.
I was part of an electrician team renovating a three-letter agency's main data center (adding UPS to an archaic government renovation). As part of this process, we had to isolate the Emergency Power Off system/buttons.
Unfortunately, the EPO ended up having been installed differently than the plans indicated [0] — there was definitely more than an hour of unexpected downtime figuring out how to "unrig" its decades of faithful misconfiguration.
During this same months-long modernization, a separate facility decided to install a temporary tap off our already-temporary taps (no engineering consultation attempted) — with resulting Megawatts of melting disaster. More hours downtime.
tl;dr: shit happens
[0] EPO's main neutral was tied into a general lighting circuit's j-box, some bullshit jerryrigging from an 80s sparkie; our apprentice was tasked with taking out old fluorescents... ended up taking out entire facility (through no fault of his own).
I remember ordering parts from Digi-Key in 1980 or so when I was in high school. The catalog was less than 1/4 inch thick, and listed various surplus things on the back.
It was cool to see them grow into a real competitor for the big distributors.
and select "Mathematics and Computer Science", you'll find the top-ranked university is the University of Electronic Science and Technology of China.
My Chinese colleagues have heard of it, but never considered it a top-ranked school, and a quick inspection of their CS faculty pages shows a distinct lack of PhDs from top-ranked Chinese or US schools. It's possible their math faculty is amazing, but I think it's more likely that something underhanded is going on...
If you're writing code professionally, then you're not in college anymore and your programs aren't simple things that run from the start of main() through to the end and then exit.
If you're providing a service that needs to keep running, you need a strategy for handling unexpected errors. It can be as simple as "fail the request" or "reboot the system", or more complicated. But you need to consider system requirements and the recovery strategy for meeting them when you're writing your code.
Long, long ago I worked with some engineers who thought it was just fine that our big piece of (prototype) telecom equipment took half an hour to boot because of poor choices on their part. Target availability for the device was 5 9s, which is 5 minutes of downtime per year. They didn't seem to realize the contradiction.
> If you're writing code professionally, then you're not in college anymore and your programs aren't simple things that run from the start of main() through to the end and then exit.
I've boycotted reviewing for Elsevier for years, but it's easy for me - I'm in CS, where ACM, USENIX and IEEE offer higher-status publication venues and Elsevier journals are decidedly second-tier.
reply