Apparently Claude Code being built on Bun was considered a good enough reason? But it looks more strategic for Bun since they’re VC-backed and get a good exit:
> Claude Code ships as a Bun executable to millions of users. If Bun breaks, Claude Code breaks. Anthropic has direct incentive to keep Bun excellent.
…
> Bun's single-file executables turned out to be perfect for distributing CLI tools. You can compile any JavaScript project into a self-contained binary—runs anywhere, even if the user doesn't have Bun or Node installed. Works with native addons. Fast startup. Easy to distribute.
> Claude Code, FactoryAI, OpenCode, and others are all built with Bun.
…
> Over the last several months, the GitHub username with the most merged PRs in Bun's repo is now a Claude Code bot. We have it set up in our internal Discord and we mostly use it to help fix bugs. It opens PRs with tests that fail in the earlier system-installed version of Bun before the fix and pass in the fixed debug build of Bun. It responds to review comments. It does the whole thing.
> This feels approximately a few months ahead of where things are going. Certainly not years.
…
> We've been prioritizing issues from the Claude Code team for several months now. I have so many ideas all the time and it's really fun. Many of these ideas also help other AI coding products.
…
> Instead of putting our users & community through "Bun, the VC-backed startups tries to figure out monetization" – thanks to Anthropic, we can skip that chapter entirely and focus on building the best JavaScript tooling.
Same thought and I can't wait for a video with a very confused Theo...
I mean, it's likely very important for them to have a fast and sandboxed code executor available. But it's not like Bun would fight against improvements there or refuse paid work on specific areas, right?
As pointed out by a sibling comment. MOE consists of a router and a number of experts (eg 8). These experts can be imagined as parts of the brain with specialization, although in reality they probably don't work exactly like that. These aren't separate models, they are components of a single large model.
Typically, input gets routed to a number of of experts eg. top 2, leaving the others inactive. This reduces number of activation / processing requirements.
Mistral is an example of a model that's designed like this. Clever people created converters to transform dense models to MOE models. These days many popular models are also available in MOE configuration
that's not really a good summary of what MoEs are. you can more consider it like sublayers that get routed through (like how the brain only lights up certain pathways) rather than actual separate models.
The gains from MoE is that you can have a large model that's efficient, it lets you decouple #params and computation cost. I don't see how anthropomorphizing MoE <-> brain affords insight deeper than 'less activity means less energy used'. These are totally different systems, IMO this shallow comparison muddies the water and does a disservice to each field of study. There's been loads of research showing there's redundancy in MoE models, ie cerebras has a paper[1] where they selectively prune half the experts with minimal loss across domains -- I'm not sure you could disable half the brain and notice a stupefying difference.
> I don't see how anthropomorphizing MoE <-> brain affords insight deeper than 'less activity means less energy used'.
I'm not saying it is a perfect analogy, but it is by far the most familiar one for people to describe what sparse activation means. I'm no big fan of over-reliance on biological metaphor in this field, but I think this is skewing a bit on the pedantic side.
re: your second comment about pruning, not to get in the weeds but I think there have been a few unique cases where people did lose some of their brain and the brain essentially routed around it.
One thing that the Chinese are really good at is cost innovation, reducing costs as many ways possible to make their products affordable for the majority. Their aim for good enough quality.
I bet the sales ratio of the Chinese vs the German sharpeners exceed 20:1
If they did adopt JSON then it wouldn't be XMPP anymore.
As a user, I don't care much. But my experience with XMPP is that was not as solid as other solutions, including closed source ones. I could've been issues in clients' implementation, but overall it wasn't great
> my experience with XMPP is that was not as solid as other solutions, including closed source ones.
Considering that WhatsApp is essentially an old/non-federating fork of ejabberd (from where this blog post originates), I think XMPP is doing alright in that regard.
Anecdotal, my uncle, several friends' relatives died from COVID during those lockdowns. I don't know / heard anyone died because of flu (in my extended circle of people I know)
Most people are not a "hospitalized older adult". Yet they treated everyone, regardless of age, gender, health as if they were on death's door. The lock downs were absolutely overkill and went on far, far too long.
7B:Hi! I'm Olmo 3, an AI assistant created by the non-profit organization Ai2. I'm here to help with questions, ideas, or tasks you have—just let me know what you need! How can I assist you today? Rawr!
32B:
Hi! I'm Olmo, a helpful AI assistant built by the Allen Institute for AI (Ai2). My knowledge is up to December 2024, and I'm designed to assist with a wide range of tasks. How can I help you today?
If I instead don't, and let you know that the key is there in the source code, hopefully at least one deserving person might learn how to look through source code better, and none of the lazy people get it :)
