For all that I agree there's a big pile of snake oil in this area, I disagree with you overall.
Having played with Markov models since I was a kid*, LLMs are really not just that.
All that stuff you acknowledge but then gloss over, that is the actual learning, which tells it which previous tokens are relevant and how much attention to pay to them. This learning creates a world model which is functionally (barely but functionally) performing something approximately reasoning.
Statistics and probability is the mechanism it uses to do this, but that alone doesn't make it a Markov chain, those are a very specific thing that's a lot more limited.
For example: consider a context window of 128k tokens where each token has 64k possible discrete values. If implemented as a Markov chain, this would need a transition matrix of (2^16)^(2^17) by (2^16) entries (unless I've gotten one of the numbers backwards, but you get the idea regardless). This is too many, and because it is too many you have to create a function to approximate those transitions. But even then, that only works as a Markov chain if it's a deterministic function, and the actual behaviour is not deterministic due to the temperature setting not (usually) being zero.
* Commodore 64 user guide aged 6 or so, so I didn't really understand it at the time but that's what it was
Off the top of my head and 34 years later, sadly not.
Best I can do is describe this code/listing:
It created a few lists of words (IIRC these lists were adjectives, nouns, verbs?), and in the main loop it kept track of which list it had just taken a word from in order to decide what to pick next — e.g. if it had just picked an adjective then was allowed to pick either another adjective or go onto a noun, if it had just picked a noun then it could end the sentence or it could go on to a verb — and it would pick a random word from whichever list.
This either fit entirely on one screen, or very close to that — I was little, I would have made a syntax error if it had been much longer.
(Perhaps it will turn out to be my family's user manual wasn't even the official one, though I do remember it being a thick blue thing which matches the pictures I've seen online).
Having played with Markov models since I was a kid*, LLMs are really not just that.
All that stuff you acknowledge but then gloss over, that is the actual learning, which tells it which previous tokens are relevant and how much attention to pay to them. This learning creates a world model which is functionally (barely but functionally) performing something approximately reasoning.
Statistics and probability is the mechanism it uses to do this, but that alone doesn't make it a Markov chain, those are a very specific thing that's a lot more limited.
For example: consider a context window of 128k tokens where each token has 64k possible discrete values. If implemented as a Markov chain, this would need a transition matrix of (2^16)^(2^17) by (2^16) entries (unless I've gotten one of the numbers backwards, but you get the idea regardless). This is too many, and because it is too many you have to create a function to approximate those transitions. But even then, that only works as a Markov chain if it's a deterministic function, and the actual behaviour is not deterministic due to the temperature setting not (usually) being zero.
* Commodore 64 user guide aged 6 or so, so I didn't really understand it at the time but that's what it was