If you wanted anything other than hand waving, you can't even define a normal distribution (because you can't define `exp`) without limits. `exp` is a transcendental function.
You don't have to lie to students to define the exponential function itself. They are familiar with pi, so it is not difficult to introduce another irrational number, and define f(x) = e^{-x^2}. There is no lying here.
The need for limits and such arises when you try to differentiate the exp function. But we don't need differentiation for basic statistics. Just integration.
How do you introduce e? Just say it's 2.718...? The usual definitions of e involve a limit, and to my knowledge there's no simple geometric definition like there is for pi. Likewise I don't know of a definition of exp that doesn't involve a limit, and there's no simple geometric one that I know of like there is for sin/cos.
(There is a geometric definition of exp that I know of, but it's that it turns a vector into an integral curve, so not so useful without calculus or limits)
You also need limits to be able to talk about the central limit theorem/when a normal even ought to be used. Otherwise you get confused people thinking everything is normally distributed by default.
Remember that my larger point is that statistics can't be taught without teaching most of calculus, so just teach calculus first anyway. But, if we hypothetically tried to get away with the minimum...
> The usual definitions of e involve a limit.... I don't know of a definition of exp that doesn't involve a limit ...
Yes, I agree with you that if you want to define e as interesting itself, you need to use limits. Similarly, the way exp(x) was introduced to me in high school was as a function whose derivative was equal to itself (i.e. as an interesting function) - which also requires limits but I think my teacher/curriculum just handwaved away that part.
But in our hypothetical curriculum, I am indeed proposing that we just say that e = 2.718..., since we are not interested in e, but in its usage for defining continuous probability distributions. Then to compute something like e^2 you just plug it into the calculator (like you do sin/cos) and it will give the answer. But again, we will have to put in effort to argue that something like e^(5/4) or e^pi is a computable real number.
> You also need limits to be able to talk about the central limit theorem
Indeed, but I think rigorous usage of the central limit theorem is quite beyond high school mathematics.
I was agreeing with/augmenting your larger point: I don't see how you can do any justice to the subject at all without calculus, just like I don't see the point in teaching a bunch of solutions to memorize to particular setups and calling it physics.
Even Bayes' theorem is, IMO, most obvious in the continuous setting where you can interpret it in terms of relative areas, which gives a nice, easy picture. Making big tables and trees obscures the basic geometry.