Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

May I suggest that you try sampling from a high-dimensional distribution and see how many samples end up near the mode? For example, try a 50-dimensional IID unit gaussian and check how often r = sqrt(x_1^2 + ... x_50^2) < 0.25 * sqrt(D)? You can also work this out analytically -- it's the classic example of concentration of measure.

By construct samples from a distribution will concentrate in neighborhoods of high probability mass, and hence the typical set.



It was you who said that it was better to exclude the mode from the typical set because "the additional evaluations near the mode would strictly add more cost". Now you tell me that evaluations near the mode are extremely unlikely. Something that, believe it or not, I understand. Maybe you would have liked it better if I had talked about one sample in one trillion being near the mode. And in that case that evaluation wouldn't be wasted because its contribution to the computed integral would be more important than if I had sampled by chance another point in the typical set with lower probability density.

Excluding the region with highest probability density from the typical set is a bit like saying that the population of New York is concentrated outside NYC because most people lives elsewhere.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: