I don't know for sure, but is very common to see parents carrying children facing backwards in their arms, while walking, with no ill effects. No big deal.
But you don't notice that speed.. (ref. Newton and Einstein and others). I agree with those saying it must be psychological.. well, mostly. Sometimes you do physically notice that you're facing backwards. At takeoff, for example. And, to some extent, landing. Maybe when banking too. I haven't tried, obviously (the backwards-facing unfolding seats for the crew would be the place to test this).
