I found this really hard in my career though - you move into an IC role and then the assumption is you won't be happy in a management role ever again. You move into a management role and somehow there's an assumption that you have been lobotomized and know nothing about the hands-on work any longer. It's not fair, but it can really create complications as you attempt to move your career into new directions or attempt to move to new employers.
I've been trying to counter that effect by saying "have you heard about the pendulum model?" and showing that article to as many people in influential positions as possible.
Eh, people will find a reason to judge you anyways. I am jarred to the point of thinking none of it matters, usually the person has an opinion of me before I even meet them anyways.
Conversely, trying to move into relatively in demand things where supply / demand curve is bent out of shape helps. You can't discriminate for willy nilly reasons if you can't find enough candidates.