I mean "in software" in that it's not hardware assisted. I have gotten VAAPI working but it's a bit flaky with some videos for some reason, so I disabled it and just do vanilla ffmpeg.
Installing backported kernels & mesa versions on my aging Debian version got me much better results with VA-API, for what it's worth (that was an AMD card).
I'll look into the distributed ffmpeg.