The downsides and caveats are very similar to writing mod_perl based web apps last I saw those docs. The downsides shouldn't be a problem for most Nagios plugins though since they shouldn't require state hopefully.