I mean developing taste is what RLHF was supposed to give you. I don't think it's actually a technical problem so much as a social one. The average person _wants_ slop, they don't want to read new yorker articles, they'd much rather read romcoms. A model trained to produce tasteful writing would almost surely have less engagement from the public (considering that engagement and lmarena-maxing is what led to the characteristic punchy style in the first place)