OK, and? I could guess what the problem with this is, but let's spell it out... maybe that they are subjective so different assessors can disagree... and this is a problem why?
What if there are some cases where it's actually important to use human-assessed metrics as one component, or even as the entire thing, cases where no appropriate 'objective' metrics are available? (In scare quotes, because these quantitative metrics are seldom _quite_ as 'objective' or independent from human judgement as assumed. There is usually human judgement involved in how the metrics are defined and measured, where different people might define and measure a metric different ways resulting in different numbers...)
There are definitely some cases where it's important to use human-assessed metrics, like cooking a meal.
But if you are the size of Facebook, Instagram, Tiktok, etc. it is too risky (and expensive) to let individuals define what success looks like - you need something measurable that you can spread to multiple projects or even across the complete company culture.
I fully agree with your point that metrics are typically not objective, like any data once it's used to communicate something.
So the question should be what metric(s), if not engagement in itself, should we measure for to create a healthy online community?
I wonder if it's less about metrics and more about principles; how does HN keep the level of quality so high? Is it because we share an interest? Because of the quality control? The UI/UX of the site? Something else?
> how does HN keep the level of quality so high? Is it because we share an interest? Because of the quality control?
My guess is that dang's moderation is a large part of it, and I think dang moderates (creates moderation policy and executes it) based on things that are "human-assessed" and not quantifiable....
> But if you are the size of Facebook, Instagram, Tiktok, etc. it is too risky (and expensive) to let individuals define what success looks like
You're probably right, but I think it's worth challenging this conventional wisdom. (Not necessarily here, we're not going to work it out, but still, I'll ask some rhetorical questions) Why is it too dangerous? Even if you have quantitative metrics, are you _sure_ this isn't still "letting individuals define what success looks like", since individuals decided how the metrics were defined and measured? So then, what's the difference? Why, specifically, is it too "dangerous" to have non-quantitative metrics defining what success looks like? Danger of... what? I would say facebook as it is, is actually incredibly dangerous to, like, human society. So... what kind of danger are we talking about? Danger to facebook's profits instead? Or what?
> So the question should be what metric(s), if not engagement in itself, should we measure for to create a healthy online community?
Sure. I don't know! I think that's the question the OP is meaning to ask too, if not completely answer. The OP suggests:
> At the start of this, I said that people join your community for support, connection, opportunities to give back, and meaningful relationships. Those are the things they value, and those are the things you should value. And if you value them, you should measure them.
> They’re not as easy to measure as engagement, sure, but they can be measured. The best part is that maximizing these metrics is always going to be good for your community.
My point is that i'm not certain this should be short-circuited with "Well of course whatever metrics these are, they need to be quantitative and have the appareance of "objectivity".
> I wonder if it's less about metrics and more about principles; how does HN keep the level of quality so high? Is it because we share an interest? Because of the quality control? The UI/UX of the site? Something else?
HN is successful because it can be moderated by one dang. Maybe a bonus helper can appear.
But once you get so big that your dang can't do it all anymore, it will eventually fall apart.
HN in general isn't aware of operationalization[1]. It's an interesting quirk of computer science being a STEM field, but CS research is qualitatively different from physics, sociology, psychology, biology, zoology, and ag science. Rather, the typical CS student has a pre-theoretical conceptualization of quantification and data collection as it pertains to research methodology.
Instead of relying on operationaluzation, the phenomena is considered strictly immeasurable. It leaves a lot on the table. In the examples above, for instance, there are a number of unsubstantiated assumptions as well as pros and cons that implicitly do not account for operationalization and it shows.
Off-topic, but learning about classic examples of operationalization is when I realized that mostly psychology isn't a real science (yet). I believe the canonical example involved me learning the phrase "bobo doll".
OK, and? I could guess what the problem with this is, but let's spell it out... maybe that they are subjective so different assessors can disagree... and this is a problem why?
What if there are some cases where it's actually important to use human-assessed metrics as one component, or even as the entire thing, cases where no appropriate 'objective' metrics are available? (In scare quotes, because these quantitative metrics are seldom _quite_ as 'objective' or independent from human judgement as assumed. There is usually human judgement involved in how the metrics are defined and measured, where different people might define and measure a metric different ways resulting in different numbers...)