Gambling can absolutely destroy lives. I've seen it in my own family. Its highly addictive nature combined with an easy way to lose everything you own is incredibly dangerous.
I've personally witnessed domestic violence from a gambler taking heavy losses as they realize they're about to be in a really bad place financially. I've never seen someone get violent over messing up a few rows of knitting.
So your beef is with domestic violence, not with gambling.
In a counterfactual where someone was doing some gambling and not doing some domestic violence, would you be still upset about the gambling in and of itself? Who does it harm?
Its not just domestic violence (though the rates of DV are usually higher with gamblers than the general population...huh I wonder why), but also just completely messing up a family's finances. This can really destroy families and relationships.
And sure, the same could be done with lots of things. But gambling is incredibly addictive.
I've watched family members gamble themselves into homelessness before. Isn't this a harm as well? Aren't the kids harmed when they lose their home and have no college savings despite the household having a net value of several million dollars a few years before?
Don't get me wrong, I'm not 100% against all forms of gambling. I gamble from time to time myself. What I'm against is this current wave of constant, relentless push of gambling into damn near everything, all the time. Back when gambling meant you'd have to go to the riverboat or fly halfway across the country or you're only dealing with small stakes poker between friends, its radically different to apps on our phones pushing notifications all the time, every sports game spending a significant amount of screen real estate to showing betting odds, news outlets talking about polymarkets all the time.
Could one have still been a problem gambler back before smartphone betting? Sure, one of those family examples I know about related to flying out to Vegas a few times a long time ago. But the rates of that happening were far lower before.
In this hypothetical scenario, is my hobby actively harmful to society?
Some people would enjoy killing people but we don’t let them do that.