I don't doubt your experience, but I wonder what the issue was. The only LDAP-related issue I find (cve-2021-40537) also requires compromised admin-credentials to exploit. Of course, with compromised admin credentials, all bets are off anyway.
I know it is only anecdotal, but I have been running OwnCloud for many years now, available without a VPN, with no security problems.
Do not allow access to owncloud without a vpn!!