And your friends are still using apps that harvest YOUR contact data. So your contact graph is still determinable. When I changed my phone I decided to not give out my new number. Now all of the spam calls come in on my Google Voice number that I share with everyone and I only share my iMessage with specific people I trust.