What you are referring to is called "white flight" and the causes were more complex than crime or immigration.

The casual link between immigration and crime is generally agreed to be false. So whatever you read that claimed immigration caused crime to go up seems incorrect.