I suppose that depends on what you define as "city center". Just checking Ikejiriohashi (arguably walking distance from Shibuya) I see plenty of 2LDKs for $2k-$3k a month. If you want inside the Yamanote line there's tons in East Shinjuku and north and east of that area that are "cheap".
And of course like you mentioned, going out of the city it gets even cheaper.
Other lines can be even cheaper, if you're looking for a minimal studio you can live within 10 mins of a major central station for < $600/mo especially if you choose less popular lines like Seibu Shinjuku/Ikebukuro line. $2k-$3k is already living quite nicely for many Japanese people, my mother in law lives in a 3 bedroom apartment for ~$1,000/mo in exchange for being ~20 mins away from Ikebukuro by train in a sleeper suburb.
Definitely, A friend had a place on the Seibu Ikebukuro line, 20 mins from Ikebukuro, 3 bed, 2 bath, 3 story duplex for $1k. Similarly 20 mins from Ikebukuro on Saikyo line 3 bed, 1 bath apartment for $1k. In both cases though it was 20 mins by express.
I wouldn't consider those close to downtown though. If you're into night life downtown and you want to go home at 3am it will cost $100 to get home (though I guess with the money you saved on rent you can do that every weekend and you're still ahead! :P )
And of course like you mentioned, going out of the city it gets even cheaper.