I'm not quite sure what you mean by the PDFs and EPUBs being rasterized - that's always happening with anything that is displayed on a screen. Do you mean that it is converted to a raster format and stored like that? (outside of a frame buffer that is)
Yes, there seems to be a cache somewhere. They have a nice trick: when you flip pages, a lower-resolution cached image renders and you can immediately start reading/writing, before being (almost immediately) replaced by the actual-resolution one. (See about 15:15 to 20:00 in this video: https://youtu.be/YWLJPyTrHnM?t=915)