Why is there a 1K comment limit and can you add a way to override that? Just hitting the reddit API looks like it will fetch further back.
I looked at my 1000th comment in the archive and it's only back to Sat May 28 2022 (I guess I comment a lot), if I want to save 10+ years I'll need a higher limit.
Edit: Looks like this might a reddit limit? I'm seeing other tools mention 1K. I guess I'll use Reddit's data export tool instead.
Yep, Reddit's paging API only gives you the most recent 1k each of comments and posts.
I'd suggest starting by fetching recent posts, then download your GDPR archive and run it through the tool as well (the `archive` command). Then it'll be as if you fetched everything from the web API (best of both worlds).
I looked at my 1000th comment in the archive and it's only back to Sat May 28 2022 (I guess I comment a lot), if I want to save 10+ years I'll need a higher limit.
Edit: Looks like this might a reddit limit? I'm seeing other tools mention 1K. I guess I'll use Reddit's data export tool instead.