Curious what your use case is that API tokens are actually significant expense at this time. I get it if you’re reselling a product but myself: abusively and naively messing around, I don’t see major prices as an end user. I sometimes hit the rate limit in ChatGPT, maybe like twice a week, but otherwise my monthly API bill in the last 6 months usually is in the dollars frame. Not tens or thousands of dollars, just dollars
Back when GPT4 came out mid last year my bills were slightly more dramatic (i am what you would probably call a semi heavy individual user) but they never surpassed $200 USD for a single month.
GPT4 is very cheap when you're hitting the api with common knowledge questions or reasoning about known things, and asking for short replies. It's very expensive when you're loading up the context with a ton of data and asking it to convert some data points into a narrative structure.
