Just incase you didn’t know you can append ? to any query and get a quick answer straight away
reply
https://www.siquick.com/blog/model-quantization-fine-tuning-...
However much they try to make us think otherwise, at this point in time there’s not really any “good guys” in the AI race.
Cold boot times are around 5m but if your usage periods are predictable it can work out ok. Works out at $2 an hour.
Still far more expensive than a ChatGPT sub.
https://modal.com/docs/examples/vllm_inference
or give this a go
https://modal.com/docs/examples/opencode_server
You get $30 free credits each month on Modal which is enough to play around (i have no affiliation, just think they run a great service)
Referencing them in AGENTS/CLAUDE.md has increased their usage for me.
I tried to signup from the nav bar but kept getting a 500 error from the sign_up.json endpoint. I had to go through the Donate flow to be able to create an account.
Just incase you didn’t know you can append ? to any query and get a quick answer straight away
reply