I would settle for even speech to text that stays local. Agree, an AI assistant that doesn't enrich google or whoever scoops my queries would be even better.
OpenAI released the weights for their Whisper model, so you can run high quality speech-to-text, locally, at better than realtime if you have a decent GPU.
I’m having a hard time understanding what this article is trying to convey. I read this as a low quality plug plug for the idea of “pods,” and it’s using the terms ChatGPT and AI to make headlines.
I understand it this way that ChatGPT (or Alexa) will run locally and you don't have to send all of your queries to some company for processing. Of course, hardware costs will have to come way down. Or some kind of zero-trust scheme will be developed to send data for processing anonymously, possibly along with some satoshis to make the whole thing economical for computation providers.
> Sure you can, if you can self host 8x A100 cards ($200k+), plus the cost of electricity and heat dissipation.
Well, you don’t need it on all the time, so if you don’t mind having the compute & model in the cloud (and since “data pods” seem to be a concept that TBL is pitching that would likely be implemented in the cloud, just with sharing being granular and white-listed, you probably don’t if you buy into that), you could just spin up an EC2 p4d.24xlarge instance for a little under $0.01/s when needed.
