ELI5 What is a token? Is it a word, question, letter, what? If I ask a question ...

andrewmunsell · on March 1, 2023

Roughly speaking, words or word parts. Open AI has an explainer:

You can also check your input using their tokenizer: https://platform.openai.com/tokenizer

So, your example is ~9 tokens

chirau · on March 1, 2023

So does the response count towards the token count?

andrewmunsell · on March 2, 2023

It does, yeah. It's tokens "processed", meaning request & response. The API response will give you a total token count so you can keep track

teaearlgraycold · on March 1, 2023

Token cost is prompt + response. In the case of ChatGPT you don't know the full prompt, but they don't charge by tokens for that app.

In the API you need to tokenize your input and tokenize the output then add the counts together.