Can the SDK tell me the number of input tokens a message will consume before sending?

[danelliottster]

I want to include as much information as possible to the LLM without exceeding the input token limit. Can I tell, exactly, how many tokens an input message will be prior to sending the request?

Thank you.

[rm2631]

Yes you can using tiktoken. This will estimate the number of tokens your request will send.
Here's a function to do so:

def _num_tokens_from_string(string: str, model_name="gpt-3.5-turbo") -> int:
    """Returns the number of tokens in a text string.
    :param string: text string
    :return: number of tokens
    """
    encoding = tiktoken.encoding_for_model(model_name)
    num_tokens = len(encoding.encode(string))
    return num_tokens

Here's the doc: https://pypi.org/project/tiktoken/

Simply call this function in a while loop and reduce the length of your prompt in the loop until you fit within your token limit. I also usually allow for some room between the limit and what I send (95% of 4096 for instance)