cohere
#10147
input_limit_exceeded_error
Too many tokens: the total number of tokens in the prompt exceeds the limit of 4081. Try using a shorter prompt or enable prompt truncating. See https://docs.cohere.com/reference/generate for more details.
This error has been identified and solved.
Reason
The 400 status error you are encountering is due to the prompt exceeding the maximum allowed token limit for the Cohere API. Specifically, the error message indicates that the total number of tokens in the prompt (4796) cannot exceed 4081 tokens, which is the maximum limit set by the API.
Solution
To fix the "too many tokens" error in the Cohere API, you need to ensure that the total number of tokens in your prompt does not exceed the maximum limit of 4081 tokens. Here are some concise solutions:
Shorten the prompt: Reduce the length of your prompt to stay within the token limit.
Enable prompt truncation: Use the API's built-in prompt truncation feature to automatically truncate the prompt to fit within the limit.
Split the prompt: If possible, split your request into multiple smaller prompts to avoid exceeding the token limit in a single request.
Adjust the model parameters: Consider adjusting parameters such as
max_tokensor other relevant settings to optimize the prompt and response within the allowed limits.
Suggested Links
https://github.com/langchain-ai/langchainjs/issues/3894
https://docs.cohere.com/v2/docs/rate-limits
https://docs.cohere.com/v2/reference/errors
https://github.com/cohere-ai/cohere-go
https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-cohere-command-r-plus.html
https://maartengr.github.io/BERTopic/api/representation/cohere.html
https://github.com/run-llama/llama_index/issues/12633
https://github.com/orgs/community/discussions/138672
https://github.com/langchain-ai/langchain-cohere/blob/main/libs%2Fcohere%2Flangchain_cohere%2Fchat_models.py
