mistral-ai
#10138
context_length_exceeded_error
The number of tokens in the prompt exceeds the model's maximum context length of 32768. Please use a shorter prompt.
This error has been identified and solved.
Reason
The error you are encountering, "Prompt contains 35868 tokens, too large for model with 32768 maximum context length," is due to the following reasons:
The input prompt you are sending to the Mistral AI model exceeds the model's maximum allowed context length. In this case, the model has a maximum context length of 32,768 tokens, but your prompt contains 35,868 tokens.
This limitation is inherent to the model's design and configuration, as Large Language Models (LLMs) have specific context lengths that determine how many tokens they can process in a single input sequence.
Solution
To resolve the 400 status error due to the prompt exceeding the maximum context length of the Mistral AI model, you need to adjust your input to comply with the model's limitations. Here are the steps to take:
Reduce the prompt length: Ensure that the total number of tokens in your prompt does not exceed 32,768 tokens.
Split the input: If necessary, split the input into multiple requests to stay within the token limit.
Optimize the context: Remove any unnecessary context or messages to minimize the token count.
Use a different model: If possible, consider using a model with a larger context length if your application requires handling longer inputs.
By adhering to these guidelines, you can prevent the error and ensure your API requests are processed successfully.
Suggested Links
https://cheatsheet.md/chatgpt-cheatsheet/openai-api-error-axioserror-request-failed-status-code-400
https://www.googlecloudcommunity.com/gc/AI-ML/Mistral-Large-2407-Inference-context-length-error/m-p/802826
https://community.openai.com/t/error-code-400-max-token-length/716391
https://community.openai.com/t/request-failed-with-status-code-400/39242
https://datanorth.ai/blog/context-length
https://learn.microsoft.com/en-us/answers/questions/2117664/my-mistral-large-2407-serverless-deployment-api-is
https://community.crewai.com/t/crewai-on-mistral-api-error/1559
