An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
Marzena Karpinska Greetings & Welcome to Microsoft Q&A forum!
(1) Is the account still being charged for the prompt?
The account will not be charged for the prompt if the request times out and no generation occurs. Azure OpenAI Service only charges for successful API requests that generate a response.
(2) Does this count to the tokens rate limit? (If so, does it count only the tokens in the prompt or prompt + max tok)
If the request times out and no generation occurs, it will not count towards the token rate limit. The token rate limit is based on the number of successful API requests that generate a response, not on the number of requests that are made. However, if the request includes a prompt and a maximum number of tokens, both the prompt and the maximum number of tokens will be counted towards the token rate limit if the request is successful and generates a response.
Please see Monitoring Azure OpenAI Service on available metrics and options to enable logging.
Also, See How to request increases to the default quotas and limits
If you are seeing these errors consistently then you probably could also apply for a quota increase request through this form.
Do let us know if that helps or have any further queries.