A team deploying a shared LLM endpoint wants to prevent any single user from consuming all endpoint capacity. Which Unity AI Gateway feature enables per-user consumption limits?

Question

Accepted Answer

C. Rate limits, configured with a "User (Default)" QPM or TPM value that applies to all users of the endpoint.. Unity AI Gateway rate limits allow specifying consumption limits at endpoint level, per-user default level, and custom level for specific users or groups. Setting a "User (Default)" QPM (queries per minute) or TPM (tokens per minute) value applies a default per-user rate limit to all users, preventing any single user from monopolizing endpoint capacity.

A team deploying a shared LLM endpoint wants to prevent any single user from consuming all endpoint capacity. Which Unity AI Gateway feature enables per-user consumption limits?

Related Questions

Discussion