evals.models.rate_limiters

class AdaptiveTokenBucket(initial_per_second_request_rate: float, maximum_per_second_request_rate: float = 1000, minimum_per_second_request_rate: float = 0.1, enforcement_window_minutes: float = 1, rate_reduction_factor: float = 0.5, rate_increase_factor: float = 0.01, cooldown_seconds: float = 5)

Bases: object

An adaptive rate-limiter that adjusts the rate based on the number of rate limit errors.

This rate limiter does not need to know the exact rate limit. Instead, it starts with a high rate and reduces it whenever a rate limit error occurs. The rate is increased slowly over time if no further errors occur.

Args: initial_per_second_request_rate (float): The allowed request rate. maximum_per_second_request_rate (float): The maximum allowed request rate. enforcement_window_minutes (float): The time window over which the rate limit is enforced. rate_reduction_factor (float): Multiplier used to reduce the rate limit after an error. rate_increase_factor (float): Exponential factor increasing the rate limit over time. cooldown_seconds (float): The minimum time before allowing the rate limit to decrease again.

async async_wait_until_ready(max_wait_time: float = 10) → None

available_requests() → float

increase_rate() → None

make_request_if_ready() → None

max_tokens() → float

on_rate_limit_error(request_start_time: float, verbose: bool = False) → None

wait_until_ready(max_wait_time: float = 300) → None

exception RateLimitError

Bases: PhoenixException

class RateLimiter(rate_limit_error: Type[BaseException] | None = None, max_rate_limit_retries: int = 3, initial_per_second_request_rate: float = 1, maximum_per_second_request_rate: float = 50, enforcement_window_minutes: float = 1, rate_reduction_factor: float = 0.5, rate_increase_factor: float = 0.01, cooldown_seconds: float = 5, verbose: bool = False)

Bases: object

alimit(fn: Callable[[ParameterSpec], Coroutine[Any, Any, GenericType]]) → Callable[[ParameterSpec], Coroutine[Any, Any, GenericType]]

limit(fn: Callable[[ParameterSpec], GenericType]) → Callable[[ParameterSpec], GenericType]

exception UnavailableTokensError

Bases: PhoenixException