evals.models.anthropic#

class AnthropicModel(default_concurrency=20, _verbose=False, _rate_limiter=<factory>, model='claude-2.1', temperature=0.0, max_tokens=256, top_p=1, top_k=256, stop_sequences=<factory>, extra_parameters=<factory>, max_content_size=None, initial_rate_limit=5)#

Bases: BaseModel

An interface for using Anthropic models.

This class wraps the Anthropic SDK library for use with Phoenix LLM evaluations. Calls to the Anthropic API are dynamically throttled when encountering rate limit errors. Requires the anthropic package to be installed.

Supports Async: ✅

If possible, makes LLM calls concurrently.

Parameters:

  • model (str, optional) – The model name to use. Defaults to “claude-2.1”.

  • temperature (float, optional) – Sampling temperature to use. Defaults to 0.0.

  • max_tokens (int, optional) – Maximum number of tokens to generate in the completion. Defaults to 256.

  • top_p (float, optional) – Total probability mass of tokens to consider at each step. Defaults to 1.

  • top_k (int, optional) – The cutoff where the model no longer selects the words. Defaults to 256.

  • stop_sequences (List[str], optional) – If the model encounters a stop sequence, it stops generating further tokens. Defaults to an empty list.

  • extra_parameters (Dict[str, Any], optional) – Any extra parameters to add to the request body (e.g., countPenalty for a21 models). Defaults to an empty dictionary.

  • max_content_size (Optional[int], optional) – If using a fine-tuned model, set this to the maximum content size. Defaults to None.

  • initial_rate_limit (int, optional) – The initial internal rate limit in allowed requests per second for making LLM calls. This limit adjusts dynamically based on rate limit errors. Defaults to 5.

Example

# Set the ANTHROPIC_API_KEY environment variable

from phoenix.evals import AnthropicModel
model = AnthropicModel(model="claude-2.1")