evals.models.vertex#

class GeminiModel(default_concurrency=5, _verbose=False, _rate_limiter=<factory>, project=None, location=None, credentials=None, model='gemini-pro', temperature=0.0, max_tokens=256, top_p=1, top_k=32, stop_sequences=<factory>, initial_rate_limit=5)#

Bases: BaseModel

An interface for using Google’s Gemini models.

This class wraps the Google’s VertexAI SDK library for using the Gemini models for Phoenix LLM evaluations. Calls to the the Gemini models dynamically throttled when encountering rate limit errors. Requires the vertexai package to be installed.

Supports Async: ✅

If possible, makes LLM calls concurrently.

Parameters:

  • model (str, optional) – The model name to use. Defaults to “gemini-pro”.

  • temperature (float, optional) – Sampling temperature to use. Defaults to 0.0.

  • max_tokens (int, optional) – Maximum number of tokens to generate in the completion. Defaults to 256.

  • top_p (float, optional) – Total probability mass of tokens to consider at each step. Defaults to 1.

  • top_k (int, optional) – The cutoff where the model no longer selects the words. Defaults to 32.

  • stop_sequences (List[str], optional) – If the model encounters a stop sequence, it stops generating further tokens. Defaults to an empty list.

  • project (str, optional) – The default project to use when making API calls. Defaults to None.

  • location (str, optional) – The default location to use when making API calls. If not set defaults to us-central-1. Defaults to None.

  • credentials (Optional[Credentials], optional) – The credentials to use when making API calls. Defaults to None.

  • initial_rate_limit (int, optional) – The initial internal rate limit in allowed requests per second for making LLM calls. This limit adjusts dynamically based on rate limit errors. Defaults to 5.

Example

# Set up your environment
# https://cloud.google.com/vertex-ai/generative-ai/docs/start/quickstarts/quickstart-multimodal#local-shell

from phoenix.evals import GeminiModel
# if necessary, use the "project" kwarg to specify the project_id to use
# project_id = "your-project-id"
model = GeminiModel(model="gemini-pro", project=project_id)