evals.utils

download_benchmark_dataset(task: str, dataset_name: str) → DataFrame

Downloads an Arize evals benchmark dataset as a pandas dataframe.

Parameters:

• task (str) – Task to be performed.

• dataset_name (str) – Name of the dataset.

Returns:

A pandas dataframe containing the data.

Return type:

pandas.DataFrame

get_tqdm_progress_bar_formatter(title: str) → str

Returns a progress bar formatter for use with tqdm.

Parameters:

title (str) – The title of the progress bar, displayed as a prefix.

Returns:

A formatter to be passed to the bar_format argument of tqdm.

Return type:

str

openai_function_call_kwargs(rails: List[str], provide_explanation: bool) → Dict[str, Any]

Returns keyword arguments needed to invoke an OpenAI model with function calling for classification.

Parameters:

• rails (List[str]) – The rails to snap the output to.

• provide_explanation (bool) – Whether to provide an explanation.

Returns:

A dictionary containing function call arguments.

Return type:

Dict[str, Any]

parse_openai_function_call(raw_output: str) → Tuple[str, str | None]

Parses the output of an OpenAI function call.

Parameters:

raw_output (str) – The raw output of an OpenAI function call.

Returns:

A tuple of the unrailed label and an optional explanation.

Return type:

Tuple[str, Optional[str]]

printif(condition: bool, *args: Any, **kwargs: Any) → None

snap_to_rail(raw_string: str | None, rails: List[str], verbose: bool = False) → str

Snaps a string to the nearest rail, or returns None if the string cannot be snapped to a rail.

Parameters:

• raw_string (str) – An input to be snapped to a rail.

• rails (List[str]) – The target set of strings to snap to.

Returns:

A string from the rails argument or “UNPARSABLE” if the input string could not be snapped.

Return type:

str