evals.generate

llm_generate(dataframe: DataFrame, template: PromptTemplate | str, model: BaseModel, system_instruction: str | None = None, verbose: bool = False, output_parser: Callable[[str, int], Dict[str, Any]] | None = None, include_prompt: bool = False, include_response: bool = False, run_sync: bool = False, concurrency: int | None = None) → DataFrame

Generates a text using a template using an LLM. This function is useful if you want to generate synthetic data, such as irrelevant responses :param dataframe: A pandas dataframe in which each row :type dataframe: pandas.DataFrame :param represents a record to be used as in input to the template. All: :param template variable names must appear as column names in the dataframe: :param (extra columns unrelated to the template are permitted).: :param template: The prompt template as either an :type template: Union[PromptTemplate, str] :param instance of PromptTemplate or a string. If the latter: :param the variable: :param names should be surrounded by curly braces so that a call to .format: :param can be made to substitute variable values.: :param model: An LLM model class. :type model: BaseEvalModel :param system_instruction: An optional system :type system_instruction: Optional[str], optional :param message.: :param verbose: If True, prints detailed information to stdout such as model :type verbose: bool, optional :param invocation parameters and retry info. Default False.: :param output_parser: An optional function :type output_parser: Callable[[str, int], Dict[str, Any]], optional :param that takes each generated response and response index and parses it to a dictionary. The: :param keys of the dictionary should correspond to the column names of the output dataframe. If: :param None: :param the output dataframe will have a single column named “output”. Default None.: :param include_prompt: If True, includes a column named prompt in the :type include_prompt: bool, default=False :param output dataframe containing the prompt used for each generation.: :param include_response: If True, includes a column named response in the :type include_response: bool, default=False :param output dataframe containing the raw response from the LLM prior to applying the output: :param parser.: :param run_sync: If True, forces synchronous request submission. Otherwise :type run_sync: bool, default=False :param evaluations will be run asynchronously if possible.: :param concurrency: The number of concurrent evals if async :type concurrency: Optional[int], default=None :param submission is possible. If not provided: :param a recommended default concurrency is set on a: :param per-model basis.:

Returns:

A dataframe where each row represents the generated output

Return type:

generations_dataframe (pandas.DataFrame)