test_runs.Utils

Utility functions and helpers for Test_Runs integration.

View module source on GitHub

EvaluatorNameToIdAndPassFailCriteria

@dataclass
class EvaluatorNameToIdAndPassFailCriteria()

This class represents an evaluator name to id and pass fail criteria.

get_local_evaluator_name_to_id_and_pass_fail_criteria_map

def get_local_evaluator_name_to_id_and_pass_fail_criteria_map(
    evaluators: List[Union[BaseEvaluator, str]]
) -> Dict[str, EvaluatorNameToIdAndPassFailCriteria]

This function returns a map of evaluator names to their corresponding ids and pass fail criteria.

get_evaluator_config_from_evaluator_name_and_pass_fail_criteria

def get_evaluator_config_from_evaluator_name_and_pass_fail_criteria(
        id: str, name: str, pass_fail_criteria: PassFailCriteria) -> Evaluator

This function returns an evaluator config from the evaluator name and pass fail criteria.