Skip to content

AutomatedEvaluationConfig

Source code in docs_src/capo_bedrock/types/automated_evaluation_config.py 
15
16
17
18
19
20
21
22
23
24
25
class AutomatedEvaluationConfig(TypedDict, closed=True):
    dataset_metric_configs: "capo_bedrock.types.evaluation_dataset_metric_configs.EvaluationDatasetMetricConfigs"
    """<p>Configuration details of the prompt datasets and metrics you want to use for your evaluation job.</p>"""
    evaluator_model_config: NotRequired[
        "capo_bedrock.types.evaluator_model_config.EvaluatorModelConfig"
    ]
    """<p>Contains the evaluator model configuration details. <code>EvaluatorModelConfig</code> is required for evaluation jobs that use a knowledge base or in model evaluation job that use a model as judge. This model computes all evaluation related metrics.</p>"""
    custom_metric_config: NotRequired[
        "capo_bedrock.types.automated_evaluation_custom_metric_config.AutomatedEvaluationCustomMetricConfig"
    ]
    """<p>Defines the configuration of custom metrics to be used in an evaluation job.</p>"""

custom_metric_config instance-attribute 

custom_metric_config: NotRequired[AutomatedEvaluationCustomMetricConfig]

Defines the configuration of custom metrics to be used in an evaluation job.

dataset_metric_configs instance-attribute 

dataset_metric_configs: EvaluationDatasetMetricConfigs

Configuration details of the prompt datasets and metrics you want to use for your evaluation job.

evaluator_model_config instance-attribute 

evaluator_model_config: NotRequired[EvaluatorModelConfig]

Contains the evaluator model configuration details. EvaluatorModelConfig is required for evaluation jobs that use a knowledge base or in model evaluation job that use a model as judge. This model computes all evaluation related metrics.