Skip to content

CustomMetricDefinition

Source code in docs_src/capo_bedrock/types/custom_metric_definition.py 
15
16
17
18
19
20
21
22
23
class CustomMetricDefinition(TypedDict, closed=True):
    name: "capo_bedrock.types.metric_name.MetricName"
    """<p>The name for a custom metric. Names must be unique in your Amazon Web Services region.</p>"""
    instructions: (
        "capo_bedrock.types.custom_metric_instructions.CustomMetricInstructions"
    )
    """<p>The prompt for a custom metric that instructs the evaluator model how to rate the model or RAG source under evaluation.</p>"""
    rating_scale: NotRequired["capo_bedrock.types.rating_scale.RatingScale"]
    """<p>Defines the rating scale to be used for a custom metric. We recommend that you always define a ratings scale when creating a custom metric. If you don't define a scale, Amazon Bedrock won't be able to visually display the results of the evaluation in the console or calculate average values of numerical scores. For more information on specifying a rating scale, see <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/model-evaluation-custom-metrics-prompt-formats.html#model-evaluation-custom-metrics-prompt-formats-schema">Specifying an output schema (rating scale)</a>.</p>"""

instructions instance-attribute 

instructions: CustomMetricInstructions

The prompt for a custom metric that instructs the evaluator model how to rate the model or RAG source under evaluation.

name instance-attribute 

name: MetricName

The name for a custom metric. Names must be unique in your Amazon Web Services region.

rating_scale instance-attribute 

rating_scale: NotRequired[RatingScale]

Defines the rating scale to be used for a custom metric. We recommend that you always define a ratings scale when creating a custom metric. If you don't define a scale, Amazon Bedrock won't be able to visually display the results of the evaluation in the console or calculate average values of numerical scores. For more information on specifying a rating scale, see Specifying an output schema (rating scale).