Skip to content

EvaluateRequest

Source code in docs_src/capo_bedrock_agentcore/types/evaluate_request.py 
16
17
18
19
20
21
22
23
24
25
26
27
28
class EvaluateRequest(TypedDict, closed=True):
    evaluator_id: "capo_bedrock_agentcore.types.evaluator_id.EvaluatorId"
    """<p> The unique identifier of the evaluator to use for scoring. Can be a built-in evaluator (e.g., <code>Builtin.Helpfulness</code>, <code>Builtin.Correctness</code>) or a custom evaluator Id created through the control plane API. </p>"""
    evaluation_input: "capo_bedrock_agentcore.types.evaluation_input.EvaluationInput"
    """<p> The input data containing agent session spans to be evaluated. Includes a list of spans in OpenTelemetry format from supported frameworks like Strands (AgentCore Runtime) or LangGraph with OpenInference instrumentation. </p>"""
    evaluation_target: NotRequired[
        "capo_bedrock_agentcore.types.evaluation_target.EvaluationTarget"
    ]
    """<p> The specific trace or span IDs to evaluate within the provided input. Allows targeting evaluation at different levels: individual tool calls, single request-response interactions (traces), or entire conversation sessions. </p>"""
    evaluation_reference_inputs: NotRequired[
        "capo_bedrock_agentcore.types.evaluation_reference_inputs.EvaluationReferenceInputs"
    ]
    """<p> Ground truth data to compare against agent responses during evaluation. Allows to provide expected responses, assertions, and expected tool trajectories at different evaluation levels. Session-level reference inputs apply to the entire conversation, while trace-level reference inputs target specific request-response interactions identified by trace ID. </p>"""

evaluation_input instance-attribute 

evaluation_input: EvaluationInput

The input data containing agent session spans to be evaluated. Includes a list of spans in OpenTelemetry format from supported frameworks like Strands (AgentCore Runtime) or LangGraph with OpenInference instrumentation.

evaluation_reference_inputs instance-attribute 

evaluation_reference_inputs: NotRequired[EvaluationReferenceInputs]

Ground truth data to compare against agent responses during evaluation. Allows to provide expected responses, assertions, and expected tool trajectories at different evaluation levels. Session-level reference inputs apply to the entire conversation, while trace-level reference inputs target specific request-response interactions identified by trace ID.

evaluation_target instance-attribute 

evaluation_target: NotRequired[EvaluationTarget]

The specific trace or span IDs to evaluate within the provided input. Allows targeting evaluation at different levels: individual tool calls, single request-response interactions (traces), or entire conversation sessions.

evaluator_id instance-attribute 

evaluator_id: EvaluatorId

The unique identifier of the evaluator to use for scoring. Can be a built-in evaluator (e.g., Builtin.Helpfulness, Builtin.Correctness) or a custom evaluator Id created through the control plane API.