Skip to content

PromptModelInferenceConfiguration

Source code in docs_src/capo_bedrock_agent/types/prompt_model_inference_configuration.py 
14
15
16
17
18
19
20
21
22
class PromptModelInferenceConfiguration(TypedDict, closed=True):
    temperature: NotRequired["capo_bedrock_agent.types.temperature.Temperature"]
    """<p>Controls the randomness of the response. Choose a lower value for more predictable outputs and a higher value for more surprising outputs.</p>"""
    top_p: NotRequired["capo_bedrock_agent.types.top_p.TopP"]
    """<p>The percentage of most-likely candidates that the model considers for the next token.</p>"""
    max_tokens: NotRequired["capo_bedrock_agent.types.maximum_length.MaximumLength"]
    """<p>The maximum number of tokens to return in the response.</p>"""
    stop_sequences: NotRequired["capo_bedrock_agent.types.stop_sequences.StopSequences"]
    """<p>A list of strings that define sequences after which the model will stop generating.</p>"""

max_tokens instance-attribute 

max_tokens: NotRequired[MaximumLength]

The maximum number of tokens to return in the response.

stop_sequences instance-attribute 

stop_sequences: NotRequired[StopSequences]

A list of strings that define sequences after which the model will stop generating.

temperature instance-attribute 

temperature: NotRequired[Temperature]

Controls the randomness of the response. Choose a lower value for more predictable outputs and a higher value for more surprising outputs.

top_p instance-attribute 

top_p: NotRequired[TopP]

The percentage of most-likely candidates that the model considers for the next token.