Skip to content

AIPromptInferenceConfiguration

Used by
Source code in docs_src/capo_qconnect/types/ai_prompt_inference_configuration.py 
13
14
15
16
17
18
19
20
21
22
23
class AIPromptInferenceConfiguration(TypedDict, closed=True):
    temperature: NotRequired["capo_qconnect.types.probability.Probability"]
    """<p>The temperature setting for controlling randomness in the generated response.</p>"""
    top_p: NotRequired["capo_qconnect.types.probability.Probability"]
    """<p>The top-P sampling parameter for nucleus sampling.</p>"""
    top_k: NotRequired["capo_qconnect.types.top_k.TopK"]
    """<p>The top-K sampling parameter for token selection.</p>"""
    max_tokens_to_sample: NotRequired[
        "capo_qconnect.types.max_tokens_to_sample.MaxTokensToSample"
    ]
    """<p>The maximum number of tokens to generate in the response.</p>"""

max_tokens_to_sample instance-attribute 

max_tokens_to_sample: NotRequired[MaxTokensToSample]

The maximum number of tokens to generate in the response.

temperature instance-attribute 

temperature: NotRequired[Probability]

The temperature setting for controlling randomness in the generated response.

top_k instance-attribute 

top_k: NotRequired[TopK]

The top-K sampling parameter for token selection.

top_p instance-attribute 

top_p: NotRequired[Probability]

The top-P sampling parameter for nucleus sampling.