Skip to content

SampleTokenUsage

Source code in docs_src/capo_sagemakertrainingsessionruntime/types/sample_token_usage.py 
 6
 7
 8
 9
10
class SampleTokenUsage(TypedDict, closed=True):
    prefill: "int"
    """<p>The number of tokens in the input prompt (prefill phase).</p>"""
    decode: "int"
    """<p>The number of tokens generated (decode phase).</p>"""

decode instance-attribute 

decode: int

The number of tokens generated (decode phase).

prefill instance-attribute 

prefill: int

The number of tokens in the input prompt (prefill phase).