Instruction Format › Key Parameters

The key parameters for chat templating and tokenization are:

• messages: List of dictionaries containing the conversation history (roles and content). Each dictionary must have role and content keys.

• tokenize: If True, returns token IDs; if False, returns a formatted string. Use False for debugging and inspecting the template output; use True when preparing inputs for model forward passes.

• add_generation_prompt: If True, adds tokens to signal the start of an assistant response. Set to True at inference time and False when the complete assistant response is already included in the messages.

• return_tensors: Specifies the format for returned data (e.g., 'pt' for PyTorch tensors). Useful when you want the tokenizer to return tensors directly rather than lists of integers.

• special_tokens: A dictionary of tokens to add to the tokenizer's vocabulary. Use this to add domain-specific structural tokens when designing a custom format.

• truncation and max_length: Control whether and how the template output is truncated to fit within a maximum sequence length. Essential for batch training where all sequences must be padded to the same length.