CLIP › Key Parameters

Understanding the key parameters for CLIP deployment helps you make informed decisions about which model variant to use and how to configure inference:

• projection_dim: Dimension of the shared embedding space (512 for ViT-B/32, 768 for larger variants). This determines the size of the vectors you store for retrieval applications and the computational cost of similarity search.
• temperature: Learned scaling parameter (initialized to approximately 0.07, typically converging to an effective value around 100 after training). Controls the sharpness of the softmax distribution over similarity scores. Higher values produce more confident predictions.
• num_hidden_layers: Number of transformer layers in the encoders (12 layers for the ViT-B/32 variant). More layers capture more complex visual and linguistic patterns but increase inference time and memory requirements.
• prompt_template: Template string used to format class labels into natural language descriptions (for example, "a photo of a {}"). This significantly affects zero-shot performance and should match the style of captions in the training data.
• patch_size: For ViT variants, the size of image patches (32, 16, or 14 pixels). Smaller patches capture finer detail but dramatically increase the number of tokens and the quadratic attention cost.