See our collection for all versions of SigLIP.

Run SigLIP with Keras 3: JAX, PyTorch, or TensorFlow

kerasformers/siglip_base_p16_multilingual_256

Paper: Sigmoid Loss for Language Image Pre-Training (arXiv:2303.15343) · HF Papers

SigLIP is a vision + text dual encoder trained with a pairwise sigmoid loss instead of CLIP's softmax contrastive loss. Per-pair training scales to large batches and often improves zero-shot accuracy at the same model size.

For more details on the model, please go to the upstream model card.

Pure-Keras 3 conversion of google/siglip-base-patch16-256-multilingual for kerasformers. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a zero-shot image-text checkpoint (SigLIPZeroShotClassify): pass image(s) and text prompts at inference time.

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from kerasformers.models.siglip import (
    SigLIPProcessor,
    SigLIPZeroShotClassify,
)

processor = SigLIPProcessor.from_weights("kerasformers/siglip_base_p16_multilingual_256")
model = SigLIPZeroShotClassify.from_weights("kerasformers/siglip_base_p16_multilingual_256")

labels = [
    "a photo of a cat",
    "a photo of a dog",
    "a photo of a car",
    "a photo of a living room",
]
inputs = processor(text=labels, image_paths="your_image.jpg")
output = model(
    {
        "images": inputs["images"],
        "token_ids": inputs["input_ids"],
    }
)
print(output["image_logits"].shape)

Load any SigLIP variant the same way with from_weights("kerasformers/<variant>"):

Variant	Hub
`siglip_base_p16_224`	`kerasformers/siglip_base_p16_224`
`siglip_base_p16_256`	`kerasformers/siglip_base_p16_256`
`siglip_base_p16_multilingual_256`	`kerasformers/siglip_base_p16_multilingual_256`
`siglip_base_p16_384`	`kerasformers/siglip_base_p16_384`
`siglip_base_p16_512`	`kerasformers/siglip_base_p16_512`
`siglip_large_p16_256`	`kerasformers/siglip_large_p16_256`
`siglip_large_p16_384`	`kerasformers/siglip_large_p16_384`
`siglip_so400m_p14_224`	`kerasformers/siglip_so400m_p14_224`
`siglip_so400m_p14_384`	`kerasformers/siglip_so400m_p14_384`

Tips

Set KERAS_BACKEND before importing Keras / kerasformers.
Prefer Processor.from_weights(...) so image size and tokenizer match the variant.
Map processor input_ids to model token_ids. No padding mask is required.
See SigLIP docs and Loading Weights.
Community / upstream safetensors still work via the hf: prefix, e.g. SigLIPZeroShotClassify.from_weights("hf:google/siglip-base-patch16-256-multilingual").