See our collection for all versions of RF-DETR.

Run RF-DETR with Keras 3: JAX, PyTorch, or TensorFlow

kerasformers/rfdetr-seg-preview

Paper: RF-DETR: Neural Architecture Search for Real-Time Detection Transformers (arXiv:2511.09554) · HF Papers

RF-DETR is Roboflow's real-time DETR, built on a windowed DINOv2 backbone with a lightweight deformable decoder. Configurations came out of a neural architecture search, so variants differ in resolution, patch size, window count, and decoder depth. Instance-segmentation checkpoints add a mask head.

For more details on the model, please go to Roboflow's original model card.

Pure-Keras 3 conversion of Roboflow/rf-detr-seg-preview for kerasformers. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is an instance segmentation checkpoint (RFDETRInstanceSegment): each query predicts a class, box, and mask.

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from kerasformers.models.rf_detr import (
    RFDETRInstanceSegment,
    RFDETRImageProcessor,
)

model = RFDETRInstanceSegment.from_weights("kerasformers/rfdetr-seg-preview")
processor = RFDETRImageProcessor.from_weights("kerasformers/rfdetr-seg-preview")

image = Image.open("your_image.jpg").convert("RGB")
output = model(processor(image)["pixel_values"], training=False)
result = processor.post_process_instance_segmentation(
    output, threshold=0.5, target_sizes=[(image.height, image.width)]
)[0]
print(result["label_names"], result["scores"])

Load any RF-DETR variant the same way with from_weights("kerasformers/<variant>") (use RFDETRInstanceSegment for this repo):

Variant	Hub	Task
`rfdetr-nano`	`kerasformers/rfdetr-nano`	object detection
`rfdetr-small`	`kerasformers/rfdetr-small`	object detection
`rfdetr-medium`	`kerasformers/rfdetr-medium`	object detection
`rfdetr-base`	`kerasformers/rfdetr-base`	object detection
`rfdetr-large`	`kerasformers/rfdetr-large`	object detection
`rfdetr-seg-preview`	`kerasformers/rfdetr-seg-preview`	instance segmentation
`rfdetr-seg-nano`	`kerasformers/rfdetr-seg-nano`	instance segmentation
`rfdetr-seg-small`	`kerasformers/rfdetr-seg-small`	instance segmentation
`rfdetr-seg-medium`	`kerasformers/rfdetr-seg-medium`	instance segmentation
`rfdetr-seg-large`	`kerasformers/rfdetr-seg-large`	instance segmentation
`rfdetr-seg-xlarge`	`kerasformers/rfdetr-seg-xlarge`	instance segmentation
`rfdetr-seg-xxlarge`	`kerasformers/rfdetr-seg-xxlarge`	instance segmentation

Tips

Set KERAS_BACKEND before importing Keras / kerasformers.
Prefer RFDETRImageProcessor.from_weights(...) so the processor resolution matches the variant (bare constructor defaults to base's 560).
Detection: RFDETRDetect + post_process_object_detection.
Segmentation: RFDETRInstanceSegment + post_process_instance_segmentation.
See RF-DETR docs and Loading Weights.
Community / upstream safetensors still work via the hf: prefix, e.g. RFDETRInstanceSegment.from_weights("hf:Roboflow/rf-detr-seg-preview").