---
license: other
license_name: nvidia-license
license_link: https://huggingface.co/nvidia/LocateAnything-3B
base_model: nvidia/LocateAnything-3B
tags:
- locate-anything.cpp
- ggml
- gguf
- object-detection
- open-vocabulary-detection
- visual-grounding
- localai
pipeline_tag: object-detection
library_name: gguf
---

# locate-anything.cpp — GGUF

GGUF builds of [`nvidia/LocateAnything-3B`](https://huggingface.co/nvidia/LocateAnything-3B)
for **[locate-anything.cpp](https://github.com/mudler/locate-anything.cpp)** — a C++/ggml
inference engine for open-vocabulary detection / visual grounding, no Python at inference time.

**Brought to you by the [LocalAI](https://github.com/mudler/LocalAI) team.**

The detections are **identical to the official PyTorch implementation** (the engine is
parity-gated against it), and run faster on CPU. See the
[benchmarks](https://github.com/mudler/locate-anything.cpp/blob/master/benchmarks/BENCHMARK.md).

## Files

| File | Bits (LM) | Size | Notes |
| ---- | --------- | ---- | ----- |
| `locate-anything-f16.gguf`  | f16  | ~9.2 GB | LM matmuls in f16, everything else f32 |
| `locate-anything-q8_0.gguf` | q8_0 | ~6.3 GB | near-lossless; **box-identical** to f32 |
| `locate-anything-q6_k.gguf` | q6_k | ~5.5 GB | |
| `locate-anything-q5_k.gguf` | q5_k | ~5.1 GB | |
| `locate-anything-q4_k.gguf` | q4_k | ~4.7 GB | smallest; sub-pixel box drift |

The full-precision `f32` GGUF (~15 GB) is reproducible from the HF weights with
`scripts/convert_locateanything_to_gguf.py` in the repo.

## Quantization policy

Only the Qwen2 language-model matmuls (`attn_{q,k,v,o}`, `ffn_{gate,up,down}`, `lm.output`)
are quantized. The MoonViT vision tower, the projector, all norms and biases, and the two
host-read f32 tensors (`lm.tok_embd`, `vit.pos_emb`) stay **f32** — so the parity-sensitive
vision path is untouched. `q8_0` is byte-identical on box output; lower bit-widths trade a
little box precision for size.

## Usage

```sh
# build the CLI (see the repo README), then:
locate-anything-cli detect \
    --model locate-anything-q8_0.gguf \
    --input image.jpg \
    --prompt "Locate all the instances that matches the following description: person</c>car." \
    --annotated out.png
# -> {"detections":[{"label":"person","box":[...]}, ...]}  + an annotated PNG
```

Decode modes: `--mode hybrid` (default), `slow`, `fast`. Separate categories in the prompt
with `</c>`.

## License

The model weights are NVIDIA's, distributed under
[NVIDIA's license](https://huggingface.co/nvidia/LocateAnything-3B); this repository
redistributes them in GGUF form for use with locate-anything.cpp (MIT).