MERaLiON-3-10B-MLX

MLX port of MERaLiON/MERaLiON-3-10B-preview for native inference on Apple Silicon. 8-bit group quantization (group_size=64), no KV-cache compression applied.

MERaLiON-3 is a multimodal audio-language model from I2R, A*STAR (Singapore), built on a Gemma-2 decoder backbone. It targets speech-to-text, speech translation, and audio understanding across English, Mandarin, Malay, Tamil, Indonesian, and other Southeast Asian languages, with particular strength on Singlish and code-switched speech.

What's in this repo

Component	Format	Notes
Decoder (Gemma-2 backbone)	8-bit MLX (group_size=64, affine)	~10 GB
Encoder (Whisper-large-v3)	float16 (unquantized)	~1.2 GB
Speech-text adaptor	float16 (unquantized)	~0.4 GB
Tokenizer	Unchanged from upstream

Approximate total size: ~11.6 GB.

Quantized from the original full-precision MERaLiON-3-10B-preview weights using mlx-lm's convert pipeline. Tokenizer, generation config, and chat template are inherited unchanged from upstream.

Quickstart

Apple Silicon, audio input

For the full multimodal pipeline (Whisper encoder + adaptor + decoder), install the MERaLiON MLX runtime:

pip install mlx-meralion

from mlx_meralion import load_model, transcribe

model = load_model("majentik/MERaLiON-3-10B-MLX")

# ASR
text = transcribe(model, "audio.wav")
print(text)

# Spoken QA
answer = transcribe(model, "audio.wav", task="sqa",
                    question="What is the speaker describing?")

Decoder-only (text generation)

The quantized Gemma-2 decoder can be loaded standalone with mlx-lm:

from mlx_lm import load, generate

model, tokenizer = load("majentik/MERaLiON-3-10B-MLX/decoder")
out = generate(model, tokenizer, prompt="Hello", max_tokens=128)
print(out)

What is not in this repo

• Training data. This is a quantization-only release. Training data, safety alignment, and evaluation infrastructure are upstream's concern.
• Benchmarks. We haven't re-run AudioBench on the MLX port. Expect results within noise of the upstream model for 8-bit; lower bit-widths (see sibling repos) will drift.
• Fine-tuning scripts. Use upstream tooling with the full-precision weights.

Known limitations

• Inherits upstream's out-of-scope warning: not intended for tool-calling, math, or coding tasks.
• 8-bit MLX quantization is conservative; quality drift vs. FP16 upstream should be minimal but has not been formally measured.
• Long-audio chunking (>30s) is handled by mlx-meralion; very long inputs may still show boundary artifacts.

Hardware requirements

• Apple Silicon (M1/M2/M3/M4), macOS
• ~16 GB unified memory recommended for the 8-bit variant
• For lower-memory machines, see MERaLiON-3-10B-MLX-4bit (~5 GB)

License

Released under the MERaLiON-3 Public Licence, inherited from the upstream model. See the license PDF.

Links

• Upstream: MERaLiON/MERaLiON-3-10B-preview
• Sibling: majentik/MERaLiON-3-10B-MLX-4bit
• Garden hub: majentik/garden
• MLX framework: ml-explore/mlx
• Runtime: mlx-audiollm