---
license: cc-by-nc-4.0
tags:
  - diffusion
  - lllite
  - controlnet
  - temporal-consistency
  - video-stylization
  - dreamlite
  - streaming
  - edit-diffusion
base_model: ByteVisionLab/DreamLite-mobile
pipeline_tag: image-to-image
library_name: diffusers
---

# Temporal LLLite v3 (DreamLite-mobile)

This repository hosts the **Temporal LLLite v3** adapter weights for
[DreamLite-mobile](https://huggingface.co/ByteVisionLab/DreamLite),
trained for streaming video stylization at video-rate throughput on a
single consumer GPU. The adapter is the artefact described in:

> *Video-Rate Streaming Stylization on a Vision-Aware MLLM-Conditioned
> Edit Diffusion: Asymmetric Batched Inference on a Distilled UNet +
> MLLM Text Encoder.* Yoshiyuki Ootani, 2026 (preprint).

The accompanying inference code, evaluation harness, training scripts,
and Zenodo-archived code release are at
[github.com/otanl/dreamlite-stream](https://github.com/otanl/dreamlite-stream).

## What this is

A ControlNet-LLLite-style attention adapter
([kohya-ss/sd-scripts](https://github.com/kohya-ss/sd-scripts)) attached
to DreamLite-mobile's 0.39 B distilled edit U-Net. The conditioning
input is the warped previous decoded frame (Farnebäck flow on the
previous output), and the adapter learns a temporal-consistency
residual that reduces inter-frame flicker on streaming video
stylization.

In the paper, this adapter pairs with three engineering mechanisms
(asymmetric side-stream / main-stream CUDA pipelining, a
compile-friendly LLLite reformulation, and a periodic
conditioning-refresh schedule) to reach sustained video-rate streaming
throughput:

| GPU | sustained fps (B=8) | e2e p50 latency |
|---|---|---|
| RTX 3090 Ti | 27.4 fps | 0.51 s |
| RTX 4090 | 54.9 fps | — (DAVIS-10 measurement) |
| RTX 5090 | 74.1 fps | — (DAVIS-10 measurement) |

All numbers at 512×512 with the v3 adapter active. End-to-end p50
latency is reported only on RTX 3090 Ti where the 480-frame sustained
test was run.

## Files

- `temporal_lllite_step001440.safetensors` (51 MB)
  - Trained for 12 epochs on 10 DAVIS-2017 sequences × 50 frames,
    AdamW8bit, post-hoc α=0.85 blended teacher target.
  - SHA-256: `88082c6bf56770469ad4ecbbca467b315ffcf4b5287fd17733751e2952fee7fc`

## Usage (sketch)

```python
from safetensors.torch import load_file

# 1. Load DreamLite-mobile via the upstream project
#    (https://github.com/ByteVisionLab/DreamLite — access via their
#    release-request process).
# 2. Apply the LLLite adapter port:
from dreamlite_lllite import apply_lllite
apply_lllite(
    unet,
    state_dict=load_file("temporal_lllite_step001440.safetensors"),
    inference_mode=True,
    hooks="down_blocks",   # 38-hook subset used in the paper
)
# 3. Run the streaming inference pipeline:
#    see `scripts/demo_camera.py` in github.com/otanl/dreamlite-stream
```

Full reproduction requires the upstream DreamLite-mobile checkpoint
(currently gated by the upstream project's release-request process).
The adapter alone is not useful without the base model.

## License

**CC BY-NC 4.0** (Creative Commons Attribution-NonCommercial 4.0).

The adapter is an **Adapted Material** of DreamLite-mobile (CC BY-NC 4.0
§1(a)) and inherits its non-commercial weight license. The inference
code on the linked GitHub repository is released under Apache-2.0 and
is unaffected by this inheritance.

See [ATTRIBUTION.md](https://github.com/otanl/dreamlite-stream/blob/main/ATTRIBUTION.md)
for the full attribution chain (DreamLite, Qwen3-VL, ControlNet-LLLite,
kohya-ss/sd-scripts).

## Citation

Until the peer-reviewed version is published, please cite the Zenodo-archived
release of the inference repository:

```bibtex
@software{ootani2026dreamlite_stream,
  author  = {Ootani, Yoshiyuki},
  title   = {{dreamlite-stream}: Video-Rate Streaming Stylization on a
             Vision-Aware MLLM-Conditioned Edit Diffusion},
  year    = {2026},
  version = {v0.1.0-tcsvt-submission},
  doi     = {10.5281/zenodo.20389428},
  url     = {https://github.com/otanl/dreamlite-stream}
}
```

The arXiv preprint will be added here once endorsed and released.

## Notes

- The adapter was trained on a single oil-painting prompt; for
  prompt-level generalisation use the v4 multi-prompt variant once
  released.
- The `down_blocks` hook subset (38 of 108 hooks) is the recommended
  inference configuration; see §III-D and Tables II / VI of the paper
  for the smoothing-artifact disclosure and cond-refresh sweep
  rationale.
- This is **not** a standalone model: it is a temporal-consistency
  side-network for DreamLite-mobile. Users must obtain DreamLite-mobile
  separately from the upstream project under their own licence.