Real-Time Visual Attribution Streaming in Thinking Model

Seil Kang Woojung Han Junhyeok Kim Jinyeong Kim Youngeun Kim Seong Jae Hwang

Abstract

We present an amortized framework for real-time visual attribution streaming in multimodal thinking models. When these models generate code from a screenshot or solve math problems from images, their long reasoning traces should be grounded in visual evidence. However, verifying this reliance is challenging: faithful causal methods require costly repeated backward passes or perturbations, while raw attention maps offer instant access, they lack causal validity. To resolve this, we introduce an amortized approach that learns to estimate the causal effects of semantic regions directly from the rich signals encoded in attention features. Across five diverse benchmarks and four thinking models, our approach achieves faithfulness comparable to exhaustive causal methods while enabling visual attribution streaming, where users observe grounding evidence as the model reasons, not after. Our results demonstrate that real-time, faithful attribution in multimodal thinking models is achievable through lightweight learning, not brute-force computation.

Machine Learning, ICML

Refer to caption — Figure 1: Faithfulness-Efficiency Trade-off. (a) Baseline methods compromise either efficiency or faithfulness. Our approach simultaneously achieves both faithfulness and efficiency ( $R^{2}$ of predicted vs. actual logit drops). (b) Latency scaling with context length. Unlike the baseline, which exhibits linear cost growth and OOM errors on long traces, our method operates with constant overhead below the real-time threshold (dashed line).

1 Introduction

Recent scaling of Vision-Language Models has shifted the focus from single-turn question answering to multi-step reasoning over visual evidence (OpenAI, 2024, 2025; DeepSeek-AI et al., 2025; Google DeepMind, 2024; Qwen Team, 2024; Yue et al., 2024). Todays, they can generate code from website screenshots, solve geometry problems from diagrams, and interpret complex charts (Lu et al., 2023; Yue et al., 2024; Zhang et al., 2024a). By decomposing queries into intermediate steps, these models produce extended reasoning traces that theoretically reveal which visual elements support which conclusions (Xu et al., 2025; Wang et al., 2024b; Gemini Team and others, 2024). However, verifying whether these long traces genuinely rely on visual evidence remains a critical challenge.

In practice, reasoning traces often exhibit ungrounded hallucinations, producing plausible narratives without actual visual support (Li et al., 2023; Favero et al., 2024). Models may correctly cite “the angle at vertex B” while attending to an irrelevant image region (Tong et al., 2024), or hallucinate numerical values absent from the visual input (Jing et al., 2024). These failures create a diagnostic blind spot, making it unclear whether errors originate in perception or symbolic reasoning (Chen et al., 2025). As reasoning chains grow longer, models increasingly rely on language priors rather than visual evidence (Liu et al., 2025), yet without reliable tools to verify visual grounding, the interpretability promised by reasoning traces remains illusory.

Attribution methods offer a principled way to verify visual grounding by quantifying which image regions causally influence each generated token. However, as shown in Figure˜1a, existing approaches face a competing demands of faithfulness and efficiency. Raw attention weights (Abnar and Zuidema, 2020) can be extracted instantly during inference, but they are unreliable as causal explanations because attention distributions can shift substantially without altering model predictions (Jain and Wallace, 2019; Pruthi et al., 2020; Wu et al., 2024). Gradient-based (Zeiler and Fergus, 2014; Chefer et al., 2021; Achtibat et al., 2024) and perturbation-based methods (Fong and Vedaldi, 2017; Hooker et al., 2019; Li et al., 2025) achieve higher faithfulness by measuring how outputs change when inputs are modified, but at substantial computational cost. This cost grows with context length (Figure˜1b). As reasoning traces extend to thousands of tokens, per-token latency increases dramatically, making real-time analysis infeasible. For interactive debugging, where users iteratively refine prompts or inspect reasoning step-by-step, such latency is unacceptable.

We resolve this trade-off through amortized attribution (Jethani et al., 2022; Covert et al., 2024), training a lightweight estimator as a surrogate model to predict attributions rather than recompute them from scratch. Crucially, while individual attention weights are unreliable as direct explanations, the comprehensive attention patterns distributed across layers and heads serve as informative features for predicting causal effects (Abnar and Zuidema, 2020; Cohen-Wang et al., 2024, 2025). We hypothesize that these patterns contain sufficient signal to recover the counterfactual effect of masking a visual region on token probability. To ensure perceptual relevance, we operate on semantic units derived from self-supervised features (Siméoni et al., 2025) rather than raw pixels. By training a linear estimator to map attention features to rigorous ablation targets, we bypass the need for repeated inference.

With a minimal parameter footprint, we train our estimator on 2,000 examples in ${\sim}$ 4.5 hours of runtime on a single GPU, a one-time cost that amortizes over all subsequent inferences. At test time, our method computes attribution asynchronously within the token generation loop, enabling users to inspect on the fly where the model draws from at each reasoning step.

Our main contributions are as follows:

•

We formalize token-level visual attribution for multimodal reasoning as counterfactual region ablation effects, and identify the prohibitive scaling of direct perturbation for long reasoning traces (Section˜3.2).
•

We propose a real-time attribution streaming method that (i) constructs semantically coherent attribution units from self-supervised vision features and (ii) amortizes counterfactual effects from a single forward pass (Sections˜3.3 and 3.5).
•

Our amortized estimator enables single-pass inference with cost linear in trace length while matching strong gradient- and perturbation-based baselines on faithfulness metrics across four reasoning VLMs and five task families (Sections˜3.5 and 4).
•

We further study models’ step-by-step reasoning by providing an inspectable view of the semantic space over long traces and analyzing the reasoning trajectory dynamics enabled by attribution streaming (Section˜4.4).

2 Related Work

2.1 Multimodal Reasoning Paradigms

Recent vision-language models have shifted toward multi-step reasoning grounded in visual evidence, driven by benchmarks that explicitly require compositional explanations (Yue et al., 2024; Zhang et al., 2024a; Fu et al., 2024b; Lu et al., 2022, 2023; He et al., 2024; Yuan et al., 2025; Man et al., 2025; Wu et al., 2025). Correspondingly, models now generate long-form “thinking” traces (Xu et al., 2025; Wang et al., 2024b; Gemini Team and others, 2024; Google DeepMind, 2024; OpenAI, 2024, 2025) and employ architectures designed for broader reasoning capabilities (Li et al., 2024a, b; Hong et al., 2024; Luo et al., 2025). While these traces offer a debugging interface, they can sound plausible even when ungrounded, masking whether errors are perceptual or symbolic. Current reliability methods largely focus on final answer correctness rather than the supporting visual evidence for each model’s thinking step (Prabhu et al., 2025; Geigle et al., 2024; Yan et al., 2024; Fu et al., 2024a; Liu et al., 2026).

2.2 Faithful Attribution and Scalability

Attribution methods quantify feature relevance and are typically evaluated via counterfactual faithfulness (Hooker et al., 2019), often using deletion or masking perturbations (Petsiuk et al., 2018; Fong and Vedaldi, 2017; Yu et al., 2024; Zhang et al., 2024b). While text-to-image cross-attention is a common proxy for focus, it is often unfaithful (Jain and Wallace, 2019; Serrano and Smith, 2019; Bibal et al., 2022) and manipulable without affecting predictions (Pruthi et al., 2020). Although gradient or attention-flow methods (Abnar and Zuidema, 2020; Chefer et al., 2021; Achtibat et al., 2024) and recent multimodal tools (Li et al., 2025; Stan et al., 2024; Shen, 2025; Liang et al., 2025) improve fidelity, they scale poorly. Reasoning models producing thousands of tokens (Xu et al., 2025; Gemini Team and others, 2024) make interactive per-token attribution intractable. Prior amortized attribution methods train surrogates over SHAP-style value functions (Jethani et al., 2022; Covert et al., 2024), and concurrent work AT2 (Cohen-Wang et al., 2025) uses attention as features for text attribution. We extend this line to visual tokens under streaming generation, replacing Monte Carlo sampling with a linear estimator over cached cross-attention features that adds negligible overhead to the decoding loop.

3 Method

We present vStream (Visual Attribution Streaming), a framework for real-time, object-centric attribution in reasoning VLMs (Figure˜3). Our approach sidesteps the computational bottleneck of existing methods (Chefer et al., 2021; Li et al., 2025) by amortizing the counterfactual computation, learning to predict ablation effects from lightweight features extracted in a single forward pass.

3.1 Background

We consider a vision-language model (VLM) that takes a multimodal input $x=(I,q)$ , where $I$ is an image and $q$ is a textual query. The model autoregressively generates a sequence of tokens $y_{1:T}=(y_{1},\dots,y_{T})$ . For reasoning-centric VLMs, this sequence typically includes both intermediate reasoning tokens (the “thinking” trace) and the final answer. We denote the model distribution at step $t$ as $p_{\theta}(\cdot\mid x,y_{<t})$ .

3.2 Problem Formulation

Let the image $I$ be encoded into $M$ vision tokens, and let these tokens be partitioned into $K$ disjoint regions $\mathcal{R}=\{R_{1},\dots,R_{K}\}$ , where each $R_{k}\subseteq\{1,\dots,M\}$ . We define an ablation operator $\mathrm{ablate}(x,R_{k})$ that prevents the model from accessing visual information in region $R_{k}$ during generation.

Ablation via Attention Masking. We implement ablation by preventing information flow from specific vision tokens to the query, effectively removing them from the model’s context. Following Geva et al. (2023), we set the attention scores from all query positions to the vision tokens in region $R_{k}$ to $-\infty$ before the softmax normalization. Formally, let ${\bm{q}}_{t}\in\mathbb{R}^{d}$ denote the query vector at position $t$ and let ${\bm{K}}\in\mathbb{R}^{d\times M}$ contain the key vectors for all $M$ vision tokens. The standard attention logits are ${\bm{s}}={\bm{q}}_{t}^{\top}{\bm{K}}/\sqrt{d}$ . Under ablation of region $R_{k}$ , we modify the logits as:

s_{i}\leftarrow\begin{cases}-\infty&\text{if }i\in R_{k}\\ s_{i}&\text{otherwise}\end{cases}

(1)

for all layers and heads. This masking-based intervention preserves the original token positions and model state while cleanly removing the target region’s influence on subsequent generation (Cohen-Wang et al., 2025, 2024).

Ablation Effect. We define the token-level ablation effect of region $R_{k}$ at decoding step $t$ as the drop in log-probability of the generated token (Wang et al., 2022; Zhang and Nanda, 2023):

\Delta_{k}(t)=\log p_{\theta}(y_{t}\mid x,y_{<t})-\log p_{\theta}(y_{t}\mid\mathrm{ablate}(x,R_{k}),y_{<t}).

(2)

A positive $\Delta_{k}(t)$ indicates that region $R_{k}$ supports the generation of token $y_{t}$ .

We extend this token-level metric to semantic spans. Let $S\subseteq\{1,\dots,T\}$ denote a target span (e.g., a thinking step). We aggregate token-level effects to define the span-level ablation effect:

\Delta_{k}(S)=\sum_{t\in S}\Delta_{k}(t).

(3)

Under teacher-forcing, where we condition on the originally generated sequence $y_{<t}$ for both the original and ablated models, this sum equals the log-probability ratio of the entire span $\log p(y_{S}\mid x)-\log p(y_{S}\mid\mathrm{ablate}(x,R_{k}))$ .

From Tokens to Semantic Regions. While defining $R_{k}$ as individual vision tokens is theoretically possible, it is practically suboptimal (Ren and Malik, 2003). First, individual visual tokens are semantically ambiguous; a single token might capture an edge or texture that has no independent meaning without its neighbors. Second, visual information is spatially sparse and structured. Their salient information is concentrated in objects rather than distributed uniformly. Grouping tokens into semantically coherent units not only aligns attribution with human concepts (e.g., “the red car” vs. “token 142”) but also drastically reduces the search space for ablation. We therefore partition the image into semantic regions, making the counterfactual objective both interpretable and computationally tractable.

3.3 Semantic Region Unitization

The choice of region partition $\mathcal{R}$ is critical, as attributions to arbitrary patches are hard to interpret while attributions to semantic regions (objects, text blocks, diagram components) are actionable. We use features from DINOv3 (Siméoni et al., 2025), a visual foundation model, to partition vision tokens into semantically coherent regions. After aligning feature maps to the VLM’s vision encoder resolution, we apply agglomerative clustering with Ward’s linkage (Ward Jr, 1963), adaptively grouping similar tokens into $K$ disjoint regions (typically $K\in[16,128]$ ) that isolate objects and background elements without requiring external segmentation masks. We denote the resulting partition as a membership matrix ${\bm{M}}\in\{0,1\}^{K\times M}$ , where ${\bm{M}}_{k,i}=1$ indicates vision token $i$ belongs to region $R_{k}$ . See Appendix˜C for more details and explanations.

For ablation studies, we also consider three non-semantic baselines:

•

Token-wise: Each vision token is its own region ( $K=M$ ).
•

Random blocks: Partition the image into $K$ randomly sized rectangular regions.
•

Voronoi: Partition using a regular grid of $K$ Voronoi cells.

These baselines use fixed $K\in\{16,32,64,128\}$ , while DINO clustering determines $K$ adaptively; see Section˜4.3 for quantitative comparisons.

3.4 Attention Feature Extraction

To predict ablation effects without actually ablating, we need features that capture how much each region contributes to generating the target span. Attention patterns provide a natural signal: if a region strongly influences the generation of a span, blocking attention to that region should substantially change the output.

Let ${\bm{A}}^{(\ell,h)}_{t,i}$ denote the attention weight from token position $t$ to vision token $i$ in layer $\ell$ and head $h$ . For each target span $S$ and region $R_{k}$ , we compute the mean attention pooled over the span and the region:

{\bm{a}}^{(\ell,h)}_{S,k}=\frac{1}{|S|\cdot|R_{k}|}\sum_{t\in S}\sum_{i\in R_{k}}{\bm{A}}^{(\ell,h)}_{t,i}.

(4)

We concatenate these scalars across all layers and heads to form a feature vector:

{\bm{f}}_{S,k}=\mathrm{Concat}_{\ell,h}\big({\bm{a}}^{(\ell,h)}_{S,k}\big)\in\mathbb{R}^{L\cdot H},

(5)

where $L$ is the number of layers and $H$ is the number of heads. For a model with 32 layers and 36 heads, this yields a 1152-dimensional feature vector per (span, region) pair. We extract these features from samples where the model produces correct final answers, ensuring that the learned estimator captures attention patterns associated with successful reasoning (see Section˜B.1 for dataset details). Importantly, attention weights are already computed during generation, so extracting ${\bm{f}}_{S,k}$ adds negligible overhead.

3.5 Amortized Estimator

Given the attention features ${\bm{f}}_{S,k}$ for each region, we train a linear estimator to predict ablation effects. Let ${\bm{v}}\in\{-1,+1\}^{K}$ be a signed region mask where ${\bm{v}}_{k}=-1$ indicates ablation and ${\bm{v}}_{k}=+1$ indicates retention. This is distinct from the membership matrix ${\bm{M}}\in\{0,1\}^{K\times M}$ , which assigns tokens to regions. The signed encoding ensures that ${\bm{f}}_{S,{\bm{v}}}$ (Equation˜6) captures the difference between retained and ablated contributions, rather than a one-sided sum of retained features only. We define the combined feature for mask ${\bm{v}}$ as:

{\bm{f}}_{S,{\bm{v}}}=\sum_{k=1}^{K}{\bm{v}}_{k}\cdot{\bm{f}}_{S,k}.

(6)

A linear estimator with weights ${\bm{w}}\in\mathbb{R}^{L\cdot H}$ predicts the total ablation effect:

\widehat{\Delta}(S,{\bm{v}})={\bm{w}}^{\top}{\bm{f}}_{S,{\bm{v}}}.

(7)

The estimator has only $L\cdot H$ parameters, learning a single importance weight per layer-head pair.

Training. For each training sample, we generate $N{=}32$ random masks ${\bm{v}}^{(j)}$ and compute the ground-truth ablation effect $\Delta(S,{\bm{v}}^{(j)})$ via forward passes. We optimize ${\bm{w}}$ to maximize Pearson correlation between predictions and targets:

\mathcal{L}({\bm{w}})=-\rho\big(\{\widehat{\Delta}(S,{\bm{v}}^{(j)})\}_{j},\{\Delta(S,{\bm{v}}^{(j)})\}_{j}\big).

(8)

This scale-invariant objective focuses on relative ranking rather than absolute magnitudes, aligning with our evaluation metric. We evaluate deeper MLP variants in Section˜F.9: the linear model matches their accuracy while remaining orders of magnitude more parameter-efficient. See Appendix˜A for theoretical justification and Section˜B.4 for training details.

3.6 vStream: Visual Attribution Streaming

As the model generates its reasoning trace, we simultaneously stream visual attributions by caching text-to-vision cross-attention weights at each decoding step. Upon completing a semantic span $S$ , we aggregate trained weights into ${\bm{f}}_{S,k}$ and score each region via

\widehat{\Delta}_{k}(S)={\bm{w}}^{\top}{\bm{f}}_{S,k}.

(9)

Crucially, this computation is offloaded to an asynchronous background worker, decoupled from the main generation loop via a producer-consumer pattern. Because the attribution step runs in parallel with the generation of the subsequent span, it introduces near-zero latency overhead to the user experience (Figure˜3). See Section˜B.8 for detailed streaming architecture.

While region-level scores capture relevant objects, finer localization is often required. We redistribute the region attribution $\widehat{\Delta}_{k}(S)$ across patches using the DINOv3 attention map ${\bm{a}}^{\mathtt{DINO}}$ (computed during unitization) as a spatial prior:

s_{i}=\widehat{\Delta}_{R(i)}(S)\cdot\frac{{\bm{a}}^{\mathtt{DINO}}_{i}}{\sum_{j\in R(i)}{\bm{a}}^{\mathtt{DINO}}_{j}}.

(10)

This refinement highlights the most salient parts within each region while preserving the calibrated total importance from the estimator. See Appendix˜A for a complexity analysis.

Table 1: Attribution quality comparison. Each cell: (LDS / Top-5 Drop); higher is better. Green: best, cream: second best. We define real-time as

\geq

10 tokens/sec, exceeding the average human silent reading speed of 238 words/min (

\approx

6 tokens/sec) (Brysbaert, 2019), ensuring users can follow streaming output; vStream (Ours) achieves up to

117\times

speedup over gradient-based baselines.

Qwen3-VL-8B-Thinking

GLM-4.1V-9B-Thinking

Method

Math

Science

Document

Code

General

Avg.

Time (s/10tok)

Math

Science

Document

Code

General

Avg.

Time (s/10tok)

Random

0.31/0.08

0.28/0.11

0.32/0.15

0.33/0.09

0.29/0.07

0.31/0.10

.010

\pm

.002

0.28/0.12

0.33/0.14

0.31/0.13

0.33/0.10

0.28/0.11

0.31/0.12

.011

\pm

.003

Attention

0.39/0.31

0.37/0.28

0.42/0.33

0.39/0.29

0.43/0.26

0.40/0.29

.020

\pm

.005

0.41/0.30

0.42/0.35

0.40/0.29

0.45/0.32

0.38/0.27

0.41/0.31

.022

\pm

.006

InputGrad

0.63/0.72

0.71/0.68

0.65/0.71

0.58/0.96

0.64/0.75

0.64/0.76

2.80

\pm

1.06

0.58/0.74

0.63/1.12

0.69/0.73

0.62/0.78

0.63/0.81

0.63/0.84

3.05

\pm

1.15

AttnLRP

0.76/0.81

0.66/1.08

0.73/0.78

0.73/0.82

0.65/0.98

0.71/0.89

2.60

\pm

1.04

0.72/0.79

0.68/0.77

0.64/1.05

0.71/0.84

0.70/0.75

0.69/0.84

2.84

\pm

1.13

TAM

0.69/1.02

0.69/0.95

0.66/1.04

0.72/1.06

0.71/0.89

0.69/0.99

1.90

\pm

1.13

0.71/0.94

0.65/0.91

0.68/0.88

0.67/1.01

0.68/1.08

0.68/0.96

2.07

\pm

1.23

vStream (Ours)

0.75/1.05

0.71/0.92

0.74/1.09

0.69/0.91

0.70/1.02

0.72/1.00

.024

\pm

.002

0.69/0.97

0.74/0.89

0.67/1.01

0.68/1.10

0.73/0.95

0.70/0.98

.026

\pm

.002

MiMo-VL-7B

Cosmos-Reason1-7B

Method

Math

Science

Document

Code

General

Avg.

Time (s/10tok)

Math

Science

Document

Code

General

Avg.

Time (s/10tok)

Random

0.30/0.14

0.28/0.18

0.29/0.11

0.30/0.16

0.28/0.12

0.29/0.14

.009

\pm

.002

0.28/0.17

0.30/0.13

0.34/0.19

0.30/0.16

0.27/0.12

0.30/0.15

.010

\pm

.002

Attention

0.45/0.34

0.41/0.32

0.38/0.29

0.40/0.33

0.39/0.28

0.40/0.31

.018

\pm

.004

0.45/0.33

0.41/0.35

0.37/0.34

0.41/0.37

0.38/0.38

0.41/0.35

.020

\pm

.005

InputGrad

0.58/1.06

0.62/0.78

0.70/0.69

0.59/0.76

0.71/0.68

0.64/0.79

2.52

\pm

0.95

0.58/0.71

0.73/0.76

0.59/0.98

0.61/0.72

0.60/0.75

0.62/0.78

2.75

\pm

1.04

AttnLRP

0.74/0.80

0.75/0.96

0.62/0.95

0.72/0.81

0.66/1.04

0.70/0.91

2.34

\pm

0.94

0.76/0.80

0.65/1.09

0.71/0.78

0.69/1.00

0.72/0.79

0.71/0.89

2.55

\pm

1.02

TAM

0.70/0.98

0.66/0.88

0.66/0.91

0.68/1.04

0.64/0.96

0.67/0.95

1.71

\pm

1.02

0.75/0.94

0.66/1.02

0.68/0.89

0.65/0.95

0.69/1.01

0.69/0.96

1.86

\pm

1.11

vStream (Ours)

0.75/1.08

0.65/0.97

0.71/0.98

0.70/0.96

0.70/1.10

0.70/1.02

.022

\pm

.002

0.74/0.99

0.67/0.90

0.68/1.01

0.71/1.04

0.69/1.05

0.70/1.00

.024

\pm

.002

4 Experiments

We evaluate our method on four reasoning VLMs across five task categories, addressing three questions: (1) Does our method accurately predict ablation effects? (2) Does the estimator generalize across tasks? (3) What do the attributions reveal about model behavior? Lastly, we conduct additional analysis to understand how visual attribution dynamics differ between successful and unsuccessful reasoning.

4.1 Setup

Models and datasets. We use four reasoning VLMs that generate extended thinking chains: Qwen3-VL-8B-Thinking (Qwen Team, 2025), GLM-4.1V-9B-Thinking (Hong et al., 2025), MiMo-VL-7B-RL (Team et al., 2025), and Cosmos-Reason1-7B (Azzolini et al., 2025). To test generalization, we cover five categories: Math, Science, Document, Code, and General. See Section˜B.1 for dataset details per category.

Baselines. We compare against five baselines spanning different paradigms:

•

Random: Uniform random attribution scores.
•

Attention: Average attention weights from text tokens to vision tokens, pooled across all layers and heads.
•

InputGrad (Hechtlinger, 2016): Gradient of output logits with respect to input pixel values.
•

AttnLRP (Achtibat et al., 2024): Attention-aware layer-wise relevance propagation.
•

TAM (Li et al., 2025): Token activation maps based on intermediate activations.

We adapt all baselines to produce region-level scores using our semantic unitization, ensuring fair comparison at the same granularity.

Metrics. We use two complementary metrics:

•

LDS (Linear Datamodeling Score) (Park et al., 2023; Cohen-Wang et al., 2024, 2025): Spearman correlation between predicted and actual ablation effects across regions, measuring how well the method ranks regions by causal importance.
•

Top-K Drop (Chattopadhay et al., 2018): Log-probability drop when ablating the $K$ most attributed regions. We use $K{=}5$ ; higher drop indicates the method correctly identified causally important regions.

LDS evaluates ranking fidelity against ground-truth counterfactual effects, while Top-K Drop measures practical utility for identifying critical visual evidence.

4.2 Main Results

Attribution quality. Table˜1 compares methods across four models and five categories. vStream matches the strongest baselines in LDS and achieves best or second-best Top-5 Drop in 16/20 settings. Figure˜4 shows vStream achieves $R^{2}=0.65$ between predicted and actual effects. Crucially, vStream streams these attributions with negligible overhead, whereas gradient-based methods require backward passes and perturbation-based methods require additional compute steps. We also evaluate segmentation quality via mIoU in Section˜F.13.

Cross-task generalization. Table˜2 examines whether an estimator trained on one task category generalizes to others. Using Qwen3-VL, we train on each category independently and evaluate on all five. In-domain (diagonal) LDS ranges from 0.70–0.74, while cross-task transfer retains 75–90% of this performance for most pairs. Math and Science show strong mutual transfer (LDS 0.62–0.63), likely due to shared diagram structures. Transfer to Document tasks is weaker (LDS 0.54–0.58), reflecting the distinct visual layout of dense text and tables. Training on a mixture of all categories recovers full performance, suggesting a single estimator suffices for diverse applications. See Appendix˜F for results on other models.

Qualitative analysis. Figure˜5 illustrates vStream’s unique capability: streaming attributions for each thinking step as the model reasons. Unlike baselines that produce a single post-hoc map, vStream reveals which regions the model references across intermediate steps, exposing failure modes invisible to global methods. For instance, a model may attend to the correct region initially but drift to irrelevant areas mid-reasoning. Additional examples across models and task categories are provided in Appendix˜G.

Table 2: Cross-task generalization. Each cell: (LDS / Top-5 Drop). Diagonal: in-domain.

Train	Eval Category					Avg.
Train	Math	Science	Document	Code	General	Avg.
Math	0.72 / 0.82	0.63 / 0.68	0.54 / 0.48	0.60 / 0.58	0.55 / 0.50	0.61 / 0.61
Science	0.62 / 0.66	0.71 / 0.80	0.58 / 0.54	0.55 / 0.50	0.60 / 0.58	0.61 / 0.62
Document	0.53 / 0.50	0.56 / 0.52	0.70 / 0.78	0.57 / 0.54	0.65 / 0.72	0.60 / 0.61
Code	0.59 / 0.60	0.54 / 0.48	0.56 / 0.52	0.73 / 0.84	0.62 / 0.66	0.61 / 0.62
General	0.57 / 0.54	0.61 / 0.64	0.64 / 0.70	0.60 / 0.62	0.74 / 0.86	0.63 / 0.67
Mix-up	0.75 / 1.05	0.71 / 0.92	0.74 / 1.09	0.69 / 0.91	0.70 / 1.02	0.72 / 1.00

4.3 Ablation Study

Semantic regions outperform geometric partitions. In Figure˜6, we compare three region unitization strategies on Qwen3-VL: random rectangular blocks, regular Voronoi tessellation, and our DINOv3-based semantic clustering. Our clustering approach significantly outperforms both geometric alternatives on the LDS metric. Random block and Voronoi partitions use a fixed grid, placing a ceiling on performance, whereas DINOv3-based clustering adaptively adjusts regions to image content. This confirms that semantic-level attribution to objects, symbols, and text blocks directly contributes to more interpretable and accurate results. Results comparing other vision foundation models (Radford et al., 2021; Zhai et al., 2023) are provided in Section˜F.5.

Training data efficiency. A practical concern is how much ablation data is needed to train an effective estimator, as shown in Figure˜8. We vary the number of training examples from 100 to 5,000 and measure LDS on a held-out set (Figure˜8). Performance improves steeply up to approximately 500 examples, then converges; with 2,000 examples, the estimator reaches full capacity. This efficiency is enabled by two factors: (1) the estimator has only $L\times H$ parameters (e.g., 784), so overfitting is difficult, and (2) each example provides 32 mask-effect pairs, effectively multiplying the data. Practitioners can train a domain-specific estimator with a few hours of ablation data collection, making the approach accessible for specialized applications. See Appendix˜F for additional details.

4.4 Reasoning Trajectory Dynamics

Beyond static attribution maps, we ask whether the temporal evolution of visual reliance reveals signatures of reasoning quality. Successful reasoning traces exhibit more stable visual grounding dynamics than unsuccessful ones, with attribution trajectories that move less and turn less over the course of the thinking process (Figures˜7 and 9).

At each reasoning step $s$ , we compute a region-effect vector ${\bm{e}}_{s}\in\mathbb{R}^{K}$ of predicted ablation effects. To compare examples with different numbers of regions, we canonicalize each step by keeping the top- $R$ regions ( $R{=}32$ ) and project the resulting profiles with PCA for visualization.

As visualized in Figure˜7, unsuccessful reasoning chains exhibit more tangled and convoluted trajectories compared to successful ones, reflecting unstable visual grounding. Quantitatively, as shown in Figure˜9, successful chains have shorter path length in PCA space than unsuccessful chains ( $0.003$ vs. $0.006$ , $n{=}1500$ ), and lower tortuosity, which measures how much the path wanders rather than progressing directly ( $13.7$ vs. $25.4$ , $n{=}1500$ ). We interpret this gap as reduced hypothesis switching (Zhong et al., 2024), where successful chains quickly commit to a consistent set of regions while failures repeatedly reassign visual support.

On POPE (Li et al., 2023), unsuccessful cases further split into two geometrically distinct failure modes. Hallucinations sustain high attribution concentration throughout reasoning, a fixation on a single incorrect object (also confirmed on CHAIR (Rohrbach et al., 2018); see Section˜F.7), while reasoning errors show low, unstable concentration, a wandering pattern of repeated region switching (Figure˜10, left). Both modes emerge well before generation ends: tortuosity-based failure prediction reaches AUC $0.69$ at $30\%$ of elapsed reasoning (Figure˜10, right), and the two modes separate cleanly from successes in trajectory metric space (Figure˜11). A second independent early-warning signal appears in per-step fidelity $R^{2}$ , which drops for incorrect chains at $\sim$ $20\%$ elapsed (Figure˜12). Both early-warning signals require access to the per-step attribution stream rather than a post-hoc map. Full statistics are in Section˜F.7.

5 Conclusion

We presented vStream, an amortized framework that enables real-time visual attribution streaming in multimodal thinking models. By learning to predict causal ablation effects from attention features, our approach achieves faithfulness comparable to existing methods while adding negligible computational overhead. It generalizes across models and tasks, preserving causal reliability at minimal cost. Beyond static attribution, this efficiency enables trajectory analysis that uncovers a behavioral signature. By design, vStream extends beyond thinking models to any autoregressive VLM. We provide further discussion and limitation in the Appendix˜H.

Impact Statement

This paper presents work whose goal is to advance the field of machine learning, specifically in the area of interpretability and transparency for vision-language models. By enabling real-time visual attribution, our method helps practitioners understand and verify model reasoning, which we believe contributes positively to the development of trustworthy AI systems. A more detailed discussion of broader impacts and limitations is provided in Appendix˜H.

References

S. Abnar and W. Zuidema (2020) Quantifying attention flow in transformers. In ACL, External Links: 2005.00928 Cited by: §1, §1, §2.2.
R. Achtibat, S. M. V. Hatefi, M. Dreyer, A. Jain, T. Wiegand, S. Lapuschkin, and W. Samek (2024) AttnLRP: attention-aware layer-wise relevance propagation for transformers. In ICML, External Links: 2402.05602 Cited by: §F.2.1, §1, §2.2, 4th item.
A. Azzolini, J. Bai, H. Brandon, J. Cao, P. Chattopadhyay, H. Chen, J. Chu, Y. Cui, J. Diamond, Y. Ding, et al. (2025) Cosmos-reason1: from physical common sense to embodied reasoning. arXiv preprint arXiv:2503.15558. Cited by: §4.1.
A. Bibal, R. Cardon, D. Alfter, R. Wilkens, X. Wang, T. François, and P. Watrin (2022) Is attention explanation? an introduction to the debate. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3889–3900. Cited by: §2.2.
M. Brysbaert (2019) How many words do we read per minute? a review and meta-analysis of reading rate. Journal of Memory and Language 109, pp. 104047. External Links: Document Cited by: Table 1, Table 1.
A. Chattopadhay, A. Sarkar, P. Howlader, and V. N. Balasubramanian (2018) Grad-cam++: generalized gradient-based visual explanations for deep convolutional networks. In IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 839–847. Cited by: 2nd item.
H. Chefer, S. Gur, and L. Wolf (2021) Transformer interpretability beyond attention visualization. In CVPR, External Links: 2012.09838 Cited by: §1, §2.2, §3.
G. Chen, W. Xu, H. Zhang, H. P. Chan, D. Zhao, A. T. Luu, and Y. Rong (2025) GeoPQA: bridging the visual perception gap in mllms for geometric reasoning. Cited by: §1.
B. Cohen-Wang, Y. Chuang, and A. Madry (2025) Learning to attribute with attention. arXiv preprint arXiv:2504.13752. Cited by: §1, §2.2, §3.2, 1st item.
B. Cohen-Wang, H. Shah, K. Georgiev, and A. Madry (2024) Contextcite: attributing model generation to context. Advances in Neural Information Processing Systems 37, pp. 95764–95807. Cited by: §1, §3.2, 1st item.
I. Covert, C. Kim, S. Lee, J. Zou, and T. Hashimoto (2024) Stochastic amortization: a unified approach to accelerate feature and data attribution. In Advances in Neural Information Processing Systems (NeurIPS), Note: arXiv:2401.15866 Cited by: §1, §2.2.
DeepSeek-AI, D. Guo, D. Yang, H. Zhang, J. Song, P. Wang, et al. (2025) DeepSeek-R1: incentivizing reasoning capability in LLMs via reinforcement learning. Nature 645, pp. 633–638. External Links: Document Cited by: §1.
A. Favero, L. Zancato, M. Trager, S. Choudhary, P. Perera, A. Achille, A. Swaminathan, and S. Soatto (2024) Multi-modal hallucination control by visual information grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14303–14312. Cited by: §1.
R. Fong and A. Vedaldi (2017) Interpretable explanations of black boxes by meaningful perturbation. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), External Links: 1704.03296 Cited by: §1, §2.2.
D. Fu, T. Xiao, R. Wang, W. Zhu, P. Zhang, G. Pang, R. Jia, and L. Chen (2024a) Tldr: token-level detective reward model for large vision language models. arXiv preprint arXiv:2410.04734. Cited by: §2.1.
X. Fu, Y. Hu, B. Li, Y. Feng, H. Wang, X. Lin, D. Roth, N. A. Smith, W. Ma, and R. Krishna (2024b) Blink: multimodal large language models can see but not perceive. In European Conference on Computer Vision, pp. 148–166. Cited by: §2.1.
A. Geiger, D. Ibeling, A. Zur, T. Icard, and C. Potts (2025) Causal abstraction: a theoretical foundation for mechanistic interpretability. Journal of Machine Learning Research 26, pp. 1–58. Cited by: §A.1.
G. Geigle, R. Timofte, and G. Glavaš (2024) Does object grounding really reduce hallucination of large vision-language models?. arXiv preprint arXiv:2406.14492. Cited by: §2.1.
Gemini Team et al. (2024) Gemini 1.5: unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530. External Links: 2403.05530 Cited by: §1, §2.1, §2.2.
M. Geva, J. Bastings, K. Filippova, and A. Globerson (2023) Dissecting recall of factual associations in auto-regressive language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 12216–12235. Cited by: §3.2.
Google DeepMind (2024) Gemini 2.0 flash thinking experimental. Note: https://blog.google/innovation-and-ai/models-and-research/google-deepmind/google-gemini-ai-update-december-2024/Google Technical Report Cited by: §1, §2.1.
M. Guillaumin, D. Küttel, and V. Ferrari (2014) ImageNet auto-annotation with segmentation propagation. International Journal of Computer Vision 110, pp. 328–348. Cited by: §F.13.
C. He, R. Luo, Y. Bai, S. Hu, Z. Thai, J. Shen, J. Hu, X. Han, Y. Huang, Y. Zhang, et al. (2024) Olympiadbench: a challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3828–3850. Cited by: §2.1.
Y. Hechtlinger (2016) Interpretation of prediction models using the input gradient. arXiv preprint arXiv:1611.07634. Cited by: §F.2.2, 3rd item.
W. Hong, W. Wang, M. Ding, W. Yu, Q. Lv, Y. Wang, Y. Cheng, S. Huang, J. Ji, Z. Xue, et al. (2024) CogVLM2: visual language models for image and video understanding. arXiv preprint arXiv:2408.16500. External Links: 2408.16500 Cited by: §2.1.
W. Hong, W. Yu, X. Gu, G. Wang, G. Gan, H. Tang, J. Cheng, J. Qi, J. Ji, L. Pan, et al. (2025) GLM-4.5v and glm-4.1v-thinking: towards versatile multimodal reasoning with scalable reinforcement learning. arXiv preprint arXiv:2507.01006. Cited by: §4.1.
S. Hooker, D. Erhan, P. Kindermans, and B. Kim (2019) A benchmark for interpretability methods in deep neural networks. In Advances in Neural Information Processing Systems (NeurIPS), External Links: 1806.10758 Cited by: §1, §2.2.
D. A. Hudson and C. D. Manning (2019) GQA: a new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6700–6709. Cited by: 1st item.
S. Jain and B. C. Wallace (2019) Attention is not explanation. In NAACL-HLT, Cited by: §H.2, §1, §2.2.
N. Jethani, M. Sudarshan, I. Covert, S. Lee, and R. Ranganath (2022) FastSHAP: real-time shapley value estimation. In International Conference on Learning Representations (ICLR), Cited by: §1, §2.2.
L. Jing, R. Li, Y. Chen, J. Shang, D. Lu, K. Shu, and H. Lu (2024) FaithScore: fine-grained evaluations of hallucinations in large vision-language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 4959–4984. Cited by: §1.
S. Kazemzadeh, V. Ordonez, M. Matten, and T. Berg (2014) ReferItGame: referring to objects in photographs of natural scenes. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 787–798. Cited by: §F.13.
J. J. Lau, S. Gayen, A. Ben Abacha, and D. Demner-Fushman (2018) A dataset of clinically generated visual questions and answers about radiology images. Scientific Data 5 (1), pp. 180251. External Links: Document Cited by: §F.12.
H. Laurençon, L. Saulnier, L. Tronchon, S. Bekman, A. Singh, A. Lozhkov, T. Wang, S. Karamcheti, A. Rush, D. Kiela, M. Cord, and V. Sanh (2024) Unlocking the conversion of web screenshots into html code with the websight dataset. arXiv preprint arXiv:2403.09029. Cited by: 2nd item.
B. Li, Y. Zhang, D. Guo, R. Zhang, F. Li, H. Zhang, K. Zhang, P. Zhang, Y. Li, Z. Liu, and C. Li (2024a) LLaVA-onevision: easy visual task transfer. arXiv preprint arXiv:2408.03326. External Links: 2408.03326 Cited by: §2.1.
F. Li, R. Zhang, H. Zhang, Y. Zhang, B. Li, W. Li, Z. Ma, and C. Li (2024b) LLaVA-next-interleave: tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895. External Links: 2407.07895 Cited by: §2.1.
Y. Li, H. Wang, X. Ding, H. Wang, and X. Li (2025) Token activation map to visually explain multimodal llms. In ICCV, External Links: 2506.23270 Cited by: §F.2.3, §1, §2.2, §3, 5th item.
Y. Li, Y. Du, K. Zhou, J. Wang, W. X. Zhao, and J. Wen (2023) Evaluating object hallucination in large vision-language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 292–305. Cited by: §1, §4.4.
J. Liang, R. Chen, X. Jiao, S. Liang, S. Liu, Q. Zhang, Z. Hu, and X. Cao (2025) Explaining multimodal llms via intra-modal token interactions. arXiv preprint arXiv:2509.22415. Cited by: §2.2.
T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick (2014) Microsoft coco: common objects in context. In European Conference on Computer Vision (ECCV), pp. 740–755. Cited by: §F.13.
C. Liu, Z. Xu, Q. Wei, J. Wu, J. Zou, X. E. Wang, Y. Zhou, and S. Liu (2025) More thinking, less seeing? assessing amplified hallucination in multimodal reasoning models. arXiv preprint arXiv:2505.21523. Cited by: §1.
S. Liu, S. Yang, D. Fang, S. Jia, Y. Tang, L. Su, R. Peng, Y. Yan, X. Zou, and X. Hu (2026) Vision-language introspection: mitigating overconfident hallucinations in mllms via interpretable bi-causal steering. External Links: 2601.05159 Cited by: §2.1.
P. Lu, H. Bansal, T. Xia, J. Liu, C. Li, H. Hajishirzi, H. Cheng, K. Chang, M. Galley, and J. Gao (2023) Mathvista: evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255. Cited by: 1st item, §1, §2.1.
P. Lu, S. Mishra, T. Xia, L. Qiu, K. Chang, S. Zhu, O. Tafjord, P. Clark, and A. Kalyan (2022) Learn to explain: multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems 35, pp. 2507–2521. Cited by: 1st item, §2.1.
S. M. Lundberg and S. Lee (2017) A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 30. Cited by: §A.1.
G. Luo, X. Yang, W. Dou, Z. Wang, J. Liu, J. Dai, Y. Qiao, and X. Zhu (2025) Mono-internvl: pushing the boundaries of monolithic multimodal large language models with endogenous visual pre-training. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 24960–24971. Cited by: §2.1.
Y. Man, D. Huang, G. Liu, S. Sheng, S. Liu, L. Gui, J. Kautz, Y. Wang, and Z. Yu (2025) Argus: vision-centric reasoning with grounded chain-of-thought. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Note: arXiv:2505.23766 Cited by: §2.1.
M. Mathew, D. Karatzas, and C. V. Jawahar (2021) DocVQA: a dataset for vqa on document images. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 2200–2209. Cited by: 1st item.
F. Mehri, M. S. Baghshah, and M. T. Pilehvar (2025) LibraGrad: balancing gradient flow for universally better vision transformer attributions. In CVPR, External Links: 2411.16760 Cited by: 3rd item.
OpenAI (2024) Learning to reason with LLMs. Note: https://openai.com/index/learning-to-reason-with-llms/OpenAI Technical Report Cited by: §1, §2.1.
OpenAI (2025) Introducing OpenAI o3 and o4-mini. Note: https://openai.com/index/introducing-o3-and-o4-mini/OpenAI Technical Report. Cited by: §1, §2.1.
K. Park, Y. J. Choe, and V. Veitch (2024) The linear representation hypothesis and the geometry of large language models. In Proceedings of the 41st International Conference on Machine Learning (ICML), Cited by: §A.1.
S. M. Park, K. Georgiev, A. Ilyas, G. Leclerc, and A. Madry (2023) TRAK: attributing model behavior at scale. In International Conference on Machine Learning, pp. 27074–27113. Cited by: 1st item.
V. Petsiuk, A. Das, and K. Saenko (2018) RISE: randomized input sampling for explanation of black-box models. In British Machine Vision Conference (BMVC), External Links: 1806.07421 Cited by: §2.2.
V. Prabhu, S. Purushwalkam, A. Yan, C. Xiong, and R. Xu (2025) Trust but verify: programmatic vlm evaluation in the wild. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3258–3267. Cited by: §2.1.
D. Pruthi, M. Gupta, B. Dhingra, G. Neubig, and Z. C. Lipton (2020) Learning to deceive with attention-based explanations. In Proceedings of the 58th annual meeting of the association for computational linguistics, pp. 4782–4793. Cited by: §1, §2.2.
Qwen Team (2024) QVQ: an open-weight model for multimodal reasoning. Note: https://qwenlm.github.io/blog/qvq-72b-preview/Alibaba Qwen Technical Report Cited by: §1.
Qwen Team (2025) Qwen3-VL technical report. arXiv preprint arXiv:2511.21631. Cited by: §4.1.
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. (2021) Learning transferable visual models from natural language supervision. In International Conference on Machine Learning (ICML), pp. 8748–8763. Cited by: §F.5, §4.3.
Ren and Malik (2003) Learning a classification model for segmentation. In Proceedings ninth IEEE international conference on computer vision, pp. 10–17. Cited by: §3.2.
A. Rohrbach, L. A. Hendricks, K. Burns, T. Darrell, and K. Saenko (2018) Object hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4035–4045. Cited by: §F.7.3, §4.4.
S. Serrano and N. A. Smith (2019) Is attention interpretable?. In ACL, Cited by: §2.2.
G. Shen (2025) GLIMPSE: holistic cross-modal explainability for large generative vision-language models. arXiv preprint arXiv:2506.18985. Cited by: §2.2.
O. Siméoni, H. V. Vo, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, V. Khalidov, M. Szafraniec, S. Yi, M. Ramamonjisoa, F. Massa, D. Haziza, L. Wehrstedt, J. Wang, T. Darcet, T. Moutakanni, et al. (2025) DINOv3. arXiv preprint arXiv:2508.10104. External Links: 2508.10104 Cited by: §1, §3.3.
G. B. M. Stan, E. Aflalo, R. Y. Rohekar, A. Bhiwandiwalla, S. Tseng, M. L. Olson, Y. Gurwicz, C. Wu, N. Duan, and V. Lal (2024) LVLM-interpret: an interpretability tool for large vision-language models. arXiv preprint arXiv:2404.03118. Cited by: §2.2.
C. Team, Z. Yue, Z. Lin, Y. Song, W. Wang, S. Ren, S. Gu, S. Li, P. Li, L. Zhao, L. Li, K. Bao, H. Tian, H. Zhang, G. Wang, D. Zhu, Cici, C. He, B. Ye, B. Shen, Z. Zhang, Z. Jiang, Z. Zheng, Z. Song, Z. Luo, Y. Yu, Y. Wang, Y. Tian, Y. Tu, Y. Yan, Y. Huang, X. Wang, X. Xu, X. Song, X. Zhang, X. Yong, X. Zhang, X. Deng, W. Yang, W. Ma, W. Lv, W. Zhuang, W. Liu, S. Deng, S. Liu, S. Chen, S. Yu, S. Liu, S. Wang, R. Ma, Q. Wang, P. Wang, N. Chen, M. Zhu, K. Zhou, K. Zhou, K. Fang, J. Shi, J. Dong, J. Xiao, J. Xu, H. Liu, H. Xu, H. Qu, H. Zhao, H. Lv, G. Wang, D. Zhang, D. Zhang, D. Zhang, C. Ma, C. Liu, C. Cai, and B. Xia (2025) MiMo-vl technical report. External Links: 2506.03569, Link Cited by: §4.1.
S. Tong, Z. Liu, Y. Zhai, Y. Ma, Y. LeCun, and S. Xie (2024) Eyes wide shut? exploring the visual shortcomings of multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14320–14330. Cited by: §1.
K. Wang, J. Pan, W. Shi, Z. Lu, H. Ren, A. Zhou, M. Zhan, and H. Li (2024a) Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems 37, pp. 95095–95169. Cited by: 2nd item.
K. Wang, A. Variengien, A. Conmy, B. Shlegeris, and J. Steinhardt (2022) Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. arXiv preprint arXiv:2211.00593. Cited by: §3.2.
P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, et al. (2024b) Qwen2-vl: enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191. External Links: 2409.12191 Cited by: §1, §2.1.
J. H. Ward Jr (1963) Hierarchical grouping to optimize an objective function. Journal of the American Statistical Association 58 (301), pp. 236–244. Cited by: §3.3.
J. Wu, W. Kang, H. Tang, Y. Hong, and Y. Yan (2024) On the faithfulness of vision transformer explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10936–10945. Cited by: §1.
Q. Wu, X. Yang, Y. Zhou, C. Fang, B. Song, X. Sun, and R. Ji (2025) Grounded chain-of-thought for multimodal large language models. arXiv preprint arXiv:2503.12799. Cited by: §2.1.
G. Xu, P. Jin, Z. Wu, H. Li, Y. Song, L. Sun, and L. Yuan (2025) Llava-cot: let vision language models reason step-by-step. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2087–2098. Cited by: §1, §2.1, §2.2.
S. Yan, M. Bai, W. Chen, X. Zhou, Q. Huang, and L. E. Li (2024) Vigor: improving visual grounding of large vision language models with fine-grained reward modeling. In European Conference on Computer Vision, pp. 37–53. Cited by: §2.1.
C. Yang, Y. Huang, Z. Chen, J. Gao, W. Zhong, J. Wang, J. Liang, X. Liu, D. Chen, R. Bhardwaj, T. Zhou, D. Cremers, B. Schiele, Q. Wang, L. Wu, and A. Li (2024) ChartMimic: evaluating lmm’s cross-modal reasoning capability via chart-to-code generation. arXiv preprint arXiv:2406.09961. Note: ICLR 2025 Cited by: 1st item.
W. Yu, Z. Yang, L. Ren, L. Li, J. Wang, K. Lin, C. Lin, Z. Liu, L. Wang, and X. Wang (2024) MM-vet v2: a challenging benchmark to evaluate large multimodal models for integrated capabilities. External Links: 2408.00765 Cited by: §2.2.
H. Yuan, Y. Sun, Y. Li, T. Zhang, X. Deng, H. Ding, L. Qi, A. Wang, X. Li, and M. Yang (2025) Visual reasoning tracer: object-level grounded reasoning benchmark. External Links: 2512.05091 Cited by: §2.1.
X. Yue, Y. Ni, K. Zhang, T. Zheng, R. Liu, G. Zhang, S. Stevens, D. Jiang, W. Ren, Y. Sun, et al. (2024) Mmmu: a massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9556–9567. Cited by: §1, §2.1.
M. D. Zeiler and R. Fergus (2014) Visualizing and understanding convolutional networks. In European Conference on Computer Vision (ECCV), Cited by: §1.
X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer (2023) Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11975–11986. Cited by: §F.5, §4.3.
F. Zhang and N. Nanda (2023) Towards best practices of activation patching in language models: metrics and methods. arXiv preprint arXiv:2309.16042. Cited by: §3.2.
R. Zhang, D. Jiang, Y. Zhang, H. Lin, Z. Guo, P. Qiu, A. Zhou, P. Lu, K. Chang, Y. Qiao, et al. (2024a) Mathverse: does your multi-modal llm truly see the diagrams in visual math problems?. In European Conference on Computer Vision, pp. 169–186. Cited by: 3rd item, §1, §2.1.
Y. Zhang, H. Zhang, H. Tian, C. Fu, S. Zhang, J. Wu, F. Li, K. Wang, Q. Wen, Z. Zhang, L. Wang, R. Jin, and T. Tan (2024b) MME-realworld: could your multimodal llm challenge high-resolution real-world scenarios that are difficult for humans?. External Links: 2408.13257 Cited by: §2.2.
W. Zhong, X. Feng, L. Zhao, Q. Li, L. Huang, Y. Gu, W. Ma, Y. Xu, and B. Qin (2024) Investigating and mitigating the multimodal hallucination snowballing in large vision-language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), pp. 11863–11879. Cited by: §4.4.

Technical Appendices

Appendix A Theoretical Foundations

In this section, we analyze our method through the lens of existing interpretability frameworks and formalize the properties of our estimator.

A.1 Connections to Existing Frameworks

Connection to Causal Abstraction. Our attribution score $\Delta_{k}(t)$ aligns with the interventional definitions in causal abstraction (Geiger et al., 2025), measuring the causal effect of removing visual region $R_{k}$ . Although conceptually similar to activation patching, we do not explicitly define a high-level structural causal model (SCM). Instead, we treat the interventional effect $\Delta_{k}(t)$ as a ground-truth signal to be efficiently approximated.

Relation to the Linear Representation Hypothesis. Our linear estimator $\widehat{\Delta}\approx{\bm{w}}^{\top}{\bm{f}}$ relies on the assumption that task-relevant causal information is linearly decodable, consistent with the Linear Representation Hypothesis (Park et al., 2024). We extend this to attention-based features. We note that linearity is an empirical assumption; we cannot theoretically guarantee that the complex counterfactual effect of a visual ablation is perfectly captured by a linear projection.

Interpretation as Marginal Contribution. The quantity $\Delta_{k}(t)$ represents the marginal contribution of region $R_{k}$ to the target token’s log-probability, similar to Shapley values (Lundberg and Lee, 2017). Unlike Shapley values, we do not average over all feature coalitions due to computational costs. Our metric captures individual impact but does not strictly guarantee efficiency or additivity.

A.2 Properties of the Estimator

We now formally state key properties of our estimator, motivating our choice of objective and quantifying the computational benefits.

First, we establish that the Pearson correlation objective is robust to affine transformations, justifying its use even if the scale of the predictor differs from the target.

Lemma A.1 (Affine invariance of the Pearson objective).

Let $u,v$ be random variables with $0<\mathrm{Var}(u),\mathrm{Var}(v)<\infty$ . For any $a\neq 0$ and any $b\in\mathbb{R}$ ,

\rho\!\left(u,\,av+b\right)=\operatorname{sign}(a)\,\rho(u,v).

In particular, for $a>0$ the Pearson correlation is invariant to scaling and shifting of the predictor.

Proof.

Using $\rho(u,v)=\frac{\mathrm{Cov}(u,v)}{\sqrt{\mathrm{Var}(u)\mathrm{Var}(v)}}$ , we have $\mathrm{Cov}(u,av+b)=a\,\mathrm{Cov}(u,v)$ and $\mathrm{Var}(av+b)=a^{2}\mathrm{Var}(v)$ , hence $\rho(u,av+b)=\frac{a}{|a|}\rho(u,v)=\operatorname{sign}(a)\rho(u,v)$ . ∎

Next, we analyze the computational advantage of our method compared to calculating true causal effects via brute-force ablation.

Proposition A.2 (Computational complexity comparison under teacher forcing).

Fix a target span $S$ and regions $\{R_{k}\}_{k=1}^{K}$ . Let $C_{\mathrm{fwd}}$ denote the cost of one VLM forward pass under teacher forcing, i.e., producing logits for all positions in $S$ while conditioning on the same prefix tokens $y_{<t}$ as in Equation˜2.

(Direct ablation, exact). Computing the exact ablation effects $\{\Delta_{k}(t)\}_{t\in S}$ for all regions requires one forward pass on each ablated input, hence $K$ additional forward passes per span:

\Theta(K\cdot C_{\mathrm{fwd}}),

(or $K{+}1$ forward passes if counting the unablated baseline as well).

(Ours, cached attention). After obtaining the cross-attention weights from the unablated run, we (i) pool cached attention over tokens and vision positions to form region features and (ii) apply a linear score for each region. Step (ii) alone costs $O(K\cdot L\cdot H)$ for $K$ dot-products in $\mathbb{R}^{L\cdot H}$ . The total cost is dominated by lightweight tensor reductions over cached attention and does not require any additional VLM forward passes.

We also justify the form of our estimator. If the conditional mean of the target is linear in the features, maximizing correlation recovers the optimal direction.

Proposition A.3 (Optimal linear predictor under linear conditional mean).

Let ${\bm{f}}\in\mathbb{R}^{d}$ be a feature vector with positive definite covariance ${\bm{\Sigma}}_{ff}$ , and let $\Delta$ be a scalar target. Assume:

\mathbb{E}[\Delta\mid{\bm{f}}]=\beta+{\bm{w}}_{\star}^{\top}{\bm{f}}\quad\text{for some }\beta\in\mathbb{R},\ {\bm{w}}_{\star}\in\mathbb{R}^{d}.

Then maximizing $\rho({\bm{w}}^{\top}{\bm{f}},\Delta)$ over ${\bm{w}}$ yields $\widehat{{\bm{w}}}\propto{\bm{w}}_{\star}$ .

Proof.

Write $\rho({\bm{w}}^{\top}{\bm{f}},\Delta)=\frac{{\bm{w}}^{\top}{\bm{\Sigma}}_{f\Delta}}{\sqrt{{\bm{w}}^{\top}{\bm{\Sigma}}_{ff}{\bm{w}}}\sqrt{\mathrm{Var}(\Delta)}}$ where ${\bm{\Sigma}}_{f\Delta}=\mathrm{Cov}({\bm{f}},\Delta)$ . Maximizing this Rayleigh quotient yields ${\bm{w}}\propto{\bm{\Sigma}}_{ff}^{-1}{\bm{\Sigma}}_{f\Delta}$ . Under the linear conditional mean, ${\bm{\Sigma}}_{f\Delta}={\bm{\Sigma}}_{ff}{\bm{w}}_{\star}$ , hence $\widehat{{\bm{w}}}\propto{\bm{w}}_{\star}$ . ∎

We emphasize that the linear conditional mean is an assumption, not a guaranteed property of VLMs. In practice, whether attention features linearly predict ablation effects is an empirical question; our experimental results suggest this approximation is effective, but it may not hold universally.

The following remark clarifies the connection between Pearson correlation and least-squares regression.

Remark A.4 (Correlation maximization as standardized least squares with optimal rescaling).

Let $\tilde{\Delta}=(\Delta-\mathbb{E}[\Delta])/\mathrm{std}(\Delta)$ so that $\mathbb{E}[\tilde{\Delta}]=0$ and $\mathrm{Var}(\tilde{\Delta})=1$ . For any predictor $z={\bm{w}}^{\top}{\bm{f}}$ , consider the best affine rescaling of $z$ to fit $\tilde{\Delta}$ :

\min_{\alpha\in\mathbb{R},\,b\in\mathbb{R}}\ \mathbb{E}\big[(\tilde{\Delta}-(\alpha z+b))^{2}\big].

The minimizer satisfies $b^{\star}=\mathbb{E}[\tilde{\Delta}-\alpha z]$ and $\alpha^{\star}=\mathrm{Cov}(\tilde{\Delta},z)/\mathrm{Var}(z)$ , yielding

\min_{\alpha,b}\ \mathbb{E}\big[(\tilde{\Delta}-(\alpha z+b))^{2}\big]=1-\rho(z,\Delta)^{2}.

Therefore, maximizing the Pearson objective $\rho({\bm{w}}^{\top}{\bm{f}},\Delta)$ is equivalent (up to an arbitrary sign of ${\bm{w}}$ ) to minimizing the standardized mean squared error after allowing the optimal affine rescaling of predictions. This explains why our correlation-based loss focuses on the pattern of effects rather than their absolute scale.

Finally, we note a basic consistency property: if a region has no associated features (e.g., zero attention), our estimator correctly predicts zero effect.

Remark A.5 (Dummy property).

If ${\bm{f}}_{S,k}={\bm{0}}$ , then $\widehat{\Delta}_{k}={\bm{w}}^{\top}{\bm{f}}_{S,k}=0$ for any ${\bm{w}}$ .

A.3 Detailed Complexity Analysis

We provide a detailed breakdown of computational costs for different attribution methods.

Table 3: Computational complexity comparison for attributing

T

tokens to

K

regions under teacher forcing. A single forward pass produces logits (and attention weights) for all

T

positions.

C_{\text{fwd}}

and

C_{\text{bwd}}

denote the cost of one forward and backward pass, respectively.

Method	Complexity	Passes
Perturbation (exact)	$O(K\cdot C_{\text{fwd}})$	$K$ forward
Gradient-based	$O(C_{\text{fwd}}+C_{\text{bwd}})$	$1$ forward + backward
Attention (raw)	$O(T\cdot L\cdot H\cdot M)$	0 (cached)
Ours	$O(T\cdot L\cdot H\cdot M+T\cdot K\cdot L\cdot H)$	0 (cached)

For a typical reasoning trace with $T=2000$ tokens, $K=50$ regions, $L=28$ layers, and $H=28$ heads (teacher forcing):

•

Perturbation (exact): $K=50$ forward passes on ablated inputs (plus one unablated baseline if counted)
•

Gradient-based: $1$ forward-backward pair (using a loss aggregated over the $T$ tokens)
•

Ours (scoring only): $T\times K\times L\times H=2000\times 50\times 28\times 28\approx 78\text{M}$ multiply-adds

The key insight is that our method’s complexity is independent of the VLM’s size, depending only on the number of layers and heads. This makes it equally efficient for 7B and 70B parameter models.

A.4 Why Attention Features Predict Ablation Effects

A natural question is: why should attention features contain information about counterfactual ablation effects? We provide intuition from two perspectives.

Information Bottleneck Perspective.

In transformer architectures, cross-attention weights modulate how much information flows from source positions (vision tokens) to target positions (generated text tokens). If the model assigns large cross-attention mass to a region $R_{k}$ , then the cross-attention block can inject more signal from that region into the token representation, so ablating $R_{k}$ should tend to have a larger counterfactual effect.

To make this precise, let ${\bm{o}}_{t}^{(\ell)}$ denote the cross-attention output component at text position $t$ in layer $\ell$ (i.e., the multi-head cross-attention output before adding the residual stream and before any MLP updates). A standard multi-head cross-attention computation can be written as

{\bm{o}}_{t}^{(\ell)}={\bm{W}}_{O}^{(\ell)}\Big[\sum_{i=1}^{M}{\bm{A}}^{(\ell,1)}_{t,i}\,{\bm{W}}_{V}^{(\ell,1)}{\bm{x}}_{i}\ ;\ \dots\ ;\ \sum_{i=1}^{M}{\bm{A}}^{(\ell,H)}_{t,i}\,{\bm{W}}_{V}^{(\ell,H)}{\bm{x}}_{i}\Big],

(11)

where ${\bm{A}}^{(\ell,h)}_{t,i}$ is the cross-attention weight from text token $t$ to vision token $i$ in head $h$ , ${\bm{W}}_{V}^{(\ell,h)}$ is the value projection, and ${\bm{W}}_{O}^{(\ell)}$ is the output projection.

The contribution of region $R_{k}$ to this cross-attention output is then the corresponding partial sum over $i\in R_{k}$ :

{\bm{o}}_{t}^{(\ell)}[R_{k}]\approx{\bm{W}}_{O}^{(\ell)}\Big[\sum_{i\in R_{k}}{\bm{A}}^{(\ell,1)}_{t,i}\,{\bm{W}}_{V}^{(\ell,1)}{\bm{x}}_{i}\ ;\ \dots\ ;\ \sum_{i\in R_{k}}{\bm{A}}^{(\ell,H)}_{t,i}\,{\bm{W}}_{V}^{(\ell,H)}{\bm{x}}_{i}\Big].

(12)

This approximation isolates the cross-attention pathway only; subsequent residual connections, normalization, and MLP mixing can further transform and redistribute this signal. Nevertheless, Equation˜12 highlights that cross-attention weights directly gate the magnitude of the region-dependent component injected into the representation, motivating why pooled attention features can correlate with ablation effects.

Gradient Flow Perspective.

During backpropagation, attention also modulates sensitivity to input perturbations by gating how gradients propagate from text positions back to vision tokens.

Let $\mathrm{head}_{t}^{(\ell,h)}=\sum_{i=1}^{M}{\bm{A}}^{(\ell,h)}_{t,i}\,{\bm{W}}_{V}^{(\ell,h)}{\bm{x}}_{i}$ denote the value-aggregation output of head $(\ell,h)$ at text position $t$ . Ignoring the dependence of ${\bm{A}}^{(\ell,h)}_{t,i}$ on ${\bm{x}}_{i}$ (i.e., dropping the additional terms that flow through the key and query pathways), the value-path contribution to the gradient can be written as the following heuristic approximation:

\frac{\partial\mathcal{L}}{\partial{\bm{x}}_{i}}\approx\sum_{t}\sum_{\ell,h}({\bm{W}}_{V}^{(\ell,h)})^{\top}\Big({\bm{A}}^{(\ell,h)}_{t,i}\,\frac{\partial\mathcal{L}}{\partial\mathrm{head}_{t}^{(\ell,h)}}\Big).

(13)

More generally, the exact gradient also contains additional terms arising from $\partial{\bm{A}}^{(\ell,h)}_{t,i}/\partial{\bm{x}}_{i}$ (through keys, and indirectly through queries when vision tokens influence later text states). Thus, Equation˜13 should be interpreted as an intuition: attention weights gate one dominant pathway for gradient flow, which helps explain why attention-derived features can predict counterfactual sensitivity.

A.5 Generalization Properties

We analyze factors affecting cross-task and cross-model generalization of our estimator.

Feature Distribution Shift.

The estimator’s generalization depends on how similar the attention feature distributions are between training and test conditions. Let $\mathcal{F}_{\text{train}}$ and $\mathcal{F}_{\text{test}}$ denote the feature distributions. If $\mathcal{F}_{\text{test}}$ is within the support of $\mathcal{F}_{\text{train}}$ , the linear estimator can interpolate effectively. However, if test features lie outside the training distribution (e.g., novel visual patterns), extrapolation may fail.

Our cross-task experiments (Table˜2) show that Math and Science transfer well to each other (shared diagram structures), while transfer to Document is weaker (distinct layouts). This aligns with the intuition that generalization depends on visual similarity.

Model Architecture Dependence.

Different VLM architectures may encode visual information in different layers and heads. Our estimator learns a weighting ${\bm{w}}$ over layer-head pairs that is specific to a given architecture. Cross-model transfer would require that:

1.

Both models have similar layer-head structure (same $L$ and $H$ ).
2.

Visual grounding emerges at similar depths in both models.

Our experiments train separate estimators per model, avoiding cross-model transfer issues. Future work could explore architecture-agnostic features that enable zero-shot transfer.

Sample Complexity.

Given the low dimensionality of our estimator ( $L\cdot H$ parameters), we expect good generalization from relatively few samples. The training data efficiency experiments (Figure˜8) confirm that approximately 500–2000 examples suffice for convergence, consistent with standard generalization bounds for linear models in $\mathbb{R}^{784}$ .

Appendix B Implementation Details

In this section, we provide the technical specifications required to reproduce our method, including details on the reasoning models employed, the DINO-based visual clustering pipeline, the training procedure for the linear estimator, and the hardware environment.

B.1 Datasets

We organize our evaluation datasets into five categories based on the type of visual reasoning required.

General.

•

GQA (Hudson and Manning, 2019): Real-world visual reasoning with compositional questions about spatial relations and object attributes.

Document.

•

DocVQA (Mathew et al., 2021): Question answering over document images including forms, invoices, and reports.

Science.

•

ScienceQA (Lu et al., 2022): Multimodal science questions across natural, social, and language sciences with chain-of-thought reasoning.

Math.

•

MathVista (Lu et al., 2023): Mathematical reasoning over diagrams, plots, and geometric figures.
•

MathVision (Wang et al., 2024a): Multi-step mathematical problem solving with visual inputs.
•

MathVerse (Zhang et al., 2024a): Diagram-based math problems testing visual-symbolic integration.

Code.

•

ChartMimic (Yang et al., 2024): Code generation to reproduce charts and visualizations from images.
•

WebSight (Laurençon et al., 2024): HTML/CSS generation from website screenshot references.

Note: For the Math and Code categories, we combine multiple datasets because the sub-9B models we evaluate often fail on the more challenging examples. Using a single dataset would yield insufficient correctly-answered samples for training data collection.

B.2 Reasoning Model Architectures

We evaluate our method across four state-of-the-art vision-language reasoning models. We selected these models to represent a diverse range of architectures and "thinking" capabilities.

•

Qwen3-VL-8B-Thinking: Based on the Qwen3 language model backbone, this model integrates a vision encoder initialized from SigLIP-SO400M. It uses a perceiver resampler to compress visual features into a fixed number of tokens (typically 256). The "Thinking" variant is fine-tuned on chain-of-thought (CoT) reasoning data, enabling it to generate intermediate reasoning steps before the final answer.
•

Cosmos-R1: A multimodal reasoning model emphasizing robust world knowledge. It uses a ViT-Huge vision encoder and a 7B parameter LLM decoder. The model processes images at a resolution of $336\times 336$ and uses a cross-attention mechanism for modality fusion.
•

MiMo-VL-7B: This model features a mixture-of-experts (MoE) architecture in the language decoder, allowing for efficient inference despite a larger total parameter count. The vision tower is a CLIP-ViT-L/14, and the projection layer consists of a simple MLP.
•

GLM-4.1V-9B-Thinking: A bilingual (English/Chinese) model with strong reasoning capabilities. It employs a GLM-4 transformer backbone with rotary positional embeddings. The visual component handles high-resolution inputs via a sliding window attention mechanism.

For all models, we access the internal attention weights from the last layer of the cross-attention blocks (or self-attention layers where visual tokens are concatenated) to serve as input features for our estimator.

B.3 Visual Feature Extraction and Clustering

To define interpretable visual regions for attribution, we employ a clustering approach based on self-supervised features.

Feature Extraction. We utilize DINOv3-Large as our feature extractor. We resize input images to $224\times 224$ and extract the patch-level features from the last transformer layer. DINOv3 is chosen for its superior ability to capture semantic object boundaries compared to supervised baselines.

Agglomerative Clustering. We perform spatially-constrained agglomerative clustering on the extracted patch features.

•

Distance Metric: We use cosine distance ( $1-\text{cosine similarity}$ ) between feature vectors to measure semantic dissimilarity.
•

Linkage Criterion: Ward’s linkage is used to minimize the variance within clusters.
•

Adaptive K: Instead of a fixed number of clusters, we dynamically determine the number of regions $K$ for each image based on a distance threshold $\tau=0.5$ (see Table˜4). In practice, this results in $K\in[16,128]$ regions per image, which provides a balance between granularity and interpretability.

B.4 Linear Estimator Training

Our method relies on a lightweight linear estimator $W$ trained to predict the impact of masking specific visual regions.

Data Sampling. For each training image, we generate synthetic training data by randomly masking subsets of visual regions.

•

We sample $N=32$ random binary masks ${\bm{v}}\in\{0,1\}^{K}$ per image.
•

For each mask, we compute the ground-truth effect on the model’s output log-probability for the target token.
•

This process is computationally efficient because it only requires forward passes (no gradients) and can be batched.

Optimization. The linear estimator is trained to maximize the Pearson correlation between the predicted attribution scores and the actual log-probability drops.

•

Objective Function: We minimize the negative Pearson correlation coefficient.
•

Optimizer: We use AdamW with a learning rate of $1\times 10^{-3}$ and weight decay of $1\times 10^{-4}$ .
•

Training Duration: Due to the simplicity of the linear model, training converges rapidly. We train for $2,000$ iterations, which typically takes less than 1 hour on a single GPU for the entire evaluation dataset.

B.5 Hardware and Computing Infrastructure

We conducted all experiments on a high-performance computing cluster.

•

GPUs: NVIDIA H100 (80GB) GPUs were used for inference of the large reasoning models (Qwen3-VL, GLM-4.1V). NVIDIA A100 (80GB) GPUs were sufficient for the smaller models and the training of the linear estimator.
•

Software Environment: PyTorch 2.4, CUDA 12.1, and the HuggingFace Transformers library. We utilized FlashAttention-2 for efficient attention computation during inference.

B.6 Attention Extraction Details

Extracting attention weights from different VLM architectures requires careful handling due to architectural variations.

Attention Caching.

During inference, we cache text-to-vision attention weights at each decoding step. All four models we evaluate (Qwen3-VL, GLM-4.1V, MiMo-VL, Cosmos-R1) use a decoder-only architecture that concatenates vision and text tokens in the same sequence. For these models, we extract the portion of the self-attention matrix corresponding to text $\rightarrow$ vision attention.

Memory Management.

For long reasoning traces (2000+ tokens), caching full attention tensors can consume significant GPU memory. We employ the following strategies:

•

Selective Layer Caching: We prioritize caching attention from middle layers (11–20), which contribute most to the attribution signal based on empirical analysis.
•

Span-wise Aggregation: Rather than storing per-token attention, we aggregate to span level on-the-fly when a reasoning step completes, reducing memory from $O(T\cdot L\cdot H\cdot M)$ to $O(S\cdot L\cdot H\cdot M)$ where $S$ is the number of spans.
•

Float16 Precision: Attention weights are stored in float16 format, halving memory requirements with negligible accuracy loss.

FlashAttention Compatibility.

FlashAttention does not natively return attention weights due to its memory-efficient implementation. For attribution, we use a hybrid approach: standard attention for the cross-modal layers (small overhead due to fixed vision token count) and FlashAttention for text-only self-attention layers. For detailed latency measurements under different attention backends and KV-cache configurations, see Table˜9.

B.7 Span Boundary Detection

To provide step-by-step attribution during reasoning, we must detect boundaries between reasoning steps. We use a combination of heuristics:

•

Sentence Boundaries: Periods, question marks, and exclamation marks followed by whitespace.
•

Structural Markers: Newlines, bullet points, and numbered list items.
•

Thinking Delimiters: Model-specific tokens like <|think|>, </think>, or Step N: patterns.

For models without explicit thinking delimiters, we default to sentence-level granularity, which typically corresponds to individual reasoning steps in chain-of-thought outputs.

B.8 Parallel Streaming Architecture

To achieve real-time attribution without slowing down generation, we employ a producer-consumer architecture that decouples token generation from attribution computation.

Asynchronous Processing.

The main thread (producer) focuses exclusively on autoregressive generation using the VLM backbone. As tokens are generated, the associated attention tensors are pushed into a thread-safe queue. A background worker thread (consumer) continuously pulls these tensors, aggregates them into span-level features ${\bm{f}}_{S,k}$ (Eq. 4), and runs the linear estimator (Eq. 9). Since the Python Global Interpreter Lock (GIL) is released during the heavy CUDA operations of both generation and estimator computation, these two processes run in true parallel on the GPU.

Latency Hiding.

This design completely hides the attribution cost behind the generation latency. While the model generates tokens for reasoning step $t+1$ , the worker computes attributions for step $t$ . Consequently, the visual attribution for a completed thought appears the moment the model begins the next thought, resulting in a system with effectively zero perceptual overhead compared to vanilla generation. By contrast, standard post-hoc methods must wait for the entire generation to finish before starting attribution, adding a delay proportional to the sequence length.

B.9 Reproducibility Checklist

To ensure reproducibility, we provide the following specifications:

Table 4: Key hyperparameters for reproducibility.

Region Unitization
Component	Value
DINOv3 variant	dinov3-vitl16-pretrain-lvd1689m
Input resolution	$224\times 224$
Clustering method	Agglomerative (Ward)
Distance threshold $\tau$	0.5
Typical region count $K$	auto
Estimator Training
Masks per sample $N$	32
Optimizer	AdamW
Learning rate	$1\times 10^{-3}$
LR scheduler	Cosine with warmup (min lr: $1\times 10^{-5}$ )
Weight decay	$1\times 10^{-4}$
Training iterations	2,000
Batch size	512
Evaluation
Top-K for Drop metric	5
Samples per category	400 (train, mix-up dataset), 1000 (test, randomly sampled from non-training samples)

Random Seeds.

We use fixed random seeds for all stochastic components: mask sampling (seed=42), model initialization (seed=42), and train/test splits (seed=42). Results are averaged over 3 random seeds for statistical robustness.

Evaluation Protocol.

For each model-task pair, we:

1.

Generate reasoning traces for the test set using greedy decoding.
2.

Filter to correctly-answered examples only (for training data quality).
3.

Train the estimator on the training split.
4.

Evaluate on held-out test examples using LDS and Top-K Drop metrics.
5.

Report mean and standard deviation across seeds.

Appendix C Semantic Region Unitization Analysis

This section provides additional analysis of our DINOv3-based semantic region unitization approach, including category-specific clustering patterns and quantitative statistics across dataset categories.

C.1 Category-Specific Clustering Patterns

Our agglomerative clustering approach exhibits distinct segmentation behaviors across different dataset categories. The DINOv3 features naturally group semantically coherent regions: objects are separated from backgrounds, text blocks are isolated in document images, and mathematical symbols are distinguished from diagram components.

We observe the following category-specific patterns:

•

General (GQA): Object boundaries are well-preserved, with distinct regions for foreground objects, background elements, and spatial contexts.
•

Document (DocVQA): Text blocks, tables, and graphical elements are cleanly separated, enabling fine-grained attribution to specific document components.
•

Science (ScienceQA): Diagrams, labels, and annotations form separate clusters. This is essential for understanding which visual elements support scientific reasoning.
•

Math (MathVista, MathVision, MathVerse): Geometric shapes, equations, and coordinate systems are partitioned into interpretable regions, though dense symbolic content sometimes leads to over-segmentation.
•

Code (ChartMimic, WebSight): Chart components (axes, legends, data points) and UI elements (buttons, text fields, images) are distinguished effectively.

C.2 Region Count Statistics

The number of semantic regions $K$ varies across images depending on visual complexity. Table˜5 shows the distribution of region counts for each dataset category. We observe:

•

Document and Code images yield the highest region counts (median $\approx$ 68–75), reflecting the dense, structured nature of text-heavy and UI-rich content.
•

Math images show high variance, with simpler geometric problems producing fewer regions and complex multi-part diagrams producing more.
•

General and Science images have moderate region counts (median $\approx$ 45–48), balancing object-level granularity with scene-level coherence.

This adaptive behavior is a key advantage over fixed-grid partitioning methods, which cannot adjust granularity based on image content. The cosine distance threshold ( $\tau=0.5$ in our experiments) provides a consistent semantic criterion across diverse visual domains.

Table 5: Region count statistics across dataset categories. The adaptive DINO-based clustering produces varying numbers of regions depending on image complexity. Document and Code images yield more regions due to dense visual content.

Category	Min	Q1	Median	Q3	Max
General (GQA)	18	32	48	72	105
Document (DocVQA)	24	45	68	95	128
Science (ScienceQA)	16	30	45	65	98
Math (MathVerse, MathVista, MathVision)	16	28	42	62	92
Code (Websight, ChartMimic)	28	52	75	102	128

C.3 Clustering Hyperparameters

We use the following hyperparameters for semantic region unitization:

•

Feature extractor: DINOv3-Large (frozen), patch size 16 $\times$ 16
•

Input resolution: Images are resized to $224\times 224$
•

Clustering algorithm: Agglomerative clustering with Ward linkage
•

Distance metric: Cosine distance ( $1-\text{cosine similarity}$ )
•

Distance threshold: $\tau=0.5$ (determines cluster granularity)
•

Resulting regions: $K\in[16,128]$ depending on image complexity

We found the distance threshold $\tau$ to be the most sensitive hyperparameter. Lower values ( $\tau<0.3$ ) produce too many small regions, making attributions noisy. Higher values ( $\tau>0.7$ ) merge semantically distinct objects, reducing interpretability. The value $\tau=0.5$ provides a good balance across all dataset categories.

Comparison with K-means Clustering.

We also compare agglomerative clustering with K-means clustering using fixed $K$ values. Table˜6 shows that agglomerative clustering with adaptive $K$ outperforms K-means across all fixed $K$ settings, particularly at lower $K$ values where semantic boundaries become critical.

Table 6: Comparison of K-means (fixed

K

) vs. Agglomerative clustering (adaptive

K

) on Qwen3-VL. Results averaged across all categories.

Method	LDS $\uparrow$	Top-5 Drop $\uparrow$
K-means ( $K=16$ )	0.58	0.72
K-means ( $K=32$ )	0.65	0.88
K-means ( $K=64$ )	0.68	0.94
K-means ( $K=128$ )	0.67	0.91
Agglomerative (adaptive)	0.70	1.00

Appendix D Pseudocode

This section provides formal algorithmic descriptions and PyTorch implementations of our amortized attribution framework. We present the training procedure for learning the linear estimator and the inference procedure for real-time attribution streaming.

Appendix E Algorithm and Implementation Details

We provide the formal algorithms for training our amortized estimator and performing real-time inference, followed by their PyTorch implementations.

E.1 Formal Algorithms

Algorithm 1 details the self-supervised training procedure used to learn the estimator $\mathcal{E}_{\theta}$ . The core idea is to train the estimator to rank random subsets of regions based on their impact on the model’s confidence, maximizing the Pearson correlation between predicted and ground-truth ablation effects.

Algorithm 1 Training the Amortized Estimator

1: Input: VLM

\mathcal{M}

, Estimator

\mathcal{E}_{\theta}

, Batch

\mathcal{B}=\{({\bm{x}},{\bm{t}},y,{\mathcal{R}})\}

2: Hyperparameters: Sample size

N=32

, Learning rate

\eta

4: // 1. Compute baseline and attention features

5: Get logits

P(y|{\bm{x}},{\bm{t}})

and attention maps

{\bm{A}}

from

\mathcal{M}({\bm{x}},{\bm{t}})

\log p_{\text{base}}\leftarrow\log P(y|{\bm{x}},{\bm{t}})

8: // 2. Sample random binary masks

9: Sample

N

random binary masks

\{{\bm{b}}^{(i)}\}_{i=1}^{N}

where

{\bm{b}}^{(i)}\in\{0,1\}^{K}

10: // 3. Compute Ground Truth Effect (Self-Supervised)

11: for

i=1

N

12: Compute ablated logits

P(y|{\bm{x}},{\bm{t}},{\bm{b}}^{(i)})

using

\mathcal{M}

with mask

{\bm{b}}^{(i)}

13:

\delta^{(i)}_{\text{true}}\leftarrow\log p_{\text{base}}-\log P(y|{\bm{x}},{\bm{t}},{\bm{b}}^{(i)})

14: end for

15:

\bm{\delta}_{\text{true}}\leftarrow[\delta^{(1)}_{\text{true}},\dots,\delta^{(N)}_{\text{true}}]

16:

17: // 4. Predict Effect using Estimator

18: Compute attribution scores

{\bm{s}}=\mathcal{E}_{\theta}({\bm{A}},{\mathcal{R}})

(See Algorithm 2)

19: Predict subset effects:

\delta^{(i)}_{\text{pred}}\leftarrow\sum_{k=1}^{K}s_{k}\cdot{\bm{b}}^{(i)}_{k}

(Linear Assumption, Equation˜9)

20:

\bm{\delta}_{\text{pred}}\leftarrow[\delta^{(1)}_{\text{pred}},\dots,\delta^{(N)}_{\text{pred}}]

21:

22: // 5. Optimization

23:

\mathcal{L}\leftarrow-\text{PearsonCorr}(\bm{\delta}_{\text{pred}},\bm{\delta}_{\text{true}})

24: Update

\theta\leftarrow\theta-\eta\nabla_{\theta}\mathcal{L}

Algorithm 2 describes the inference process. The trained estimator maps attention patterns directly to attribution scores without requiring multiple forward passes.

Algorithm 2 Real-Time Attribution Inference (Asynchronous)

1: Input: VLM

\mathcal{M}

, Estimator

\mathcal{E}_{\theta}

2: Output: Stream of tokens

y_{t}

and attributions

{\bm{s}}_{S}

4: // Initialize concurrent queues

\mathcal{Q}_{\text{attn}}\leftarrow\text{Queue}()

7: Thread 1: Generation (Producer)

8: for

t=1

T

y_{t},{\bm{A}}_{t}\leftarrow\mathcal{M}(x,y_{<t})

(Generate next token)

10: yield

y_{t}

11:

\mathcal{Q}_{\text{attn}}.\text{push}({\bm{A}}_{t})

12: if

y_{t}

ends span

S

then

13:

\mathcal{Q}_{\text{attn}}.\text{push}(\text{EndOfSpan})

14: end if

15: end for

16:

17: Thread 2: Attribution (Consumer)

18: while generation active do

19:

{\bm{A}}_{\text{buffer}}\leftarrow[\ \ ]

20: while next item in

\mathcal{Q}_{\text{attn}}

is not EndOfSpan do

21:

{\bm{A}}_{\text{buffer}}.\text{append}(\mathcal{Q}_{\text{attn}}.\text{pop}())

22: end while

23:

{\bm{A}}_{S}\leftarrow\text{Aggregate}({\bm{A}}_{\text{buffer}})

(Equation˜4)

24:

{\bm{s}}_{S}\leftarrow\mathcal{E}_{\theta}({\bm{A}}_{S})

(Equation˜9)

25: yield

{\bm{s}}_{S}

(Stream attribution for completed span)

26: end while

E.2 PyTorch Implementation

We provide the corresponding PyTorch implementation for reference. Implementation LABEL:lst:training_code shows the training step, and Implementation LABEL:lst:inference_code shows the amortized attributor module.

⬇

1def train_step(model, estimator, batch, optimizer):

2 """

3 Performs a single training step for the Amortized Estimator.

5 Args:

6 model: The frozen VLM backbone.

7 estimator: The AmortizedAttributor being trained.

8 batch: Dictionary containing ’img’, ’text’, ’regions’, ’target’.

9 """

10 # 1. Clean pass to get baselines and attention features

11 with torch.no_grad():

12 # model returns logits and attentions [B, L, H, T, M]

13 # L=Layers, H=Heads, T=Time steps, M=Visual tokens

14 out = model(batch[’img’], batch[’text’], output_attentions=True)

15 base_logp = out.logits.log_softmax(-1).gather(-1, batch[’target’])

16 attn = out.attentions

18 # 2. Sample random binary masks for K regions (N=32 samples)

19 # masks: [B, N, K] where N is sample size, K is num regions

20 B = batch[’img’].shape[0]

21 K = batch[’regions’].shape[0] # regions: [K, M] (shared across batch)

22 N = 32

23 masks = torch.randint(0, 2, (B, N, K)).to(model.device).float()

25 # 3. Compute Ground Truth Effect (delta_true) via VLM ablation

26 delta_true_list = []

27 with torch.no_grad():

28 for i in range(N):

29 # Forward pass with attention masking applied to regions

30 # mask=masks[:, i] prevents VLM from attending to masked regions

31 out_abl = model(batch[’img’], batch[’text’], mask=masks[:,i])

32 abl_logp = out_abl.logits.log_softmax(-1).gather(-1, batch[’target’])

34 # Delta: Positive value means ablation hurt performance (drop in log-prob)

35 delta_true_list.append(base_logp - abl_logp)

37 delta_true = torch.cat(delta_true_list, dim=1) # [B, N]

39 # 4. Predict Effect (delta_pred) using Estimator

40 # scores: [B, K] (Predicted importance per region)

41 scores = estimator(attn, batch[’regions’], batch[’span_mask’])

43 # Linear assumption: Effect of subset = sum of scores (Eq. 9)

44 # delta_pred: [B, N]

45 delta_pred = torch.einsum(’bk,bnk->bn’, scores, masks)

47 # 5. Loss: Maximize Pearson Correlation

48 # We focus on ranking subsets correctly rather than absolute magnitude regression

49 loss = -pearson_correlation(delta_pred, delta_true)

51 optimizer.zero_grad()

52 loss.backward()

53 optimizer.step()

55 return loss.item()

Implementation 1: PyTorch implementation of the Training Step. The estimator is trained to rank random subsets of regions by maximizing the Pearson correlation between predicted scores and actual ablation drops.

⬇

1import torch

2import torch.nn as nn

4class AmortizedAttributor(nn.Module):

5 def __init__(self, n_layers: int, n_heads: int):

6 super().__init__()

7 # Learnable linear projection W (Eq. 9)

8 # Maps flattened attention features (L*H) to a scalar score

9 self.estimator = nn.Linear(n_layers * n_heads, 1, bias=False)

11 def forward(self, attn_weights, regions, span_mask):

12 """

13 Compute attribution scores for visual regions.

15 Args:

16 attn_weights: [B, L, H, T, M] Attention maps from VLM

17 regions: [K, M] Binary membership masks for K regions

18 span_mask: [T] Binary mask for target time span S

20 Returns:

21 scores: [B, K] Attribution score for each region

22 """

23 # Eq. 4: Temporal Pooling over target span S

24 # Select time steps where span_mask is 1 and average

25 # attn: [B, L, H, M]

26 masked_attn = attn_weights * span_mask[None, None, None, :, None]

27 attn = masked_attn.sum(dim=3) / (span_mask.sum() + 1e-6)

29 # Eq. 4: Regional Pooling over visual regions R_k

30 # Aggregate attention for tokens belonging to each region k

31 # features: [B, K, L, H]

32 features = torch.einsum(’blhm,km->bklh’, attn, regions)

34 # Normalize by region size

35 region_sizes = regions.sum(dim=1).view(1, -1, 1, 1)

36 features = features / (region_sizes + 1e-6)

38 # Flatten layer/head dimensions to create feature vector

39 # features: [B, K, L*H]

40 features = features.flatten(start_dim=2)

42 # Eq. 9: Predict attribution scores via linear projection

43 # scores: [B, K, 1] -> [B, K]

44 scores = self.estimator(features).squeeze(-1)

46 return scores

Implementation 2: PyTorch implementation of the Amortized Attributor. The module efficiently pools attention features and maps them to attribution scores in a single forward pass.

Appendix F Additional Experimental Results

In this section, we provide a more detailed analysis of the experimental results presented in the main text. We focus on cross-model generalization, sensitivity to hyperparameters, and an extended comparison with baseline methods, including computational efficiency.

F.1 Sensitivity Analysis

We investigate the robustness of our approach to the size of the training dataset.

F.1.1 Effect of Training Data Size

Our lightweight estimator is highly data-efficient. We trained the estimator using subsets of our training data ranging from 100 to 10,000 samples.

We observe that the estimator converges rapidly. With just 2,000 training samples (our default setting), the model achieves 98.6% of the performance of the fully trained model (using 10k samples). This low data requirement makes it feasible to train custom estimators for new domains or models in minutes.

Table 7: Effect of training set size on estimator performance.

Training Samples	Relative Performance (%)
100	65.4
500	82.1
1,000	95.3
2,000 (ours)	98.6
5,000	99.1
10,000	100.0

F.2 Baseline Method Details

We provide detailed descriptions of the baseline attribution methods used in our experiments, including their computational procedures and adaptation to our region-based evaluation setting.

F.2.1 Attention-Based Methods

Raw Attention.

The simplest baseline extracts attention weights from the cross-attention layers where text tokens attend to visual tokens. For a given text token at position $t$ , we aggregate attention weights across all layers $\ell$ and heads $h$ :

\text{Attn}_{i}=\frac{1}{L\cdot H}\sum_{\ell=1}^{L}\sum_{h=1}^{H}{\bm{A}}^{(\ell,h)}_{t,i}

(14)

where ${\bm{A}}^{(\ell,h)}_{t,i}$ is the attention weight from position $t$ to vision token $i$ in layer $\ell$ , head $h$ . To obtain region-level scores, we sum over tokens within each region: $\text{Attn}_{k}=\sum_{i\in R_{k}}\text{Attn}_{i}$ .

AttnLRP (Attention-aware Layer-wise Relevance Propagation) (Achtibat et al., 2024).

AttnLRP extends classical LRP to transformer architectures by incorporating attention patterns into the relevance propagation rules. Starting from the output layer with relevance $R^{(L)}_{j}=\delta_{j,t}\cdot\log p(y_{t})$ (where $\delta$ is the Kronecker delta), relevance is propagated backward through each layer:

R^{(\ell-1)}_{i}=\sum_{j}\frac{{\bm{A}}^{(\ell)}_{j,i}\cdot{\bm{V}}^{(\ell)}_{i}\cdot{\bm{W}}^{(\ell)}_{O}}{\sum_{k}{\bm{A}}^{(\ell)}_{j,k}\cdot{\bm{V}}^{(\ell)}_{k}\cdot{\bm{W}}^{(\ell)}_{O}+\epsilon}R^{(\ell)}_{j}

(15)

This propagation rule ensures conservation of relevance (the total relevance is preserved across layers) while accounting for attention-based information routing. AttnLRP requires a full backward pass through the network, making it computationally expensive but more faithful than forward-only methods.

F.2.2 Gradient-Based Methods

InputGrad (Hechtlinger, 2016).

InputGrad computes the gradient of the output logit with respect to input pixels:

\text{InputGrad}_{i}=\left\|\frac{\partial\log p(y_{t})}{\partial{\bm{x}}_{i}}\right\|_{2}

(16)

where ${\bm{x}}_{i}$ represents the pixels corresponding to vision token $i$ . This captures the local sensitivity of the output to input perturbations. While computationally efficient (single backward pass), InputGrad often produces noisy, high-frequency attribution maps that do not align well with semantic regions.

F.2.3 Perturbation-Based Methods

TAM (Token Activation Maps) (Li et al., 2025).

TAM improves upon raw attention by incorporating the norm of value vectors. This captures not just “where” the model attends but “how much” information flows. For each layer $\ell$ and head $h$ :

\text{TAM}^{(\ell,h)}_{i}={\bm{A}}^{(\ell,h)}_{t,i}\cdot\|{\bm{V}}^{(\ell,h)}_{i}\|_{2}

(17)

where ${\bm{V}}^{(\ell,h)}_{i}$ is the value vector for vision token $i$ . The final attribution is aggregated across layers and heads, with later layers typically weighted more heavily. TAM requires access to intermediate activations but avoids gradient computation, making it faster than gradient-based methods.

F.2.4 Adaptation to Region-Based Evaluation

All baseline methods produce token-level or pixel-level attribution scores. To ensure fair comparison at the same granularity as our method, we aggregate these scores to region level using our DINO-based semantic unitization:

\text{Score}_{k}=\sum_{i\in R_{k}}\text{Score}_{i}

(18)

This aggregation ensures that all methods operate on the same semantic units, isolating the effect of the attribution algorithm from the effect of region definition.

F.3 Extended Baseline Comparisons

We compare our method against computationally intensive baselines that are typically too slow for real-time applications but serve as reference points for attribution quality. Specifically, we compare against:

•

Feature Ablation: Systematically removing visual tokens and measuring output changes.
•

Attention Rollout: Heuristic aggregation of raw attention weights across layers.
•

LibraGrad (Mehri et al., 2025): A gradient-based method that corrects gradient flow imbalances in Transformers through backward path pruning and scaling.

Table˜8 highlights the critical advantage of our approach: speed. While feature ablation offers high fidelity, it requires multiple forward passes per token generated. Our method requires a single, lightweight forward pass of the linear estimator, adding negligible overhead ( $\sim 2$ ms/token).

Table 8: Runtime and performance comparison. Latency is measured in milliseconds per token on an NVIDIA A100 GPU. Results averaged across all models and categories. Speedup is relative to Feature Ablation.

Method	LDS $\uparrow$	Top-5 Drop $\uparrow$	Latency (ms/tok)	Speedup vs Ablation
Random	0.30	0.13	0.0	N/A
Attention Rollout	0.41	0.32	0.5	$300\times$
LibraGrad	0.66	0.85	12.0	$12.5\times$
Feature Ablation	0.72	1.02	150.0	$1\times$ (reference)
vStream (Ours)	0.70	1.00	2.1	$\approx 71\times$

Our method achieves performance comparable to the expensive feature ablation baseline (within 3% LDS and 2% Top-5 Drop) while being $71\times$ faster than feature ablation. Note that in the main paper (Table˜1), we report up to $117\times$ speedup over gradient-based methods (e.g., InputGrad), which have higher latency than feature ablation in our region-based setting.

F.3.1 Attention Backend Compatibility

Our method, vStream, is fully compatible with modern attention backends, including PyTorch SDPA (Scaled Dot-Product Attention) and FlashAttention. While our default configuration utilizes the KV cache for maximum efficiency, disabling the KV cache (requiring full attention recomputation at each step) results in an approximately $4\times$ increase in latency. However, even in this worst-case scenario, the latency remains under 0.01 seconds per token, preserving a substantial speedup ( $\approx 18\times$ ) over feature ablation. Table˜9 details the latency measurements across different configurations.

Table 9: Latency under different attention backends and KV-cache configurations. We measure the per-token latency of our attribution method compared to feature ablation. Even without KV caching, our method remains computationally efficient.

Configuration	KV Cache	Latency (sec/token) $\downarrow$	Speedup vs Ablation $\uparrow$
Ours (SDPA + Flash)	Enabled	0.002	71 $\times$
Ours (SDPA + Flash)	Disabled	$\approx 0.008$	$\approx 18\times$
Ours (SDPA math)	Enabled	$\approx 0.004$	$\approx 43\times$
Feature Ablation	-	0.150	1 $\times$

F.4 Cross-Model Generalization

We evaluate whether our estimator generalizes across different VLM architectures. Tables˜10, 11 and 12 present cross-task generalization results for GLM-4.1V-9B-Thinking, MiMo-VL-7B, and Cosmos-R1, respectively, complementing the Qwen3-VL results in the main paper (Table˜2). Across all models, we observe consistent patterns: (1) in-domain performance (diagonal) ranges from 0.65–0.75 LDS; (2) Math and Science show strong mutual transfer due to shared diagram structures; (3) Document tasks exhibit weaker transfer from other domains due to distinct visual layouts; and (4) training on a mixture of all categories (Mix-up) recovers full performance, suggesting a single estimator suffices for diverse applications.

Table 10: Cross-task generalization for GLM-4.1V-9B-Thinking. Each cell shows (LDS / Top-5 Drop). Diagonal entries (shaded) indicate in-domain performance.

Train	Eval Category					Avg
Train	Math	Science	Doc	Code	General	Avg
Math	0.69/0.78	0.61/0.65	0.52/0.46	0.57/0.55	0.54/0.48	0.59/0.58
Science	0.60/0.63	0.74/0.82	0.56/0.52	0.53/0.48	0.58/0.56	0.60/0.60
Document	0.51/0.48	0.54/0.50	0.67/0.75	0.55/0.52	0.63/0.70	0.58/0.59
Code	0.57/0.58	0.52/0.46	0.54/0.50	0.68/0.80	0.60/0.64	0.58/0.60
General	0.55/0.52	0.59/0.62	0.62/0.68	0.58/0.60	0.73/0.84	0.61/0.65
Mix-up	0.69/0.97	0.74/0.89	0.67/1.01	0.68/1.10	0.73/0.95	0.70/0.98

Table 11: Cross-task generalization for MiMo-VL-7B. Each cell shows (LDS / Top-5 Drop).

Train	Eval Category					Avg
Train	Math	Science	Doc	Code	General	Avg
Math	0.75/0.85	0.58/0.62	0.51/0.44	0.59/0.56	0.52/0.46	0.59/0.59
Science	0.57/0.60	0.65/0.74	0.54/0.50	0.51/0.46	0.56/0.54	0.57/0.57
Document	0.50/0.46	0.52/0.48	0.71/0.78	0.53/0.50	0.61/0.68	0.57/0.58
Code	0.55/0.56	0.50/0.44	0.52/0.48	0.70/0.82	0.58/0.62	0.57/0.58
General	0.53/0.50	0.57/0.60	0.60/0.66	0.56/0.58	0.70/0.80	0.59/0.63
Mix-up	0.74/1.08	0.64/0.97	0.70/0.98	0.69/0.96	0.68/1.10	0.69/1.02

Table 12: Cross-task generalization for Cosmos-R1. Each cell shows (LDS / Top-5 Drop).

Train	Eval Category					Avg
Train	Math	Science	Doc	Code	General	Avg
Math	0.74/0.80	0.60/0.64	0.50/0.43	0.58/0.54	0.53/0.47	0.59/0.58
Science	0.59/0.62	0.67/0.76	0.55/0.51	0.52/0.47	0.57/0.55	0.58/0.58
Document	0.49/0.45	0.53/0.49	0.68/0.74	0.54/0.51	0.62/0.69	0.57/0.58
Code	0.56/0.57	0.51/0.45	0.53/0.49	0.71/0.84	0.59/0.63	0.58/0.60
General	0.54/0.51	0.58/0.61	0.61/0.67	0.57/0.59	0.69/0.78	0.60/0.63
Mix-up	0.72/0.99	0.65/0.90	0.66/1.01	0.70/1.04	0.67/1.05	0.68/1.00

F.5 Vision Backbone Comparison

We compare different vision foundation models for semantic region unitization. In addition to DINOv3 (our default), we evaluate CLIP ViT-L/14 (Radford et al., 2021) and SigLIP ViT-SO400M (Zhai et al., 2023).

Table 13: Comparison of vision backbones for semantic region unitization. Results on Qwen3-VL across all five categories. DINOv3-L consistently outperforms supervised alternatives.

	Math		Science		Document		Code		General		Average
Backbone	LDS	Top-5	LDS	Top-5	LDS	Top-5	LDS	Top-5	LDS	Top-5	LDS	Top-5
CLIP ViT-L/14	0.68	0.89	0.64	0.78	0.62	0.85	0.60	0.76	0.61	0.80	0.63	0.82
SigLIP ViT-SO400M	0.70	0.93	0.66	0.82	0.64	0.90	0.63	0.81	0.64	0.85	0.65	0.86
DINOv3-L	0.76	1.05	0.74	0.89	0.70	1.01	0.68	0.91	0.73	0.95	0.72	0.96

DINOv3 outperforms both CLIP and SigLIP by 7–9% in LDS and 10–15% in Top-5 Drop across all categories. We attribute this to DINOv3’s self-supervised training objective, which emphasizes local feature correspondence and produces sharper object boundaries compared to contrastive language-image pretraining. This finding suggests that attribution quality depends critically on the semantic coherence of the underlying region partition.

F.6 Extended Top-K Drop Analysis

The main paper reports Top-5 Drop. Here we provide extended results for Top-1 and Top-3 Drop (Tables˜14 and 15), which test whether our method correctly identifies the single most important region and the top few regions, respectively. vStream achieves best or second-best performance across most K values and categories. The improvement is most pronounced for Top-1 Drop in Document tasks, where correctly identifying the single most relevant text block or table is crucial for understanding model behavior.

Table 14: Top-K Drop comparison across K values on Qwen3-VL. Higher values indicate better identification of causally important regions.

	Math			Science
Method	Top-1	Top-3	Top-5	Top-1	Top-3	Top-5
Random	0.02	0.05	0.08	0.03	0.07	0.11
Attention	0.08	0.19	0.31	0.07	0.17	0.28
InputGrad	0.22	0.48	0.72	0.21	0.45	0.68
AttnLRP	0.26	0.54	0.81	0.32	0.71	1.08
TAM	0.31	0.68	1.02	0.29	0.63	0.95
vStream	0.33	0.72	1.05	0.28	0.61	0.92

Table 15: Top-K Drop comparison (continued) for Document, Code, and General categories.

	Document			Code			General
Method	Top-1	Top-3	Top-5	Top-1	Top-3	Top-5	Top-1	Top-3	Top-5
Random	0.04	0.09	0.15	0.02	0.06	0.09	0.02	0.05	0.07
Attention	0.09	0.21	0.33	0.08	0.18	0.29	0.07	0.16	0.26
InputGrad	0.21	0.46	0.71	0.28	0.62	0.96	0.23	0.50	0.75
AttnLRP	0.24	0.51	0.78	0.25	0.54	0.82	0.30	0.65	0.98
TAM	0.31	0.68	1.04	0.33	0.72	1.06	0.27	0.59	0.89
vStream	0.34	0.73	1.09	0.28	0.61	0.91	0.31	0.68	1.02

F.7 Reasoning Trajectory Analysis: Extended Results

We provide extended statistics and visualizations for the reasoning trajectory analysis introduced in Section˜4.4.

F.7.1 Quantitative Metrics

We compute two geometric metrics for each reasoning trajectory:

•

Path Length: Total Euclidean distance traveled in the PCA-projected attribution space across all reasoning steps.
•

Tortuosity: Ratio of path length to net displacement (start-to-end distance). A value of 1.0 indicates a straight path; higher values indicate more wandering.

Table 16: Trajectory statistics across models and outcome types. Successful reasoning exhibits shorter, less tortuous paths in attribution space. Values show mean

\pm

std across

n=1500

samples per group.

	Path Length $\downarrow$		Tortuosity $\downarrow$
Model	Success	Failure	Success	Failure
Qwen3-VL	$0.003\pm 0.002$	$0.006\pm 0.005$	$13.7\pm 12.0$	$25.4\pm 26.1$
GLM-4.1V	$0.004\pm 0.003$	$0.007\pm 0.006$	$14.8\pm 13.5$	$28.3\pm 29.7$
MiMo-VL	$0.003\pm 0.002$	$0.005\pm 0.004$	$11.9\pm 10.8$	$22.7\pm 24.2$
Cosmos-R1	$0.004\pm 0.003$	$0.006\pm 0.005$	$15.2\pm 14.1$	$27.9\pm 28.5$

Across all four models, successful reasoning chains exhibit approximately 50% shorter path lengths and 40–50% lower tortuosity compared to unsuccessful chains. This suggests that valid reasoning corresponds to a more stable, directed traversal of the visual attribution manifold.

F.7.2 Interpretation

We interpret these geometric differences through the lens of “hypothesis switching.” Unsuccessful chains frequently reassign visual attention to different regions mid-reasoning, manifesting as erratic, high-tortuosity trajectories in attribution space. In contrast, successful chains quickly commit to a consistent set of visual evidence and maintain stable attention throughout the reasoning process.

This observation has practical implications: trajectory metrics could potentially serve as early warning signals for hallucination detection, flagging reasoning chains that exhibit unusually high path length or tortuosity before the final answer is generated.

F.7.3 3-Way Error Analysis: Wandering vs. Fixation

We decompose unsuccessful POPE examples ( $n{=}3{,}000$ ) into Reasoning Failures (RF; incorrect answer, plausible object) and Hallucinations (H; object not in image). Table˜17 reports the full metric breakdown.

Table 17: 3-way trajectory metrics on POPE (

n{=}3{,}000

). S = Success, RF = Reasoning Failure, H = Hallucination. Path geometry separates errors from success; concentration separates Fixation (H) from Wandering (RF).

Metric	S	RF	H	KW $p$	$d$ (S–H)
Path Length	$0.004\pm 0.002$	$0.006\pm 0.005$	$0.005\pm 0.003$	$<.001$	$-0.73$
Tortuosity	$5.6\pm 3.9$	$9.4\pm 10.7$	$8.0\pm 5.0$	$<.001$	$-0.60$
Concentration	$0.209\pm 0.054$	$0.196\pm 0.058$	$0.228\pm 0.059$	$<.001$	$-0.34$

To test whether the Fixation pattern extends beyond binary POPE judgments to open-ended captioning, we compute per-caption concentration on $2{,}000$ COCO captions and regress against CHAIR_i (Rohrbach et al., 2018), a standard object-hallucination score for captions.

F.8 Per-Step Attribution Fidelity

Per-step attribution fidelity results are reported in the main text (Figure˜12). Correct reasoning chains maintain stable $R^{2}$ throughout generation, while incorrect chains degrade at roughly $20\%$ of reasoning elapsed, corroborating the early tortuosity signal in Figure˜10.

F.9 Estimator Architecture: Linear vs. MLP

We compare the linear estimator used by vStream against MLP variants of increasing width on Qwen3-VL (3 seeds each).

Table 18: Linear vs. MLP estimator (Qwen3-VL, 3 seeds). MLP variants yield diminishing returns despite

>100\times

more parameters.

Estimator	Params	$R^{2}$	Pearson $\rho$	Latency (ms)
Linear	1,152	$0.65\pm.01$	$0.81\pm.01$	0.03
MLP-64	73,856	$0.66\pm.01$	$0.82\pm.01$	0.12
MLP-128	147,712	$0.67\pm.01$	$0.82\pm.00$	0.15
MLP-256	295,424	$0.67\pm.01$	$0.82\pm.01$	0.18

The linear estimator achieves $R^{2}{=}0.65$ with 1,152 parameters and 0.03 ms inference latency. MLP-256 gains $+0.02$ in $R^{2}$ (reaching $0.67$ ) but requires $256{\times}$ more parameters and $6{\times}$ higher latency. The linear estimator is therefore the preferred choice for streaming deployment.

F.10 Cross-Architecture Estimator Weight Patterns

F.11 Context-Length Robustness

F.12 Out-of-Distribution Generalization: VQA-RAD

We test whether the estimator trained exclusively on natural images generalizes to medical radiology images (VQA-RAD (Lau et al., 2018)) without retraining.

Table 19: Attribution fidelity on VQA-RAD (OOD). vStream estimator trained on natural images only, applied without retraining. Format: LDS / Top-5 Drop; higher is better.

Method	LDS	Top-5 Drop
Random	0.29	0.14
Attention	0.41	0.32
InputGrad	0.58	0.71
AttnLRP	0.66	0.79
TAM	0.63	0.88
vStream (Ours)	0.67	0.84

Despite the large domain shift, vStream achieves competitive fidelity with no domain adaptation, demonstrating that the learned attention-to-ablation mapping transfers across image modalities.

F.13 Segmentation Evaluation

While our primary goal is visual attribution rather than semantic segmentation, we evaluate how well our attribution maps align with ground-truth object masks. We use three benchmarks: ImageNet-Segmentation (Guillaumin et al., 2014), which provides pixel-level annotations for a subset of ImageNet validation images; COCO (Lin et al., 2014), with instance segmentation masks; and the RefCOCO family of datasets (Kazemzadeh et al., 2014) for referring expression grounding. For RefCOCO evaluation, we report results on the val split, which is the standard evaluation protocol. The RefCOCO family includes three variants: RefCOCO (general expressions), RefCOCO+ (excludes absolute location terms like “left” or “right”), and RefCOCOg (longer, more complex expressions). We use RefCOCO-val as it provides a balanced evaluation across diverse object categories. Note that attribution methods naturally highlight causally relevant regions rather than full object extents, making segmentation a challenging proxy task.

Table 20: Segmentation performance (mIoU) on Qwen3-VL across three benchmarks. For RefCOCO, we report results on the val split. All methods achieve comparable scores, confirming that attribution maps do not directly correspond to object boundaries.

Method	ImageNet-Seg	COCO	RefCOCO-val
Attention	0.21	0.19	0.23
InputGrad	0.23	0.21	0.25
AttnLRP	0.27	0.24	0.27
TAM	0.27	0.25	0.27
vStream (Ours)	0.27	0.24	0.27

The results indicate that all methods achieve comparable mIoU scores in the 0.20–0.30 range, which is typical for attribution-to-segmentation evaluation. This confirms that attribution maps, designed to highlight causally relevant evidence, do not necessarily align with full object boundaries.

Appendix G Qualitative Analysis

We provide additional qualitative results in Figure˜23–Figure˜38. These figures show step-by-step visual attribution results from our main model (Qwen3-VL) and all baseline methods across all five task categories: Math, Science, Document, Code, and General.

Each figure displays the attribution heatmap for individual reasoning steps, enabling direct comparison between vStream and baseline attribution methods. The visualizations demonstrate that vStream produces focused, semantically coherent attribution maps that track the model’s reasoning process in real-time.

G.1 Failure Analysis

Despite strong performance, we identify distinct failure modes where attribution remains challenging. These failure cases are not directly visualized in the figures but are described below based on our analysis.

Ambiguity in Visual References.

When the visual input contains multiple identical objects (e.g., “count the red apples”), the attribution map often splits intensity across all candidates simultaneously rather than sequentially focusing on individual instances. While strictly “correct” in terms of feature matching, this does not reflect the sequential nature of human counting and can make traces harder to interpret.

Hallucination and Disconnected Reasoning.

When the model generates plausible text that is not grounded in the image (hallucination), our attribution maps often become diffuse or uniform, lacking a clear focal point. This suggests that diffuse attribution could serve as a detector for hallucinated content, an avenue for future research.

Dense Text and Small Objects.

For images containing dense text (documents, code screenshots) or small objects, the semantic regions from DINO clustering may not perfectly isolate individual characters or tiny elements. In these cases, attribution may highlight a region containing the relevant element but lack sub-region precision.

Abstract Reasoning Steps.

During purely symbolic computation steps (e.g., “Therefore, 2x + 3 = 7 implies x = 2”), visual attribution naturally becomes weak because the reasoning genuinely does not depend on visual input. Our method correctly produces low attribution scores in these cases. Users should interpret weak attribution during computational steps as expected behavior rather than a failure mode.

Appendix H Broader Impact and Limitations

H.1 Broader Impact

Advancing Trustworthy AI. The primary contribution of this work is to enhance the transparency and interpretability of Vision-Language Models (VLMs). As these models are increasingly deployed in high-stakes domains—such as medical diagnosis, autonomous navigation, and legal analysis—the ability to verify why a model made a decision is paramount. By providing fast and accurate attribution maps, our method allows human operators to verify that model reasoning relies on relevant visual evidence rather than spurious correlations or hallucinations. This is a critical step towards safe deployment of reasoning models.

Potential Risks. While intended to reveal model reasoning, interpretability tools can be double-edged. There is a risk that accurate-looking attribution maps could be used to generate convincing justifications for incorrect or biased model decisions, potentially leading users to over-trust a flawed system. Furthermore, if the estimator itself is adversarially manipulated, it could hide the model’s reliance on sensitive or protected attributes (e.g., race or gender) in decision-making processes. It is crucial that these tools are used as part of a holistic auditing framework, not as a standalone guarantee of safety.

H.2 Limitations

We identify several limitations of our approach that should be considered when applying vStream in practice.

Linearity Assumption. Our method employs a linear estimator to predict the causal effect of feature ablation. While our empirical results suggest that linear directions in the activation space of modern transformers capture significant causal information, this is a simplifying assumption. Complex, non-linear interactions between visual features (where the suppression of one feature only matters if another is also present) may not be fully captured by our current formulation. Future work could explore non-linear estimators (e.g., MLPs or attention-based predictors) for the attribution head, though this may trade off interpretability and training stability.

Dependency on Visual Backbone (DINO). Our approach relies on the quality of the underlying visual representations (specifically DINOv3) to define semantic regions for attribution. If the visual encoder fails to semantically separate relevant objects or concepts, or if the feature resolution is too coarse, our estimator cannot recover precise attributions. This dependency means:

•

For images with unusual visual content not well-represented in DINO’s training data, region quality may degrade.
•

Very small objects or fine-grained text may not be isolated into separate regions.
•

The computational cost of running DINOv3 adds approximately 45ms per image.

Training Data Quality. The estimator is trained only on examples where the VLM produces correct final answers. This design choice ensures we learn attention patterns associated with successful reasoning. However, it also means:

•

The estimator may be less calibrated for incorrect reasoning traces.
•

Attribution quality depends on having sufficient correctly-answered examples in each domain.
•

For very challenging tasks where models rarely succeed, collecting training data becomes difficult.

Generalization Across Domains. While our cross-task experiments show reasonable transfer between related domains (e.g., Math $\leftrightarrow$ Science), transfer to visually distinct domains (e.g., Math $\rightarrow$ Document) is weaker. This suggests that practitioners may need to collect domain-specific training data for optimal performance in specialized applications.

Attention as a Proxy for Information Flow. Our method assumes that attention weights are informative proxies for information flow in transformers. However, recent work has shown that attention can be manipulated without affecting outputs (Jain and Wallace, 2019), and that alternative pathways (e.g., residual connections, MLP layers) can route information independently of attention. Our empirical success suggests attention captures sufficient signal for attribution, but it may not be a complete picture of the model’s information processing.

Computational Requirements for Ground-Truth Collection. While inference is fast, collecting ground-truth ablation effects for training requires multiple forward passes per example. For a training set of 500 examples with 32 masks each, this amounts to 16,000 forward passes. On an H100 GPU, this takes approximately 2–4 hours per model-task pair.

Model-Specific Training Requirement. Our estimator must be trained separately for each VLM, as the learned linear projection is inherently tied to the specific model’s internal representation space. This limitation is fundamental rather than incidental: different VLMs employ distinct architectures, attention mechanisms, and learned feature spaces, making cross-model transfer of the estimator impractical without retraining. While this per-model training requirement adds deployment overhead, we note that (1) the training cost is modest (2–4 hours on a single GPU per model-task pair), and (2) this constraint is shared by virtually all model-specific interpretability methods that operate on internal representations. We view the development of architecture-agnostic attribution features as an important direction for future work, as discussed in Section˜H.3.

H.3 Future Directions

Several promising directions could extend and improve upon our work. On the methodological side, exploring non-linear estimators such as attention-based or MLP-based architectures could capture complex feature interactions that our linear model may miss, potentially improving attribution quality on images with intricate visual relationships. Developing architecture-agnostic features would enable zero-shot cross-model transfer, allowing practitioners to train an estimator on one VLM and apply it to another without retraining. This would dramatically reduce the barrier to deploying attribution tools across the rapidly evolving landscape of vision-language models.

On the application side, our observation that hallucinated content produces diffuse attribution maps suggests a natural extension toward automatic hallucination detection, where attribution entropy or concentration metrics could serve as early warning signals for unreliable model outputs. Attribution maps could also guide targeted interventions for model editing, enabling surgical corrections to model behavior without full retraining. Finally, extending our framework to video understanding and multi-image reasoning would address the growing importance of temporal and multi-view visual reasoning, where tracking visual attribution across frames could reveal how models integrate information over time.

[Uncaptioned image] — Figure 18: Reasoning trajectory dynamics for Qwen3-VL-8B-Thinking. Visual attribution trajectories projected into PCA space for successful (left) and unsuccessful (right) reasoning chains.