SSD: Spatially Speculative Decoding
Accelerates Autoregressive Image Generation

Shilong Xiang, Zirui Zhang, Lijun Yu*, Chengzhi Mao*
Rutgers University
*Equal advising
arXiv Code BibTeX

One Forward-Pass Budget, Six Decoders

Same prompt, same seed, same Janus-Pro-7B backbone. Every panel is advanced by the same number of backbone forward evaluations (NFE), so at any instant the row shows what each decoder has produced for an identical compute budget.

Animation NFE-aligned decoding on Janus-Pro-7B. Ungenerated tokens are shown as flat noise; drafted-but-unverified blocks are outlined in amber. On this prompt SSD completes the 576-token image in 80 forward passes, while the strongest prior accelerators need 156–504 and plain AR needs 576. SSD is already past the halfway mark at NFE 38, when AR has committed 38 of 576 tokens.
DPG-Bench — the same six methods on all three backbones, averaged over the full benchmark.
Method Latency ↓ Step ↑ Latency ↑ DPG Overall ↑
Janus-Pro (7B)
Autoregressive7.87s1.00×1.00×84.23
SJD5.25s1.81×1.50×84.01
GSD3.31s3.07×2.38×82.97
LANTERN9.32s1.17×0.84×84.07
ZipAR3.04s2.76×2.59×83.78
SSD (Ours)1.38s7.20×5.70×83.40
Lumina-mGPT (7B)
Autoregressive96.23s1.00×1.00×76.30
SJD61.08s2.23×1.58×74.03
GSD38.23s2.89×2.52×73.37
LANTERN59.00s2.00×1.63×74.45
ZipAR36.45s2.94×2.64×74.62
SSD (Ours)13.09s8.40×7.35×74.57
Emu3 (8B)
Autoregressive281.87s1.00×1.00×78.69
SJD260.41s2.24×1.08×78.38
GSD97.07s3.46×2.90×77.57
LANTERN182.50s2.16×1.54×76.12
ZipAR79.71s5.41×3.54×73.36
SSD (Ours)25.55s14.34×11.03×76.03
Accelerating Autoregressive Vision via 2D Spatial Anticipation
Figure 1 Accelerating Autoregressive Vision via 2D Spatial Anticipation. (a) Standard AR flattens the visual world into a 1D sequence and predicts one token per backbone forward evaluation. (b) Speculative Decoding accelerates generation locally but remains constrained by this linear raster-scan geometry. (c) Our SSD aligns the predictive objective with the intrinsic geometry of images and drafts entire spatial blocks in parallel. This reduces the number of backbone forward evaluations and alleviates the memory bottleneck of autoregressive decoding. Right: Applied to Emu3 (8B), this geometric shift yields a 11.03× speedup while preserving high-resolution visual fidelity.

Abstract

Autoregressive image models treat images as 1D token sequences, inheriting the next-token factorization of language models. This flattening discards a useful property of images: nearby tokens are correlated in two dimensions, not one.

We introduce Spatially Speculative Decoding (SSD), an inference-time decoding framework that exploits this spatial structure. Rather than speculating only along the flattened sequence, SSD predicts both the adjacent horizontal token and the token directly below it, allowing multiple spatial directions to advance in parallel. This reduces the number of backbone forward evaluations and alleviates the memory bottleneck of autoregressive decoding.

SSD accelerates image generation by up to 11.03× in wall-clock time while maintaining generation quality on DPG-Bench and GenEval. These results show that spatial structure provides a simple and effective source of parallelism for autoregressive image generation.

11.03×
Wall-clock Speedup
14.34×
Fewer Forward Passes
2D
Spatial Drafting

Motivation: Predictive Dependency is 2D

The Two-Dimensional Nature of Predictive Dependency
Figure 2 The Two-Dimensional Nature of Predictive Dependency. To demonstrate that spatial correlations are inherently 2D, we corrupt the sequential context during Janus-Pro-7B generation by replacing the second half of each row with random tokens (red outlines). Despite this severe disruption to the 1D sequence, visual coherence is preserved wherever the token directly above was accurately generated (blue outlines). This confirms that vertical prediction relies fundamentally on spatial adjacency rather than position in the flattened raster-scan order. (Right) Acceptance rates of horizontal vs. vertical drafting heads at matching spatial offsets confirm that predictability is governed by 2D spatial locality.

Results

MTP drafting cost on Lumina-mGPT-7B
Token acceleration through parallel verification

Why spatial drafting pays off, measured on Lumina-mGPT-7B (48×48 grid) and normalized by one AR step. (Left) The drafting heads are cheap: even at 240 drafted tokens the overhead stays below 0.1 AR steps. (Right) Verification is nearly free per token: wall-clock speedup over sequential AR scales near-linearly with the number of simultaneously verified tokens K, reaching 205× at K = 240, because parameter loading — not arithmetic — dominates under the memory wall.

Qualitative results
Qualitative Qualitative results. Side-by-side comparison of the AR baseline and SSD across three models. Our method yields up to 11.03× speedup while preserving strong visual quality.

BibTeX

@article{xiang2026ssd,
  title={SSD: Spatially Speculative Decoding Accelerates Autoregressive Image Generation},
  author={Xiang, Shilong and Zhang, Zirui and Yu, Lijun and Mao, Chengzhi},
  journal={arXiv preprint arXiv:2606.20543},
  year={2026}
}