FROM SCORES TO SAMPLES
From Scores to Samples: Elastic Forcing
for Autoregressive Video Generation
01 / THE CORE IDEA
Elastic Forcing post-trains few-step autoregressive video generators directly against reference videos. Frozen video representations make the two distributions comparable; maximum mean discrepancy (MMD) supplies the training signal.
Distribution Matching Distillation uses a diffusion teacher and an online fake-score model alongside the generator. Those auxiliary models add memory and computation, and the teacher defines the supervision.
Match generated and reference video features directly. Only the generator is optimized during post-training; feature encoders and precomputed reference statistics stay frozen.
This change applies to distributional post-training. The standard experiments still start from pretrained, ODE-initialized checkpoints; it is not a claim of training from scratch without a pretrained model.
02 / HOW IT WORKS
Reliable distribution matching needs many video samples. Elastic Forcing separates the statistics needed to estimate the loss from the computation needed to differentiate it.
V-JEPA 2 captures predictive temporal features. VideoMAE adds spatiotemporal structure. The three-encoder variant also uses DINOv3 for frame-level semantics.
MMD compares the generated and reference distributions within each frozen feature space. The reference–reference term is constant with respect to the generator.
Objective · Section 3.1 ↗Generated–generated interactions + generated–reference interactions
A persistent Nyström summary incorporates the full reference collection without comparing every query against every video. Its finite rank can introduce approximation bias.
Monte Carlo estimates from sampled references are unbiased but noisy. Blending the two balances approximation bias and sampling variance.
Hybrid estimator · Section 3.2 ↗One hybrid reference estimate · Equation 9
Evaluate the MMD interactions across a large population of fresh rollouts in feature space. Then replay a uniformly sampled subset to propagate the feature gradients through the generator.
Rescaling by B/b gives a conditionally unbiased estimate of the evaluated full-batch gradient. Sparse boundary states make replay practical without retaining every rollout graph.
Selective differentiation · Section 3.3 ↗Only the generator is updated.
03 / RESULTS
1.3B, two encoders
vs. 83.80 for Self-Forcing
1.3B comparison
as reported in Table 1
80 updates · 23.2 hours
5 denoising steps per chunk
| Method | Total ↑ | Quality ↑ | Semantic ↑ | FPS ↑ |
|---|---|---|---|---|
| CausVid | 82.88 | 83.93 | 78.69 | 17.0 |
| Self-Forcing | 83.80 | 84.59 | 80.64 | 17.0 |
| Elastic Forcing 3 encoders | 84.25 | 85.06 | 80.99 | 17.0 |
| Elastic Forcing 2 encoders | 84.64 | 85.43 | 81.48 | 17.0 |
The Elastic Forcing / Self-Forcing comparison uses matched architecture and ODE initialization. Evaluation uses expanded versions of 946 VBench prompts, with five seeds per prompt. Two encoders: V-JEPA 2 + VideoMAE; three encoders: additionally DINOv3. Table 1 and evaluation details ↗
Removing the diffusion teacher and fake-score model makes 14B post-training possible on a single node of eight H200 GPUs. Different offloading strategies trade GPU-hours against peak memory.
In the paper’s 14B comparison, Elastic Forcing achieves a VLM Total score of 4.217 versus 4.115 for Krea Realtime. The 42-participant human evaluation reports mean ratings of 3.354 and 3.054, respectively.
Human results are average ratings, not preference percentages. The 14B experiment uses Krea’s initialization and is separate from the 1.3B VBench comparison.
04 / BEYOND THE TEACHER
Change the reference videos to change what the generator learns—without first adapting a target-specific diffusion teacher.
Wide-angle references encourage panoramic composition.
Character-specific videos teach the appearance of Nailoong.
Black-and-white references lead to monochrome generations.
05 / THE VIDEO COLLECTION
Characters. Worlds. Motion.
21 video examples.
READ THE PAPER
Elastic Forcing for Autoregressive Video Generation