Non-Autoregressive Language Models for Fast & Flexible Text Generation
A workshop on language generation beyond next-token prediction — spanning diffusion, flow matching, and any-order autoregression.
Announcements
- Submissions are open: the OpenReview portal is live, and we are recruiting reviewers — sign up to volunteer.
- First invited talk announced: Shansan Gong (University of Hong Kong) on flexible generation order in diffusion language models. See the program.
- NonAR-LM is confirmed as an official workshop at COLM 2026 in San Francisco.
Language generation beyond next-token prediction
Autoregressive next-token prediction has long been the dominant paradigm for language modeling, thanks to its simplicity, scalability, and strong empirical performance. Yet the left-to-right factorization imposes constraints that limit efficiency, controllability, and global coherence.
Recent advances in non-autoregressive modeling offer a fundamentally different approach to discrete sequence generation. Instead of committing to a fixed left-to-right order, these models enable parallel decoding, generate tokens in any order, and can revise earlier decisions. They span masked and uniform-state diffusion, discrete flow matching, and any-order autoregression, and are now competitive at scale and increasingly deployed in industry systems. This workshop brings the community together around three core challenges:
Sequential decoding bottleneck
Tokens are generated one at a time, preventing parallelism across the sequence and leaving hardware underutilized.
Limited controllability
Conditioning on global constraints or future tokens is indirect, often needing complex prompting, rejection sampling, or constrained decoding.
Limited global consistency
Local, token-level decisions can drift into incoherence over long horizons, since the model cannot revise earlier outputs.
Call for contributed work
We invite contributions on training and/or inference of non-autoregressive language models — including diffusion, flow matching, and any-order autoregression.
Modeling & Training
New model classes and training objectives — discrete diffusion, uniform-state diffusion, flow-based, and any-order approaches.
Inference & Sampling
Inference-time algorithms: iterative refinement, parallel decoding, controllable and constrained generation, planning and correction.
Evaluation & Efficiency
Evaluation beyond left-to-right likelihood, plus parallel generation, latency-constrained inference, and systems for scaling.
Applications
Applications across language, code, and biological sequences — including comparative studies of when iterative models help.
Submit your work
Submissions may present new results, works in progress, negative results, empirical evaluations, or forward-looking position papers relevant to the workshop themes.
- Up to 9 pages, excluding references and an optional appendix; shorter submissions are equally welcome.
- Use the COLM 2026 or NeurIPS 2026 template. Format submissions with the official COLM 2026 or NeurIPS 2026 LaTeX style files, submission mode.
- Non-archival. Submitting does not preclude publishing elsewhere.
- Double-blind review. Each submission receives at least three reviews.
- Four contributed oral talks selected from submissions; all accepted work is presented as posters.
- Submissions due June 30, 2026; notifications by July 24, 2026.
Join the review committee
We are recruiting reviewers to help evaluate submissions and shape the program. Volunteering is a hands-on way to engage closely with the newest work in the field.
- Light load. Reviewers handle only a small number of non-archival submissions.
- Reviewing in July 2026. Submissions close June 30 and notifications go out July 24, so reviews fall in early through mid-July.
- Who should sign up. Students and early-career researchers working on or curious about non-autoregressive, diffusion, flow-based, or any-order generation.
- Why it matters. Reviews directly inform which submissions are accepted and selected for oral presentation.
Schedule
A full day of 6 invited talks, 4 contributed oral talks, 2 poster sessions, and a panel discussion, followed by an evening social (all times in Pacific Time).
Venue
Union Square 22 · Fourth Floor
Hilton San Francisco Union Square
333 O’Farrell Street, San Francisco, CA 94102
Arrival & check-in
The workshop room opens at 08:00; opening remarks begin at 09:00.
COLM’s Friday registration desk is in the Plaza Ballroom, open 08:00–13:00.
Refreshments & lunch
COLM provides coffee, tea, and water 08:30–17:30, plus morning and afternoon snacks.
Lunch is on your own. Our lunch break is 12:40–13:50.
Poster presenters: please use only the approved tape provided by COLM to mount posters on walls.
Talk abstract
Diffusion language models offer an alternative to left-to-right autoregressive generation by enabling flexible generation orders and iterative refinement. In this talk, I will discuss our recent work on exploiting these properties for code generation, infilling, and reasoning, with a focus on how generation order affects model behavior and how diffusion-style decoding can enable more flexible and parallel computation. I will also highlight some open challenges in scaling diffusion language models and improving their inference efficiency.
Zhihan Yang, Wei Guo, Shuibai Zhang, Subham Sekhar Sahoo, Yongxin Chen, Arash Vahdat, Morteza Mardani, John Thickstun
10 min presentation + 5 min Q&A · Presenter to be confirmed
Jaehoon Yoo, Wonjung Kim, Floor Eijkelboom, Chanhyuk Lee, Nicholas Matthew Boffi, Seunghoon Hong, Jinwoo Kim
10 min presentation + 5 min Q&A · Presenter to be confirmed
Xingyu Su, Jacob Helwig, Shubham Parashar, Atharv Chagi, Lakshmi Jotsna Madhavarapu, Degui Zhi, James Caverlee, Dileep Kalathil, Shuiwang Ji
10 min presentation + 5 min Q&A · Presenter to be confirmed
Haozhe Jiang, Haiwen Feng, Pieter Abbeel, Jiantao Jiao, Angjoo Kanazawa, Nika Haghtalab
10 min presentation + 5 min Q&A · Presenter to be confirmed
Invited talk titles and oral presenter names will be added as they become available. Each oral talk has 10 minutes for presentation and 5 minutes for Q&A.
Speakers
All invited speakers and panelists have confirmed
The future of diffusion language generation
Our panel examines when iterative discrete generation offers qualitatively different capabilities from standard autoregressive modeling, and what technical barriers remain in scaling, controllability, inference-time computation, and evaluation — bringing together complementary viewpoints from academia and industry.
Junior Organizers
Senior Organizers
Inclusion
Our speakers, panelists, and organizers span career stages, institution types, and geographies across North America, Europe, the Middle East, and Asia, bridging academia and industry.
We support broad participation through an open, non-archival call, poster presentations for all accepted papers, and oral talks for selected submissions.