BERT is just a single text diffusion step(nathan.rs)
435 points by nathan-barry | 102 comments
Blog post arguing BERT-style masked language modeling is a single step of discrete text diffusion, with experiments fine-tuning RoBERTa as a diffusion text generator.