nathan.rs on Hacker News

3 stories on the front page, newest first. Scroll down to watch the site change over time.

Screenshot of nathan.rs: Can gzip be a language model?

Can gzip be a language model?(nathan.rs)

384 points by networked 1 month ago | 149 comments

A blog post by Nathan Barry exploring whether gzip compression can be used as a language model to generate text, with an interactive beam search demo.

gzip compression language-models text-generation

Screenshot of nathan.rs: BERT is just a single text diffusion step

BERT is just a single text diffusion step (nathan.rs)

435 points by nathan-barry 1 year ago | 102 comments

Blog post arguing BERT-style masked language modeling is a single step of discrete text diffusion, with experiments fine-tuning RoBERTa as a diffusion text generator.

text-diffusion bert roberta masked-language-models

Screenshot of nathan.rs: Running GPT-2 in WebGL: Rediscovering the Lost Art of GPU Shader Programming

Running GPT-2 in WebGL: Rediscovering the Lost Art of GPU Shader Programming (nathan.rs)

136 points by nathan-barry 1 year ago | 35 comments

nathan.rs on Hacker News

Can gzip be a language model?(nathan.rs)

BERT is just a single text diffusion step(nathan.rs)

Running GPT-2 in WebGL: Rediscovering the Lost Art of GPU Shader Programming(nathan.rs)

BERT is just a single text diffusion step (nathan.rs)

Running GPT-2 in WebGL: Rediscovering the Lost Art of GPU Shader Programming (nathan.rs)