High-Speed Large Language Model Serving on PCs with Consumer-Grade GPUs (github.com) | HN Screenshots

YHacker Newsscreenshots

Screenshot of github.com: High-Speed Large Language Model Serving on PCs with Consumer-Grade GPUs

First screen, captured 2 years ago.

Looks similar

Screenshot of github.com: Show HN: I've made an easy to extend and flexible JavaScript logger

Show HN: I've made an easy to extend and flexible JavaScript logger (github.com)

14 points by inshinrei 1 year ago | 7 comments

Screenshot of github.com: Clojure Civitas – Publish Clojure Ideas and Explorations

Clojure Civitas – Publish Clojure Ideas and Explorations (github.com)

47 points by TheWiggles 1 year ago | 6 comments

Screenshot of github.com: ILGPU: Write GPU programs with C# and F#

ILGPU: Write GPU programs with C# and F#(github.com)

135 points by neonsunset 2 years ago | 18 comments

Screenshot of github.com: MatX: Efficient C++17 GPU numerical computing library with Python-like syntax

MatX: Efficient C++17 GPU numerical computing library with Python-like syntax (github.com)

84 points by cl3misch 3 years ago | 64 comments

Screenshot of github.com: Run ancient UNIX on modern hardware

Run ancient UNIX on modern hardware (github.com)

106 points by doener 11 months ago | 26 comments

Screenshot of github.com: Sane C++ Libraries

Sane C++ Libraries (github.com)

57 points by cozis 2 years ago | 104 comments

Related stories

Screenshot of codeconfessions.substack.com: What every developer should know about GPU computing

What every developer should know about GPU computing (codeconfessions.substack.com)

422 points by Anon84 3 years ago | 149 comments

A Substack primer by Abhinav Upadhyay explaining GPU architecture and computing concepts for software developers familiar mainly with CPUs.

gpu gpu-architecture parallel-computing cuda

Screenshot of github.com: Heretic: Automatic censorship removal for language models

Heretic: Automatic censorship removal for language models (github.com)

705 points by melded 11 months ago | 339 comments

GitHub repository for Heretic, an AGPL-licensed tool for fully automatic censorship removal (abliteration) in transformer language models.

abliteration llm censorship-removal transformers

Screenshot of hazyresearch.stanford.edu: GPUs Go Brrr

GPUs Go Brrr (hazyresearch.stanford.edu)

1056 points by nmstoker 2 years ago | 243 comments

Stanford Hazy Research blog post on what GPU hardware wants for efficient AI kernels, introducing the ThunderKittens CUDA kernel library.

gpu-kernels thunderkittens cuda flashattention

Screenshot of github.com: Show HN: Local pretrained classifiers, GPU not needed

Show HN: Local pretrained classifiers, GPU not needed (github.com)

6 points by nico 22 days ago | 1 comments

GitHub repository for Jeffy, a Python library of pretrained text classifiers that run and retrain locally on CPU without a GPU.

text-classification python cpu-inference sentiment-analysis

Screenshot of github.com: GigaToken: ~1000x faster Language model tokenization

GigaToken: ~1000x faster Language model tokenization (github.com)

585 points by syrusakbary 3 months ago | 116 comments

GitHub repository for gigatoken, an open-source library for fast language model tokenization at GB/s speeds, MIT licensed.

tokenization llm nlp rust

Screenshot of github.com: Nvidia Warp: A Python framework for high performance GPU simulation and graphics

Nvidia Warp: A Python framework for high performance GPU simulation and graphics (github.com)

453 points by jarmitage 2 years ago | 125 comments

GitHub repository for NVIDIA Warp, a Python framework for writing high-performance GPU simulation and graphics code.

nvidia-warp python gpu cuda