Llama.cpp: Full CUDA GPU Acceleration(github.com)

640 points by gzer0 3 years ago | 265 comments

A draft GitHub pull request to llama.cpp adding full CUDA GPU acceleration for all ggml tensors, including storing the KV cache in VRAM.

SDL Now Supports DOS(github.com)

240 points by Jayschwa 6 months ago | 93 comments

ONNX Runtime merges WebGPU backend(github.com)

165 points by b_mc2 3 years ago | 30 comments

New exponent functions that make SiLU and SoftMax 2x faster, at full accuracy(github.com)

368 points by weinzierl 2 years ago | 72 comments

GitHub pull request to llama.cpp by jart that upstreams llamafile's vectorized expf() functions to compute SiLU and softmax faster and more accurately on CPU.

AI has a deep understanding of how this code works(github.com)

212 points by theresistor 11 months ago | 93 comments

Apple's MLX adding CUDA support(github.com)

522 points by nsagent 1 year ago | 181 comments

A GitHub pull request (closed, WIP) in Apple's ml-explore/mlx repository adding an experimental CUDA backend to the MLX framework.