Performance of llama.cpp on Apple Silicon A-series (github.com)

Screenshot of github.com: Performance of llama.cpp on Apple Silicon A-series

First screen, captured 2 years ago.

Looks similar

Screenshot of github.com: Qwen3 30B A3B Hits 13 token/s on 4xRaspberry Pi 5

Qwen3 30B A3B Hits 13 token/s on 4xRaspberry Pi 5 (github.com)

313 points by b4rtazz 1 year ago | 133 comments

Screenshot of github.com: Deepseek R1 Distill 8B Q40 on 4 x Raspberry Pi 5

Deepseek R1 Distill 8B Q40 on 4 x Raspberry Pi 5 (github.com)

256 points by b4rtazz 1 year ago | 124 comments

Screenshot of github.com: New exponent functions that make SiLU and SoftMax 2x faster, at full accuracy

New exponent functions that make SiLU and SoftMax 2x faster, at full accuracy (github.com)

368 points by weinzierl 2 years ago | 72 comments

GitHub pull request to llama.cpp by jart that upstreams llamafile's vectorized expf() functions to compute SiLU and softmax faster and more accurately on CPU.

llama-cpp ggml simd softmax

Screenshot of github.com: llama.cpp: Roadmap May 2023

llama.cpp: Roadmap May 2023 (github.com)

73 points by tosh 3 years ago | 4 comments

Screenshot of github.com: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI (github.com)

807 points by lairv 8 months ago | 211 comments

GitHub Discussions announcement by Georgi Gerganov that ggml.ai, the founding team of llama.cpp, is joining Hugging Face to support local AI.

llama-cpp ggml hugging-face local-ai

Screenshot of github.com: Llama.cpp: Full CUDA GPU Acceleration

Llama.cpp: Full CUDA GPU Acceleration (github.com)

640 points by gzer0 3 years ago | 265 comments

A draft GitHub pull request to llama.cpp adding full CUDA GPU acceleration for all ggml tensors, including storing the KV cache in VRAM.

llama-cpp cuda gpu-acceleration llm-inference

Looks similar

Qwen3 30B A3B Hits 13 token/s on 4xRaspberry Pi 5(github.com)

Deepseek R1 Distill 8B Q40 on 4 x Raspberry Pi 5(github.com)

New exponent functions that make SiLU and SoftMax 2x faster, at full accuracy(github.com)

llama.cpp: Roadmap May 2023(github.com)

Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI(github.com)

Llama.cpp: Full CUDA GPU Acceleration(github.com)