New exponent functions that make SiLU and SoftMax 2x faster, at full accuracy(github.com)

368 points by weinzierl 2 years ago | 72 comments

GitHub pull request to llama.cpp by jart that upstreams llamafile's vectorized expf() functions to compute SiLU and softmax faster and more accurately on CPU.

Llama.cpp Now Supports Qwen2-VL (Vision Language Model)(github.com)

143 points by BUFU 1 year ago | 37 comments

LLama.cpp now has a web interface(github.com)

287 points by xal 3 years ago | 45 comments

Llama.cpp: Full CUDA GPU Acceleration(github.com)

640 points by gzer0 3 years ago | 265 comments

A draft GitHub pull request to llama.cpp adding full CUDA GPU acceleration for all ggml tensors, including storing the KV cache in VRAM.

Mistral Integration Improved in Llama.cpp(github.com)

82 points by decide1000 1 year ago | 13 comments

ONNX Runtime merges WebGPU backend(github.com)

165 points by b_mc2 3 years ago | 30 comments

Llama.cpp: Full CUDA GPU Acceleration(github.com)

640 points by gzer0 3 years ago | 265 comments

A draft GitHub pull request to llama.cpp adding full CUDA GPU acceleration for all ggml tensors, including storing the KV cache in VRAM.

Llama.cpp 30B runs with only 6GB of RAM now(github.com)

1182 points by msoad 3 years ago | 356 comments

GitHub pull request #613 in llama.cpp by jart, merged, which uses mmap to make loading model weights 10-100x faster and reduce RAM usage.

Llamafile lets you distribute and run LLMs with a single file(github.com)

972 points by tfinch 2 years ago | 253 comments

GitHub repository for Mozilla-Ocho's llamafile, a tool to distribute and run large language models as a single executable file.

Llama2.c: Inference llama 2 in one file of pure C(github.com)

659 points by anjneymidha 3 years ago | 158 comments

GitHub repository by Andrej Karpathy for llama2.c, a minimal implementation for running Llama 2 model inference in a single file of pure C.

Llama.vim – Local LLM-assisted text completion(github.com)

496 points by kgwgk 1 year ago | 97 comments

GitHub repository for llama.vim, a Vim plugin that provides local LLM-assisted code and text completion using llama.cpp.

Talk-Llama(github.com)

428 points by plurby 3 years ago | 126 comments

GitHub directory for talk-llama, a whisper.cpp example that lets you talk to a LLaMA model locally using speech recognition and text-to-speech.