New exponent functions that make SiLU and SoftMax 2x faster, at full accuracy(github.com)

368 points by weinzierl 2 years ago | 72 comments

GitHub pull request to llama.cpp by jart that upstreams llamafile's vectorized expf() functions to compute SiLU and softmax faster and more accurately on CPU.

LLama.cpp now has a web interface(github.com)

287 points by xal 3 years ago | 45 comments

Mistral Integration Improved in Llama.cpp(github.com)

82 points by decide1000 1 year ago | 13 comments

Llama.cpp: Full CUDA GPU Acceleration(github.com)

640 points by gzer0 3 years ago | 265 comments

A draft GitHub pull request to llama.cpp adding full CUDA GPU acceleration for all ggml tensors, including storing the KV cache in VRAM.

Java Panama Vector API Integrated with Apache Lucene(github.com)

67 points by kurhan 3 years ago | 12 comments

M2 Ultra can run 128 streams of Llama 2 7B in parallel(github.com)

194 points by behnamoh 3 years ago | 126 comments

Llama2.c: Inference llama 2 in one file of pure C(github.com)

659 points by anjneymidha 3 years ago | 158 comments

GitHub repository by Andrej Karpathy for llama2.c, a minimal implementation for running Llama 2 model inference in a single file of pure C.

Llama: Add grammar-based sampling(github.com)

379 points by davepeck 3 years ago | 95 comments

A GitHub pull request to llama.cpp adding an API for grammar-based sampling that constrains LLM output with a BNF-like context-free grammar.

Run Llama 13B with a 6GB graphics card(gist.github.com)

564 points by rain1 3 years ago | 233 comments

A GitHub Gist with instructions for running Meta's Llama 13B model on a 6GB GPU using llama.cpp's CUDA/cuBLAS layer offloading.

Llama.cpp: Full CUDA GPU Acceleration(github.com)

640 points by gzer0 3 years ago | 265 comments

A draft GitHub pull request to llama.cpp adding full CUDA GPU acceleration for all ggml tensors, including storing the KV cache in VRAM.

Show HN: Llama 3.2 Interpretability with Sparse Autoencoders(github.com)

542 points by PaulPauls 1 year ago | 80 comments

GitHub repository with an end-to-end PyTorch pipeline for LLM interpretability using sparse autoencoders on Llama 3.2, replicating Anthropic/OpenAI/DeepMind research.

Llamafile lets you distribute and run LLMs with a single file(github.com)

972 points by tfinch 2 years ago | 253 comments

GitHub repository for Mozilla-Ocho's llamafile, a tool to distribute and run large language models as a single executable file.