llama.cpp

A C/C++ engine for running LLMs locally.

ToolAI & MLlocal-llminferencelogprobslocal-inferencellmggufcpu-offload

Recommended in 3 threads

IBM CEO says open weight models are now good and cheap on-prem
theregister.com · 12 hours ago
Gives decent performance on a single GPU with RAM offload.
Five Qwen3.8-27B models match Fable 5 Using GVS5H Harness
github.com · 1 day ago
Used with a detailed config to run Qwen 27B on a 24GB GPU.
Using LLM for confidence scoring is a bad idea
rumple.github.io · 5 days ago
Can expose next-token likelihoods over constrained label tokens without patching.

lean4-mlir ↗AI & ML
A project using Lean 4 to drive PJRT/MLIR for ML workloads.
1 thread
clojure-mcp AI & ML
MCP tooling that lets LLM agents edit Clojure code and interact with a REPL.
1 thread
Lacuna Research MCP ↗AI & ML
An MCP server that lets Claude or Codex search the Lacuna corpus of generated research proposals.
1 thread
chatjimmy ↗AI & ML
A chatbot service known for extremely fast text generation.
1 thread
Liquid AI Decision Models ↗AI & ML
Models from Liquid AI trained specifically to produce decisions with calibrated confidence.
1 thread
LIME ↗AI & ML
A technique for locally explaining individual predictions of any classifier.
1 thread
DeepSeek V4.1 Flash AI & ML
A low-cost DeepSeek language model available via API.
1 thread
DeepSeek AI & ML
A family of large language models offered via a low-cost API.
1 thread

OpenRouter ↗
A marketplace that routes API requests to many LLM inference providers.
6 threads
GLM ↗
An open-weights large language model family from Zhipu AI.
4 threads
GoModel ↗
LLM gateway written in Go, offered as a LiteLLM alternative.
3 threads
OpenCode Go ↗
A subscription plan for accessing models through the OpenCode coding tool.
2 threads
Cerebras ↗
A hosted fast inference service offering Qwen models.
1 thread
Claude
Anthropic's AI model and command-line assistant.
1 thread
Claude Code gateways ↗
Documented setup for routing Claude Code through an LLM gateway for centralized control and alternate model providers.
1 thread
Cloudflare Workers AI ↗
Cloudflare's serverless platform for running hosted AI models.
1 thread
CodeCarbon ↗
An open-source Python library for estimating the carbon emissions of compute workloads.
1 thread
Codex
OpenAI's coding agent.
1 thread
Codex for OSS ↗
An OpenAI program that gives open source maintainers direct access to its models.
1 thread
DeepSeek
A family of large language models offered via a low-cost API.
1 thread