YHacker News

Strata github.com ↗

An open-source project for running large MoE models on small-VRAM GPUs with system RAM.

ToolAI & MLlocal-llminferencegpu-offload

Recommended in 1 thread

  • New iMac with M6

    apple.com ·

    Several users reported much better speed and quality for local models with it.

Similar tools

  • Parakeet AI & ML

    NVIDIA's family of open speech recognition models.

    1 thread
  • mlx-vlm ↗AI & ML

    A library for running vision-language models on Apple Silicon via MLX.

    1 thread
  • nanochat AI & ML

    A minimal-code repository for training a small GPT-2-style chat model.

    1 thread
  • FUTO ASR4All ↗AI & ML

    FUTO's small CPU-optimised streaming English ASR models.

    1 thread
  • Hermes AI & ML

    A personal AI agent framework that can run locally with local or cloud models.

    2 threads
  • NVIDIA Nemotron Labs Dory ↗AI & ML

    A small 4B-parameter language model from NVIDIA published on Hugging Face.

    0 threads
  • Mamba ↗AI & ML

    A state-space model architecture for sequence modeling, proposed as an alternative to attention.

    1 thread
  • OpenCode Go ↗AI & ML

    A subscription plan for accessing models through the OpenCode coding tool.

    2 threads

Also in AI & ML

  • OpenRouter ↗

    A marketplace that routes API requests to many LLM inference providers.

    8 threads
  • GLM ↗

    An open-weights large language model family from Zhipu AI.

    6 threads
  • llama.cpp ↗

    A C/C++ engine for running LLMs locally.

    6 threads
  • Claude

    Anthropic's AI model and command-line assistant.

    3 threads
  • DeepSeek V4.1 Flash ↗

    A low-cost DeepSeek language model available via API.

    3 threads
  • Epoch Capabilities Index ↗

    Epoch AI's aggregate index of model capabilities over time.

    3 threads
  • GoModel ↗

    LLM gateway written in Go, offered as a LiteLLM alternative.

    3 threads
  • oh-my-pi

    A setup built around the pi coding agent.

    3 threads
  • DeepSeek

    A family of large language models offered via a low-cost API.

    2 threads
  • Gemini Robotics ↗

    Google DeepMind's models for embodied reasoning and robot control.

    2 threads
  • Hermes

    A personal AI agent framework that can run locally with local or cloud models.

    2 threads
  • OpenCode Go ↗

    A subscription plan for accessing models through the OpenCode coding tool.

    2 threads

Link from a commenter, not checked or endorsed by us: ↗ https://github.com/Niko1221/Strata