We fine-tuned Llama 405B on AMD GPUs (publish.obsidian.md) | HN Screenshots

YHacker Newsscreenshots

Screenshot of publish.obsidian.md: We fine-tuned Llama 405B on AMD GPUs

First screen, captured 2 years ago.

Looks similar

Screenshot of fergusfinn.com: Bringing Up DeepSeek-V4-Flash on AMD MI300X

Bringing Up DeepSeek-V4-Flash on AMD MI300X (fergusfinn.com)

107 points by kkm 4 months ago | 17 comments

Screenshot of mwilczynski.dev: Igniting the GPU: From Kernel Plumbing to 3D Rendering on RISC-V

Igniting the GPU: From Kernel Plumbing to 3D Rendering on RISC-V (mwilczynski.dev)

90 points by michalwilczynsk 10 months ago | 8 comments

Screenshot of baseten.co: Running GPT-OSS-120B at 500 tokens per second on Nvidia GPUs

Running GPT-OSS-120B at 500 tokens per second on Nvidia GPUs (baseten.co)

215 points by philipkiely 1 year ago | 142 comments

Screenshot of blog.mlc.ai: Making AMD GPUs competitive for LLM inference (2023)

Making AMD GPUs competitive for LLM inference (2023)(blog.mlc.ai)

297 points by plasticchris 1 year ago | 199 comments

Screenshot of makie.org: Raycore: GPU accelerated and modular ray intersections

Raycore: GPU accelerated and modular ray intersections (makie.org)

20 points by simondanisch 11 months ago | 3 comments

Screenshot of blog.vllm.ai: vLLM large scale serving: DeepSeek 2.2k tok/s/h200 with wide-ep

vLLM large scale serving: DeepSeek 2.2k tok/s/h200 with wide-ep (blog.vllm.ai)

138 points by robertnishihara 9 months ago | 46 comments

Related stories

Screenshot of cerebras.ai: Llama 3.1 405B now runs at 969 tokens/s on Cerebras Inference

Llama 3.1 405B now runs at 969 tokens/s on Cerebras Inference (cerebras.ai)

404 points by benchmarkist 1 year ago | 130 comments

Cerebras blog post announcing that Llama 3.1 405B runs at 969 tokens per second on its Cerebras Inference service.

cerebras llama-3-1 llm-inference ai-hardware

Screenshot of mlops.systems: My finetuned models beat OpenAI's GPT-4

My finetuned models beat OpenAI's GPT-4 (mlops.systems)

376 points by majc2 2 years ago | 86 comments

Blog post evaluating finetuned Mistral, Llama3 and Solar LLMs against OpenAI's GPT-4 on accuracy for structured data extraction from press releases.

fine-tuning llms gpt-4 mistral

Screenshot of gist.github.com: Run Llama 13B with a 6GB graphics card

Run Llama 13B with a 6GB graphics card (gist.github.com)

564 points by rain1 3 years ago | 233 comments

A GitHub Gist with instructions for running Meta's Llama 13B model on a 6GB GPU using llama.cpp's CUDA/cuBLAS layer offloading.

llama llama-cpp cuda cublas

Screenshot of twitter.com: Meta AI releases Code Llama 70B

Meta AI releases Code Llama 70B (twitter.com)

448 points by albert_e 2 years ago | 233 comments

A post from Meta AI on X announcing the release of Code Llama 70B, a larger version of its code-generation language model.

code-llama meta-ai llama large-language-models

Screenshot of github.com: Llama2.c: Inference llama 2 in one file of pure C

Llama2.c: Inference llama 2 in one file of pure C (github.com)

659 points by anjneymidha 3 years ago | 158 comments

GitHub repository by Andrej Karpathy for llama2.c, a minimal implementation for running Llama 2 model inference in a single file of pure C.

llama-2 c llm-inference karpathy

Screenshot of fly.io: We were wrong about GPUs

We were wrong about GPUs (fly.io)

834 points by mxstbr 1 year ago | 529 comments

Fly.io blog post reflecting on how the company's bet on offering GPU machines to developers did not pan out as expected.

fly-io gpus cloud-hosting ai-inference