Run Llama 13B with a 6GB graphics card(gist.github.com)
564 points by rain1 | 233 comments
A GitHub Gist with instructions for running Meta's Llama 13B model on a 6GB GPU using llama.cpp's CUDA/cuBLAS layer offloading.
#gpu-offloading on Hacker News
1 story tagged gpu-offloading, newest first.