Llama.cpp can do 40 tok/s on M2 Max, 0% CPU usage, using all 38 GPU cores(twitter.com)40 points by samwillis 3 years ago | 9 comments
OpenAI API keys leaking through app binaries(twitter.com)105 points by archiv 3 years ago | 59 comments
Why MMAP in llama.cpp hides true memory usage(twitter.com)95 points by flurly 3 years ago | 69 comments
State-of-the-art open-source chatbot, Vicuna-13B, just released model weights(twitter.com)241 points by weichiang 3 years ago | 113 comments