Speculative Decoding in vLLM on AMD GPUs(vllm.ai)
130 points by ankitg12 | 47 comments
130 points by ankitg12 | 47 comments
376 points by majc2 | 86 comments
Blog post evaluating finetuned Mistral, Llama3 and Solar LLMs against OpenAI's GPT-4 on accuracy for structured data extraction from press releases.