KV Sharing, MHC, and Compressed Attention(magazine.sebastianraschka.com)

24 points by gmays 5 months ago | 1 comments

Coding Self-Attention, Multi-Head Attention, Cross-Attention, Causal-Attention(magazine.sebastianraschka.com)

102 points by rasbt 2 years ago | 7 comments

Understanding Reasoning LLMs(magazine.sebastianraschka.com)

442 points by sebg 1 year ago | 163 comments

Sebastian Raschka's Ahead of AI newsletter article explaining the four main approaches to building and refining reasoning LLMs.

Language models for text classification: From bag-of-words to Jev(magazine.sebastianraschka.com)

181 points by Anon84 27 days ago | 9 comments

Sebastian Raschka's Ahead of AI newsletter post giving a visual guide to text classification, from bag-of-words through RNNs, CNNs, and transformers to Jev-like APIs and calibration.

Building LLMs from the Ground Up: A 3-Hour Coding Workshop(magazine.sebastianraschka.com)

910 points by mdp2021 2 years ago | 110 comments

Sebastian Raschka's Ahead of AI newsletter post presenting a 3-hour video coding workshop on implementing, training, and using LLMs from scratch.

Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch(magazine.sebastianraschka.com)

85 points by rasbt 2 years ago | 10 comments