Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch (magazine.sebastianraschka.com)

Screenshot of magazine.sebastianraschka.com: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch

First screen, captured 2 years ago.

Looks similar

Screenshot of magazine.sebastianraschka.com: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)

Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)(magazine.sebastianraschka.com)

311 points by rasbt 2 years ago | 24 comments

Screenshot of magazine.sebastianraschka.com: Training and aligning LLMs with RLHF and RLHF alternatives

Training and aligning LLMs with RLHF and RLHF alternatives (magazine.sebastianraschka.com)

82 points by rasbt 3 years ago | 10 comments

Screenshot of magazine.sebastianraschka.com: Understanding Reasoning LLMs

Understanding Reasoning LLMs (magazine.sebastianraschka.com)

442 points by sebg 1 year ago | 163 comments

Sebastian Raschka's Ahead of AI newsletter article explaining the four main approaches to building and refining reasoning LLMs.

reasoning-models llm inference-time-scaling reinforcement-learning

Screenshot of magazine.sebastianraschka.com: Coding Self-Attention, Multi-Head Attention, Cross-Attention, Causal-Attention

Coding Self-Attention, Multi-Head Attention, Cross-Attention, Causal-Attention (magazine.sebastianraschka.com)

102 points by rasbt 2 years ago | 7 comments

Screenshot of magazine.sebastianraschka.com: Language models for text classification: From bag-of-words to Jev

Language models for text classification: From bag-of-words to Jev (magazine.sebastianraschka.com)

181 points by Anon84 27 days ago | 9 comments

Sebastian Raschka's Ahead of AI newsletter post giving a visual guide to text classification, from bag-of-words through RNNs, CNNs, and transformers to Jev-like APIs and calibration.

text-classification language-models transformers bag-of-words

Screenshot of magazine.sebastianraschka.com: LLM architecture comparison

LLM architecture comparison (magazine.sebastianraschka.com)

357 points by mdp2021 1 year ago | 23 comments

Sebastian Raschka's Ahead of AI article comparing the architectures of modern LLMs, from DeepSeek-V3 to Kimi K2.

llm-architecture deepseek-v3 kimi-k2 transformers

Looks similar

Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)(magazine.sebastianraschka.com)

Training and aligning LLMs with RLHF and RLHF alternatives(magazine.sebastianraschka.com)

Understanding Reasoning LLMs(magazine.sebastianraschka.com)

Coding Self-Attention, Multi-Head Attention, Cross-Attention, Causal-Attention(magazine.sebastianraschka.com)

Language models for text classification: From bag-of-words to Jev(magazine.sebastianraschka.com)

LLM architecture comparison(magazine.sebastianraschka.com)

Training and aligning LLMs with RLHF and RLHF alternatives (magazine.sebastianraschka.com)

Understanding Reasoning LLMs (magazine.sebastianraschka.com)

Coding Self-Attention, Multi-Head Attention, Cross-Attention, Causal-Attention (magazine.sebastianraschka.com)

Language models for text classification: From bag-of-words to Jev (magazine.sebastianraschka.com)

LLM architecture comparison (magazine.sebastianraschka.com)