Jul 238 min read
Multi-Head Attention To Flash Attention
What actually changes when you go from manual attention to a fused kernel

No hype. Just experiments, projects, and lessons from building AI systems.
What actually changes when you go from manual attention to a fused kernel