From Words to Vectors: Understanding Tokens and Embeddings in Modern LLMs
A practical introduction to how language models turn text into token IDs and vectors.

Founder of Empirq. Building a home for AI/ML practitioners while documenting my journey through machine learning, LLMs, AI systems, and technical publishing.
A practical introduction to how language models turn text into token IDs and vectors.
Opening the box behind modern AI. One concept at a time: understanding the five core pieces of a neural network: data, weights, activation functions, loss, and learning.
AI, machine learning, and deep learning are often used like they mean the same thing. They do not.
“Practical, production-minded writing on AI systems, RAG, reliability, and the trade-offs that matter beyond the demo.”
“Clear, hands-on writing about AI systems and inference, with useful explanations of what changes when models move from concept to implementation.”
“Practical, engineering-first writing on generative AI, LLMs, and multi-agent systems, by data scientist and AI researcher Chidiebere Njoku.”