BitsExplorer

BitsExplorer finds what people are building with AI, every day. See today's picks →

Preview of Llm Inference Explained

Llm Inference Explained

Interactive explainer of real LLM inference: KV cache, roofline, batching, paging, FlashAttention, speculative decoding, quantisation, parallelism and serving metrics. Companion to the Transformer Decoder Explainer.

Description excerpted from the original listing, which is linked below.

Open it Source code

The week in AI-built software

Every Monday: how many projects were built with AI that week, which tools built them, where they came from, and the week's picks.

Read the latest week