#systems

The canonical, timeless reference for POSIX socket programming, IP routing, client/server architectures, and low-level networking in C.

• Essential fundamentals for distributed systems and custom network transports.

Deep dive into XGrammar’s grammar-guided constrained decoding algorithms.

• Efficiently pre-computes grammar masks across Context-Free Grammars (CFGs) and JSON schemas.
• Achieves near-zero latency penalty during token generation.

Rigorous economic and hardware analysis of serving LLMs at scale.

• Models memory bandwidth limits vs compute-bound FLOPs across prefill and decode stages.
• Breaks down token cost structures and batching efficiency.

An ultra-compact static embedding model requiring only 8 MB RAM.

• Capable of indexing massive corpora on constrained edge hardware in minutes.
• Demonstrates extreme quantization and lexical-dense hybridization.

Clear mental models breaking down how packets traverse routers, switches, DNS lookups, TCP handshakes, and NAT tables.

• Great refresher for backend and distributed systems engineering.

Luke Huang on the challenges of asynchronous, distributed policy gradient methods at frontier model scale.

• Evaluates staleness penalties, off-policy correction heuristics, and gradient synchronization tradeoffs.

Step-by-step mathematical derivation and algorithmic mechanics of speculative decoding.

• Covers draft model generation, target verification trees, and modified rejection sampling proofs.

Harshal’s ultra-compact model router trained with sep-CMA-ES.

• Dynamically selects open-source models (DeepSeek, GLM, Kimi) based on query complexity and expected compute cost.

Proposes treating LLM agent memory analogous to OS virtual memory systems (registers, cache lines, paging, and persistent swap).

• Evaluates hierarchical retrieval strategies for persistent long-horizon autonomy.

LMSYS technical update introducing diffusion-assisted token drafting and advanced tree verification algorithms delivering >3x decoding acceleration.

DatologyAI’s architecture marrying inverted index lexical matching speeds with neural dense semantic embeddings in a unified vector index.

Vlad Feinberg’s concrete roadmap for preparing for systems and infrastructure roles at OpenAI, Anthropic, and DeepMind.

• Focuses on CUDA profiling, distributed networking primitives, and hardware failure recovery.

An exceptional open handbook on parallelizing transformers with JAX.

• Deep dive into tensor parallelism (TP), pipeline parallelism (PP), Fully Sharded Data Parallel (FSDP), and overlap communication.