Notes from the edge of inference.

Rust vs Python for LLM Inference: I Benchmarked Everything So You Don't Have To

Candle + Actix-Web head-to-head against HuggingFace, vLLM and TensorRT-LLM — P50/P99 latency, cold starts and memory footprint, measured properly.