Microarchitecture and RTL for the arithmetic unit that GenAI inference spends its time in. This site collects my notes on floating-point arithmetic, MX data formats, and AI for chip design.
I lead microarchitecture and RTL design of the Dot Product Engine (DPE) for Meta’s MTIA GenAI inference accelerators, supporting 22+ single- and mixed-precision data-format combinations. I write the microarchitecture specifications for the DV and performance-modeling teams, and work with the C2RTL and Formal teams on bit-accurate datapath models and numerical sign-off.
I led design and implementation of the floating-point unit (FPU) for Tensix Neo tensor cores. The FPU supports 12 data formats (BF16, FP16, TF32, INT8, MXFP4, among others) with IEEE/OCP compliance and 4:1 structured sparsity. I also built its bit-accurate Algorithmic C model, used for microarchitecture exploration.
Three years on AI accelerators: NoC bridges between internal protocols and external IPs such as PCIe and HBM — credit-based NoC, DMA, controllers, CDC, STA, synthesis.
Research on inference-latency modelling of BERT on multi-core ARM processors.
Most read · Data formats · Jan 2025
INT8, BFloat, FP8, MSFP, MX — if one factor drove the biggest AI efficiency gains of the past decade, it is the evolution of data formats.
Datapath · Feb 2025
“A bunch of tensors getting multiplied and added” — this is what 70% of AI workloads boil down to. Adders consume 4 to 30 times less energy than multipliers. A walk through what actually happens under the hood when hardware adds two floating-point numbers.
Memory · Dec 2024
On-chip SRAM excels at bandwidth, but relying on it alone is inefficient for LLMs: LLaMA-3-70B needs 140 GB of weights plus several GB of KV cache. The memory footprint math, unpacked.
Questions or ideas — you can reach me on X or LinkedIn.
the asic brew.