Skip to content

Training and serving on AI ASICs.

Google TPU, AWS Trainium and Inferentia, Intel Gaudi: what runs on them, what a token costs against a GPU, and where the compiler stops you. One study measured on the chips, and the tools and talks around it.

Read the studyWill your model run on an ASIC?

  • Inferentia2$0.507
  • MI300X$0.209
  • H200$0.279
Llama 3.1 8B: dollars per million output tokens at 32 concurrent streams, the highest load all three chips ran, at on-demand list prices. Lower is cheaper.

Measured on the chips

Tools and talks

  • Calculator

    will-it-asic

    Can that model run on that ASIC? Memory math and compiler support for Google TPU, AWS Trainium and Inferentia and Intel Gaudi against NVIDIA GPUs, every spec cited.

    Source

  • Agent skills

    XLA agentic development

    Claude Code and Codex skills for Google TPUs: Pallas kernels, XProf profiling, XLA compiler artifacts and PyTorch/XLA lowering problems.

  • PyTorch Europe conference talk

    De-mystifying PyTorch for ASICs

    The talk's code and assets. Its companion repository fine-tunes three models, SFT then DPO with LoRA, on both GPU and TPU.

    Pipeline code

  • Talk code, DevFest 2025

    PyTorch/XLA on Google TPU

    Google TPU code for our talks: PyTorch/XLA basics, MNIST, and inner and outer products the way a systolic array computes them.

  • Code

    First steps on Trainium1

    Deep-learning projects on Trainium1 instances, starting with MNIST, and a write-up of why it first trained on the CPU instead.