Hugging Face Blog·· 11d agoSelected
PyTorch 2.15 Adds FP8, Accelerates MoE Decoding, and Cuts Wan2.2 Video VAE Memory
Original source: Hugging Face BlogRead original
AI brief
PyTorch 2.15 introduces FP8 support on MPS, enabling half‑size storage of float8_e4m3fn tensors and scaled matrix multiplication. Mixture‑of‑experts models now decode 5–6.5× faster on an M3 Ultra, with Qwen3.6‑35B‑A3B jumping from 9.4 to 61.3 tokens/s. Wan2.2’s video VAE memory usage drops from 270 GiB to about 8 GiB. The update also speeds up linear layers, softmax, and various audio‑video operations, and requires macOS 15+ and Python 3.11+.
Why it matters
These upgrades give AI developers on macOS faster local LLM inference, lower memory footprints for video generation, and broader support for low‑precision FP8, all of which are critical for efficient on‑device AI workloads.
Source: Hugging Face Blog · huggingface.co