AWS demonstrates three ways to train Qwen3 on a single trn2.48xlarge using torchtitan on Trainium

The AWS Machine Learning Blog details three distinct configurations for training the Qwen3-4B-Instruct-2507 model on a single trn2.48xlarge instance equipped with 16 Trainium2 accelerators. The methods include using Transformers v5 with AutoModelForCausalLM, utilizing torchtitan’s native Qwen3 configuration, and implementing a custom Neuron Kernel Interface (NKI) mixture-of-experts (MoE) kernel. All configurations use the same torchrun launch pattern and require specific versions of torch, torch_neuronx, transformers, and torchtitan-neuron.