Skills DirectorySearch all skills ↗

AI agent skill · by @aiming-lab

Mixed Precision

Use FP16/BF16 mixed precision to accelerate training and reduce memory. Use when optimizing GPU performance.

Archived 2026-10-23 · Source updated 2026-08-19 · License: MIT. Check the original repository for the latest version.

SKILL.md instructions

## Mixed Precision Training Best Practice
Use torch.cuda.amp for automatic mixed precision:
- Wrap forward pass in torch.cuda.amp.autocast()
- Use GradScaler for loss scaling
- BF16 preferred over FP16 on Ampere+ GPUs (RTX 3xxx, A100, RTX 4xxx)
- Watch for NaN gradients — reduce learning rate if needed
- Do NOT use amp with custom CUDA kernels unless tested

Related skills