PyTorch / CUDA: torch.cuda.OutOfMemoryError: CUDA out of memory
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 4.20 GiB (GPU 0; 23.68 GiB total capacity; 21.14 GiB already allocated; 1.12 GiB free; 21.80 GiB reserved in total by PyTorch)
Immediate Remediation
bash
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True python train.py
Root Cause Analysis
The GPU memory is exhausted because PyTorch's caching allocator reserves more memory than available, often due to large batch sizes or model activations. The error occurs when a new tensor allocation cannot fit in the remaining free memory.
Verification & Guardrails
- Requires PyTorch 1.9+ for expandable_segments
- May not fully solve if model is too large; consider gradient accumulation or quantization
- Verify with nvidia-smi to check memory usage
Have a custom or uncategorized crash?
Run your trace through our in-memory client privacy sandbox for instant SRE remediation.