#distributed-training

AI & Machine Learning

Monarch API: 16 Gbps RDMA Speeds on AWS EFA [PyTorch Update]

Distributed training on supercomputers feels like wrestling a hydra—cut one head, two more debugging nightmares sprout. Monarch's Python API claims to tame it, hitting 16 Gbps file syncs that make clusters act like your laptop.

AI & Machine Learning

PyTorch 2.11: 2723 Commits Later, FlashAttention Speeds Up — But TorchScript's Dead

2723 commits from 432 contributors. PyTorch 2.11 promises distributed training breakthroughs and GPU speedups. But deprecating TorchScript? That's a gut punch for legacy code.