Hugging Face Blog·· 2022-05-02AI 评分38
用 PyTorch FullyShardedDataParallel 加速大模型训练
Accelerate Large Model Training using PyTorch Fully Sharded Data Parallel
AI 导读
Hugging Face 介绍如何通过 Accelerate 库无需改代码即可使用 PyTorch FullyShardedDataParallel（FSDP）训练大模型。
来源：Hugging Face Blog · huggingface.co