This is the Linux app named FairScale whose latest release can be downloaded as v0.4.13sourcecode.tar.gz. It can be run online in the free hosting provider OnWorks for workstations.
Download and run online this app named FairScale with OnWorks for free.
Befolgen Sie diese Anweisungen, um diese App auszuführen:
- 1. Diese Anwendung auf Ihren PC heruntergeladen.
- 2. Geben Sie in unserem Dateimanager https://www.onworks.net/myfiles.php?username=XXXXX den gewünschten Benutzernamen ein.
- 3. Laden Sie diese Anwendung in einem solchen Dateimanager hoch.
- 4. Starten Sie den OnWorks Linux-Online- oder Windows-Online-Emulator oder den MACOS-Online-Emulator von dieser Website.
- 5. Rufen Sie vom gerade gestarteten OnWorks Linux-Betriebssystem aus unseren Dateimanager https://www.onworks.net/myfiles.php?username=XXXXX mit dem gewünschten Benutzernamen auf.
- 6. Laden Sie die Anwendung herunter, installieren Sie sie und führen Sie sie aus.
SCREENSHOTS
Ad
FairScale
BESCHREIBUNG
FairScale is a collection of PyTorch performance and scaling primitives that pioneered many of the ideas now used for large-model training. It introduced Fully Sharded Data Parallel (FSDP) style techniques that shard model parameters, gradients, and optimizer states across ranks to fit bigger models into the same memory budget. The library also provides pipeline parallelism, activation checkpointing, mixed precision, optimizer state sharding (OSS), and auto-wrapping policies that reduce boilerplate in complex distributed setups. Its components are modular, so teams can adopt just the sharding optimizer or the pipeline engine without rewriting their training loop. FairScale puts emphasis on correctness and debuggability, offering hook points, logging, and reference examples for common trainer patterns. Although many ideas have since landed in core PyTorch, FairScale remains a valuable reference and a practical toolbox for squeezing more performance out of multi-GPU and multi-node jobs.
Eigenschaften
- Fully Sharded Data Parallel style parameter, grad, and optimizer sharding
- Pipeline parallelism utilities with schedule control
- Activation checkpointing to trade compute for memory
- Optimizer State Sharding (OSS) drop-in optimizers
- Mixed precision and auto-wrap policies for easy adoption
- Examples and hooks for production-grade distributed training
Programmiersprache
Python
Kategorien
This is an application that can also be fetched from https://sourceforge.net/projects/fairscale.mirror/. It has been hosted in OnWorks in order to be run online in an easiest way from one of our free Operative Systems.
