Skip to content

Commit f8035dd

Browse files
authored
fix wrong docstring (#262)
1 parent a49c6db commit f8035dd

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

torchft/local_sgd.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -561,8 +561,8 @@ def _allreduce_bucketized(self) -> None:
561561

562562
class DiLoCo:
563563
"""
564-
DiLoCo is a subclass of LocalSGD that overrides the synchronization
565-
mechanism to average and synchronize the pseudogradients (delta of the previous global weight and current local weights).
564+
DiLoCo implements distributed optimization by averaging and synchronizing
565+
pseudogradients (delta of the previous global weight and current local weights).
566566
567567
The class implements a more general version of DiLoco, Streaming DiLoCo,
568568
which synchronizes fragments of pseudogradients at different steps.

0 commit comments

Comments
 (0)