Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add partitioners property to FederatedDataset #3096

Merged
merged 10 commits into from
Mar 13, 2024
19 changes: 19 additions & 0 deletions datasets/flwr_datasets/federated_dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,25 @@ def load_full(self, split: str) -> Dataset:
self._check_if_split_present(split)
return self._dataset[split]

@property
def partitioners(self) -> Dict[str, Partitioner]:
"""Dictionary mapping split to associated partitioners.

The returned partitioners have the splits of the dataset assigned to them.
"""
# This function trigger the dataset download (laizy download) and checks
# the partitioner specification correctness (which can also happen lazily only
# after the dataset download).
if not self._dataset_prepared:
self._prepare_dataset()
if self._dataset is None:
raise ValueError("Dataset is not loaded yet.")
partitioners_keys = list(self._partitioners.keys())
for split in partitioners_keys:
self._check_if_split_present(split)
self._assign_dataset_to_partitioner(split)
return self._partitioners

def _check_if_split_present(self, split: str) -> None:
"""Check if the split (for partitioning or full return) is in the dataset."""
if self._dataset is None:
Expand Down