Update TFDS to v4.1.0

* It is now possible to manually download the data for all datasets (if the automated download fail for any reason). See [doc](https://www.tensorflow.org/datasets/overview#load_a_dataset).
* Simplification of the dataset creation API.
  * We've made it is easier to create datasets outside TFDS repository (see our updated [dataset creation guide](https://www.tensorflow.org/datasets/add_dataset)).
  * `_split_generators` should now returns `{'split_name': self._generate_examples(), ...}` (but current datasets are backward compatible).
  * All dataset inherit from `tfds.core.GeneratorBasedBuilder`. Converting a dataset to beam now only require changing `_generate_examples` (see [example and doc](https://www.tensorflow.org/datasets/beam_datasets#instructions)).
  * `tfds.core.SplitGenerator`, `tfds.core.BeamBasedBuilder` are deprecated and will be removed in future version.

* Better `pathlib.Path`, `os.PathLike` compatibility:
  * `dl_manager.manual_dir` now returns a pathlib-Like object. Example:

  ```python
  text = (dl_manager.manual_dir / 'downloaded-text.txt').read_text()
  ```

  * Note: Other `dl_manager.download`, `.extract`,... will return pathlib-like objects in future versions
  * `FeatureConnector`,... and most functions should accept `PathLike` objects. Let us know if some functions you need are missing.
  * Add a `tfds.core.as_path` to create pathlib.Path-like objects compatible with GCS (e.g. `tfds.core.as_path('gs://my-bucket/labels.csv').read_text()`).

* Other bug fixes and improvement. E.g.
  * Add `verify_ssl=` option to `tfds.download.DownloadConfig` to disable SSH certificate during download.
  * `BuilderConfig` are now compatible with Beam datasets #2348
  * `--record_checksums` now assume the new dataset-as-folder model
  * `tfds.features.Images` can accept encoded `bytes` images directly (useful when used with `img_name, img_bytes = dl_manager.iter_archive('images.zip')`).
  * Doc API now show deprecated methods, abstract methods to overwrite are now documented.
  * You can generate `imagenet2012` with only a single split (e.g. only the validation data). Other split will be skipped if not present.
* And of course, new datasets...

Thank you to all our contributors for improving TFDS!

PiperOrigin-RevId: 340614460

Author: Conchylicultor

tensorflow_datasets/stable_versions.txt

@@ -174,18 +174,27 @@ c4/en.noclean/2.2.0
 c4/en.noclean/2.2.1
 c4/en.noclean/2.3.0
 c4/en.noclean/2.3.1
-c4/en.realnewslike/2.2.0
-c4/en.realnewslike/2.2.1
-c4/en.realnewslike/2.3.0
-c4/en.realnewslike/2.3.1
-c4/en.webtextlike/2.2.0
-c4/en.webtextlike/2.2.1
-c4/en.webtextlike/2.3.0
-c4/en.webtextlike/2.3.1
+c4/en.noclean/3.0.1
 c4/en/2.2.0
 c4/en/2.2.1
 c4/en/2.3.0
 c4/en/2.3.1
+c4/en/3.0.1
+c4/multilingual/2.2.0
+c4/multilingual/2.2.1
+c4/multilingual/2.3.0
+c4/multilingual/2.3.1
+c4/multilingual/3.0.1
+c4/realnewslike/2.2.0
+c4/realnewslike/2.2.1
+c4/realnewslike/2.3.0
+c4/realnewslike/2.3.1
+c4/realnewslike/3.0.1
+c4/webtextlike/2.2.0
+c4/webtextlike/2.2.1
+c4/webtextlike/2.3.0
+c4/webtextlike/2.3.1
+c4/webtextlike/3.0.1
 caltech101/3.0.0
 caltech_birds2010/0.1.1
 caltech_birds2011/0.1.1
@@ -386,6 +395,7 @@ cifar10_corrupted/zoom_blur_3/1.0.0
 cifar10_corrupted/zoom_blur_4/1.0.0
 cifar10_corrupted/zoom_blur_5/1.0.0
 citrus_leaves/0.1.1
+cityscapes/semantic_segmentation/1.0.0
 civil_comments/CivilComments/1.0.0
 civil_comments/CivilCommentsIdentities/1.0.0
 clevr/3.0.0
@@ -434,9 +444,9 @@ cos_e/0.0.1
 cosmos_qa/1.0.0
 covid19sum/1.0.0
 crema_d/1.0.0
-curated_breast_imaging_ddsm/original-calc/2.0.1
-curated_breast_imaging_ddsm/original-mass/2.0.1
-curated_breast_imaging_ddsm/patches/2.0.1
+curated_breast_imaging_ddsm/original-calc/3.0.0
+curated_breast_imaging_ddsm/original-mass/3.0.0
+curated_breast_imaging_ddsm/patches/3.0.0
 cycle_gan/apple2orange/2.0.0
 cycle_gan/cezanne2photo/2.0.0
 cycle_gan/cityscapes/2.0.0
@@ -448,8 +458,7 @@ cycle_gan/monet2photo/2.0.0
 cycle_gan/summer2winter_yosemite/2.0.0
 cycle_gan/ukiyoe2photo/2.0.0
 cycle_gan/vangogh2photo/2.0.0
-deep_weeds/1.0.0
-deep_weeds/2.0.0
+deep_weeds/3.0.0
 definite_pronoun_resolution/1.1.0
 dementiabank/1.0.0
 diabetic_retinopathy_detection/1M/3.0.0
@@ -473,6 +482,7 @@ dsprites/2.0.0
 dsprites/2.1.0
 dtd/3.0.1
 duke_ultrasound/1.0.0
+e2e_cleaned/0.1.0
 emnist/balanced/3.0.0
 emnist/byclass/3.0.0
 emnist/bymerge/3.0.0
@@ -648,8 +658,8 @@ imagenet_resized/32x32/0.1.0
 imagenet_resized/64x64/0.1.0
 imagenet_resized/8x8/0.1.0
 imagenet_v2/matched-frequency/1.0.0
-imagenet_v2/threshold-0.7/0.1.1
-imagenet_v2/topimages/0.1.1
+imagenet_v2/threshold-0.7/1.0.0
+imagenet_v2/topimages/1.0.0
 imagenette/160px-v2/0.1.0
 imagenette/160px/0.1.0
 imagenette/320px-v2/0.1.0
@@ -668,6 +678,7 @@ iris/2.0.0
 kitti/3.1.0
 kitti/3.2.0
 kmnist/3.0.1
+lambada/1.0.0
 lfw/0.1.0
 librispeech/plain_text/1.1.0
 librispeech/subwords32k/1.1.0
@@ -751,6 +762,13 @@ math_dataset/polynomials__simplify_power/1.0.0
 math_dataset/probability__swr_p_level_set/1.0.0
 math_dataset/probability__swr_p_sequence/1.0.0
 mctaco/1.0.0
+mlqa/ar/1.0.0
+mlqa/de/1.0.0
+mlqa/en/1.0.0
+mlqa/es/1.0.0
+mlqa/hi/1.0.0
+mlqa/vi/1.0.0
+mlqa/zh/1.0.0
 mnist/3.0.1
 mnist_corrupted/brightness/1.0.0
 mnist_corrupted/canny_edges/1.0.0
@@ -861,6 +879,7 @@ paws_x_wiki/ko/1.0.0
 paws_x_wiki/zh/1.0.0
 pet_finder/1.0.0
 pg19/0.1.1
+piqa/1.0.0
 places365_small/2.0.0
 plant_leaves/0.1.0
 plant_village/1.0.2
@@ -948,6 +967,7 @@ starcraft_video/move_unit_to_border_64/1.0.0
 starcraft_video/road_trip_with_medivac_128/1.0.0
 starcraft_video/road_trip_with_medivac_64/1.0.0
 stl10/1.0.0
+story_cloze/1.0.0
 sun397/standard-part1-120k/4.0.0
 sun397/standard-part10-120k/4.0.0
 sun397/standard-part2-120k/4.0.0
@@ -1068,6 +1088,7 @@ wiki40b/uk/1.3.0
 wiki40b/vi/1.3.0
 wiki40b/zh-cn/1.3.0
 wiki40b/zh-tw/1.3.0
+wiki_bio/0.1.0
 wikihow/all/1.2.0
 wikihow/sep/1.2.0
 wikipedia/20190301.aa/1.0.0

tensorflow_datasets/version.py

@@ -24,8 +24,8 @@
 
 # We follow Semantic Versioning (https://semver.org/)
 _MAJOR_VERSION = '4'
-_MINOR_VERSION = '0'
-_PATCH_VERSION = '1'
+_MINOR_VERSION = '1'
+_PATCH_VERSION = '0'
 
 __version__ = '.'.join([
     _MAJOR_VERSION,

tensorflow_datasets/version_stable.py

@@ -20,8 +20,8 @@
 
 # We follow Semantic Versioning (https://semver.org/)
 _MAJOR_VERSION = '4'
-_MINOR_VERSION = '0'
-_PATCH_VERSION = '1'
+_MINOR_VERSION = '1'
+_PATCH_VERSION = '0'
 
 __version__ = '.'.join([
     _MAJOR_VERSION,