repro.data.dataset_readers.datasets#
- class repro.data.dataset_readers.datasets.HuggingfaceDatasetsDatasetReader(dataset_name, split)#
- repro.data.dataset_readers.datasets.hf_dataset_exists_locally(name, version=None)#
Checks to see if a Huggingface datasets dataset exists locally in the cache. The logic checks to see if the directory exists where the data should be, but does not do any further verification.
- Parameters
name (str) – The name of the dataset, like “cnn_dailymail”
version (str, default=None) – The version of the dataset, like “3.0.0”. If None, then the default version is used if one exists.
- Returns
True if the dataset exists, False otherwise.
- Return type
bool