Datasets is a lightweight library providing two main features: - one-line dataloaders for many public datasets: one-liners to download and pre-process any of the number of datasets major public datasets (text datasets in 467 languages and dialects, image datasets, audio datasets, etc.) provided on the HuggingFace Datasets Hub. With a simple command like squad_dataset = load_dataset("squad"), get any of these datasets ready to use in a dataloader for training/evaluating a ML model (Numpy/Pandas/PyTorch/TensorFlow/JAX), - efficient data pre-processing: simple, fast and reproducible data pre-processing for the above public datasets as well as your own local datasets in CSV/JSON/text/PNG/JPEG/etc. With simple commands like `processed_dataset = dataset.map(process_example)`, efficiently prepare the dataset for inspection and ML model evaluation and training.
| Uploaded | Mon Mar 31 21:27:33 2025 |
| md5 checksum | fe99d40d25afa26b2acd229a9d6b11d1 |
| arch | x86_64 |
| build | py310h06a4308_0 |
| depends | aiohttp, dill <0.3.7, fsspec >=2021.11.1, huggingface_hub >=0.2.0,<1.0.0, multiprocess, numpy >=1.17,<2.0a0, packaging, pandas, pyarrow >=6.0.0, python >=3.10,<3.11.0a0, python-xxhash, pyyaml >=5.1, requests >=2.19.0, responses <0.19, tqdm >=4.62.1 |
| license | Apache-2.0 |
| license_family | Apache |
| md5 | fe99d40d25afa26b2acd229a9d6b11d1 |
| name | datasets |
| platform | linux |
| sha1 | 9f3d058702bc94fc63d2908d7b67de75933e571f |
| sha256 | c1877574f828f9284fabd99987ac831ac6683cd8b49c8560aea22fa9756116f7 |
| size | 658742 |
| subdir | linux-64 |
| timestamp | 1679487100841 |
| version | 2.10.1 |