Datasets is a lightweight library providing two main features: - one-line dataloaders for many public datasets: one-liners to download and pre-process any of the number of datasets major public datasets (text datasets in 467 languages and dialects, image datasets, audio datasets, etc.) provided on the HuggingFace Datasets Hub. With a simple command like squad_dataset = load_dataset("squad"), get any of these datasets ready to use in a dataloader for training/evaluating a ML model (Numpy/Pandas/PyTorch/TensorFlow/JAX), - efficient data pre-processing: simple, fast and reproducible data pre-processing for the above public datasets as well as your own local datasets in CSV/JSON/text/PNG/JPEG/etc. With simple commands like `processed_dataset = dataset.map(process_example)`, efficiently prepare the dataset for inspection and ML model evaluation and training.
| Uploaded | Mon Mar 31 21:27:29 2025 |
| md5 checksum | a462a1089e95cfb23d261bc208a0177e |
| arch | x86_64 |
| build | py312h06a4308_0 |
| constrains | pillow >=9.4.0, jaxlib >=0.3.14, jax >=0.3.14, soundfile >=0.12.1, tensorflow-base >=2.6.0, soxr >=5.1, apache-beam >=2.26.0 |
| depends | aiohttp, dill >=0.3.0,<0.3.9, filelock, fsspec >=2023.1.0,<=2024.12.0, huggingface_hub >=0.24.0, multiprocess <0.70.17, numpy >=1.17, packaging, pandas, pyarrow >=15.0.0, python >=3.12,<3.13.0a0, python-xxhash, pyyaml >=5.1, requests >=2.32.2, tqdm >=4.66.3 |
| license | Apache-2.0 |
| license_family | Apache |
| md5 | a462a1089e95cfb23d261bc208a0177e |
| name | datasets |
| platform | linux |
| sha1 | 6a85179b23a0c2498f076c5ec4192553e45a705d |
| sha256 | bd7677f1223347678d363a655569058424264c839f8adf08ab3b6d8b864b2e4b |
| size | 866475 |
| subdir | linux-64 |
| timestamp | 1741368065295 |
| version | 3.3.2 |