This repository provides metapackages for pip and conda that centralize the Dask version dependency across RAPIDS.
Dask's API instability means that each RAPIDS release must pin to a very specific Dask release to avoid incompatibilities.
These metapackages provide a centralized, versioned storehouse for that pinning.
The rapids-dask-dependency package encodes both dask and distributed requirements.
This package is versioned just like the rest of RAPIDS: using CalVer, with alpha tags (trailing a*) for nightlies.
Nightlies of the metapackage should be consumed just like nightlies of any other RAPIDS package:
- conda packages should pin up to the minor version with a trailing .*, i.e. ==23.10.*. Conda will allow nightlies to match, so no further intervention is needed.
- pip packages should have the same pin, but wheel building scripts must add an alpha spec >=0.0.0a0 when building nightlies to allow rapids-dask-dependency nightlies. This is the same strategy used to have RAPIDS repositories pull nightly versions of other RAPIDS dependencies (e.g. cudf requires rmm nightlies).
In addition to functioning as a metapackage, rapids-dask-dependency also includes code for patching dask itself.
This package is never intended to be manually imported by the user.
Instead, upon installation it installs a .pth file (see the site module documentation for how these work) that will be run whenever the Python interpreter starts.
This file installs a custom meta path loader that intercepts all calls to import dask modules.
This loader is set up to apply RAPIDS-specific patches to the modules, ensuring that regardless of import order issues dask modules will always be patched for RAPIDS-compatibility in environments where RAPIDS packages are installed.
