conda-forge / packages / adaptive-scheduler 0.1.7

An asynchronous scheduler using MPI for Adaptive

Installers

conda install

  • linux-64  v0.1.7
  • osx-64  v0.1.7
To install this package with conda run:
conda install -c conda-forge adaptive-scheduler

Description

The Adaptive scheduler solves the following problem, you need to run a few 100 learners and can use >1k cores. You can't use a centrally managed place that is responsible for all the workers (like with dask or ipyparallel) because >1k cores is too many for them to handle. You also don't want to use dask or ipyparallel inside a job script because they write job scripts on their own. Having a job script that runs code that creates job scripts... With adaptive-scheduler you only need to define the learners and then it takes care of the running (and restarting) of the jobs on the cluster.

PRIVACY POLICY  |  EULA (Anaconda Cloud v2.33.29) © 2019 Anaconda, Inc. All Rights Reserved.