pip install -i https://pypi.anaconda.org/octo/simple pyddapi
pip install -i https://pypi.anaconda.org/octo/label/DD-191/simple pyddapi
pip install -i https://pypi.anaconda.org/octo/label/dev/simple pyddapi
pip install -i https://pypi.anaconda.org/octo/label/master_github/simple pyddapi
pip install -i https://pypi.anaconda.org/octo/label/v3_test/simple pyddapi
The following section describes the main concepts used in the Data Driver environment.
A Data Driver workflow is a network of tasks (python function) linked together. This workflow is typically described as a DAG (Direct Acyclic Graph). The jobs can execute all kind of Python code like data loading, feature engineering, model fitting, alerting, etc.
A workflow can be scheduled and monitored in the Data Driver architecture with the tool Airflow. The Data Driver API adds data science capabilities to Airflow and the capacity to audit the input / output of each task.
ddapi is a Python library. It is the access layer to Data Driver and you can use it to manipulated datasets and workflows. Some main usages are described below, for more informations and tutorials you can access to the OCTO notebook tutorials repository.
import dd
Ddapi is composed of several modules.
import dd.db
DB is an easier way to interact with your databases. You can use it to explore your databases or import new data.
from dd.api.contexts.distributed import AirflowContext
from dd.api.contexts.local import LocalContext
The context is an object which will allow you to communicate with your environment during your exploration. As such, it needs to be able to communicate with your database. This is done by creating a DB object and passing it to the context constructor.
import dd.api.workflow.dataset
You may consider datasets as wrappers around Pandas DataFrames. It gives you access to some methods you may recognise if you are familiar with this awesome library.
last release
pip install pyddapi
last build from master
pip install -i https://pypi.anaconda.org/octo/label/dev/simple pyddapi
virtualenv venv
source venv/bin/activate
pip install -e .
pip install -r ci/tests_requirements.txt
ddapi only supports python versions 2.7 and 3.6/ Running ddapi with other versions is not advised, so avoid it if possible, or do it at your own risk.
You can find the package in anaconda cloud repository
In case you want to contribute to the code, do not forget to check our
Developer Guidelines