Tool for Notebook Workflows?

Ploomber (disclaimer: I’m the author) is a good option for this. It allows you to create notebook-based pipelines (scripts and functions can be used as well). It requires minimal “pipeline code”, just list your notebooks in a YAML file, for full flexibility there is a Python API available. Exporting pipelines to Airflow and Kubernetes (via Argo workflows) is supported.

Github repository.

Feel free to reach out directly if there are any questions! Twitter: edublancas

1 Like