ploomber / ploomber
- пятница, 18 марта 2022 г. в 00:30:35
https://github.com/ploomber/ploomber
The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️
Join our community | Newsletter | Contact us | Docs | Blog | Website | YouTube
Ploomber is the fastest way to build data pipelines
Installation
Compatible with Python 3.6 and higher.
Install with pip:
pip install ploomberOr with conda:
conda install ploomber -c conda-forgeGetting started
Open a hosted JupyterLab instance:
Run an example locally:
# ML pipeline example
ploomber examples -n templates/ml-basic -o ml-basic
cd ml-basic
# install dependencies
pip install -r requirements.txt
# run pipeline
ploomber buildYou just ran a Ploomber pipeline!
Check out the output/nb.html report with model results!
The pipeline.yaml contains the pipeline declaration. Feel free to modify any of the tasks, then call ploomber build again to update the results (Note: if using VSCode or PyCharm, execute ploomber nb -i before editing the files).
What's next?
Ready to migrate your project? Click here.
Do you want to learn more? Check out the introductory tutorial.
Run more examples.
Community
Main Features
⚡️ Get started quickly
A simple YAML API to get started quickly, a powerful Python API for total flexibility.
get-started.mp4
⏱ Shorter development cycles
Automatically cache your pipeline’s previous results and only re-compute tasks that have changed since your last execution.
shorter-cycles.mp4
☁️ Deploy anywhere
Run as a shell script in a single machine or distributively in Kubernetes, Airflow, AWS Batch, or SLURM.
deploy.mp4
📙 Automated migration from legacy notebooks
Bring your old monolithic notebooks, and we’ll automatically convert them into maintainable, modular pipelines.
refactor.mp4
I want to migrate my notebook.