$ skilldirectory
— zsh — registry.skilldirectory.dev
skill library blog

home / DevOps / Ai Pipeline Orchestration Skill by @BagelHole

Ai Pipeline Orchestration Skill by @BagelHole

FREE
github stars★ 1,111updatedSep 2026repo@BagelHole
$git clone https://github.com/BagelHole/DevOps-Security-Agent-Skills

The skill is the devops/ai/ai-pipeline-orchestration folder inside that repository. Copy it into the folder your agent loads skills from; check your agent's documentation for where that is.

Read the opening of the original SKILL.md
# AI Pipeline Orchestration

Build reliable, observable AI workflows — from document ingestion to batch inference to model training pipelines.

## When to Use This Skill

Use this skill when:
- Scheduling recurring RAG document ingestion and re-indexing
- Orchestrating multi-step batch LLM processing workflows
- Running nightly model evaluation and fine-tuning jobs
- Building ETL pipelines that feed into AI models
- Managing dependencies between data preparation and model serving

## Tool Selection

| Tool | Best For | Complexity | GPU Jobs |
|------|----------|------------|----------|
| **Prefect** | Modern Python-first; easy to adopt | Low | Good |
| **Airflow** | Complex DAGs; large teams; existing usage | High | Good |
| **Dagster** | Asset-centric; strong data lineage | Medium | Excellent |
| **Temporal** | Long-running workflows; reliability-first | Medium | Good |

## Prefect — Quick Start

```bash
pip install prefect prefect-kubernetes
…

Excerpt only, quoted from the author's repository. Open the repository for the complete file.