Argo Workflows is a Kubernetes-native workflow engine for orchestrating parallel jobs. Each step runs in a container. Steps can form DAGs (directed acyclic graphs) with dependencies, retries, and conditional execution.
When to Use Argo Workflows
- Data pipelines — ETL jobs that process data through multiple stages
- ML training — hyperparameter tuning, model training, evaluation in parallel
- CI/CD — build pipelines with complex dependency graphs
- Infrastructure automation — multi-step provisioning with rollback
Installation
kubectl create namespace argo
kubectl apply -n argo -f \
https://github.com/argoproj/argo-workflows/releases/latest/download/install.yaml
# Install CLI
brew install argoMaster this topic with hands-on labs
Go beyond reading — build real projects in sandboxed environments with expert video guidance.
Browse Courses →Simple Workflow
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: hello-
spec:
entrypoint: main
templates:
- name: main
steps:
- - name: step1
template: echo
arguments:
parameters:
- name: message
value: "Step 1 complete"
- - name: step2
template: echo
arguments:
parameters:
- name: message
value: "Step 2 complete"
- name: echo
inputs:
parameters:
- name: message
container:
image: alpine:3.19
command: [echo]
args: ["{{inputs.parameters.message}}"]Steps in the same list item run in parallel. Steps in sequential list items run after the previous completes.
DAG Workflows
For complex dependency graphs:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: data-pipeline-
spec:
entrypoint: pipeline
templates:
- name: pipeline
dag:
tasks:
- name: extract
template: run-job
arguments:
parameters: [{name: stage, value: extract}]
- name: transform-users
depends: "extract"
template: run-job
arguments:
parameters: [{name: stage, value: transform-users}]
- name: transform-orders
depends: "extract"
template: run-job
arguments:
parameters: [{name: stage, value: transform-orders}]
- name: load
depends: "transform-users && transform-orders"
template: run-job
arguments:
parameters: [{name: stage, value: load}]
- name: run-job
inputs:
parameters: [{name: stage}]
container:
image: myorg/data-pipeline:latest
command: [python, run.py]
args: ["--stage", "{{inputs.parameters.stage}}"]extract runs first. transform-users and transform-orders run in parallel. load runs after both transforms complete.
Artifacts
Pass data between steps:
templates:
- name: generate-data
container:
image: python:3.12
command: [python, -c]
args: ["import json; json.dump({'count': 42}, open('/tmp/data.json', 'w'))"]
outputs:
artifacts:
- name: data
path: /tmp/data.json
- name: process-data
inputs:
artifacts:
- name: data
path: /tmp/input.json
container:
image: python:3.12
command: [python, -c]
args: ["import json; d=json.load(open('/tmp/input.json')); print(f'Count: {d[\"count\"]}')"]Artifacts are stored in S3/GCS/Minio between steps.
Get weekly IT automation tips
Docker, Ansible, Terraform, MLOps — curated insights delivered to your inbox. No spam.
Subscribe Free →Retries and Timeouts
- name: flaky-step
retryStrategy:
limit: 3
retryPolicy: Always
backoff:
duration: "10s"
factor: 2
maxDuration: "1m"
activeDeadlineSeconds: 300
container:
image: myorg/processor:latestRetry up to 3 times with exponential backoff. Timeout after 5 minutes.
Cron Workflows
Schedule recurring workflows:
apiVersion: argoproj.io/v1alpha1
kind: CronWorkflow
metadata:
name: nightly-etl
spec:
schedule: "0 2 * * *"
timezone: "Europe/Rome"
workflowSpec:
entrypoint: pipeline
templates:
- name: pipeline
dag:
tasks:
# ... same as aboveArgo Workflows vs Alternatives
| Tool | Strength | Limitation |
|---|---|---|
| Argo Workflows | K8s-native, DAGs, artifacts | Requires Kubernetes |
| Apache Airflow | Mature, Python DSL, rich ecosystem | Heavy, not K8s-native |
| Tekton | CI/CD focused, K8s-native | Less suited for data pipelines |
| GitHub Actions | SaaS, easy to start | Not self-hosted, limited DAGs |
Choose Argo Workflows when you need complex, Kubernetes-native workflow orchestration with parallel execution and artifact passing.
---
Ready to go deeper? Master Kubernetes orchestration with hands-on courses at CopyPasteLearn.
Ready to learn by doing?
Stop reading tutorials — start building. Expert video courses with hands-on labs in real sandboxed environments.
Related Articles
Argo Events Kubernetes Automation
Argo Events triggers Kubernetes workflows from webhooks, message queues, S3 uploads, and cron schedules. Learn how to build event-driven automation pipelines.
Tekton Cloud Native CI/CD
Tekton runs CI/CD pipelines as Kubernetes custom resources. Learn how Tekton works, how to build pipelines with Tasks and Pipelines, and when to choose it.
Kubebuilder Custom Operators Guide
Kubebuilder scaffolds Kubernetes operators in Go. Learn how to create custom controllers, define CRDs, and build operators that automate complex application.
Autonomous Industrial Systems
Deploy and manage autonomous industrial systems with IIoT platforms, digital twins, predictive maintenance, and industrial DevOps practices.
AWS IAM Roles and Policies Guide
Understand AWS IAM roles, policies, and best practices for secure cloud access. Least privilege principles, role assumption, and Terraform IAM automation.
Backstage Developer Portal Guide
Spotify Backstage is the most popular open-source developer portal. Learn how to set it up, create software templates, build a service catalog, and integrate.
Explore topics
Browse more articles on the topics covered here.