Skip to content

Commit

Permalink
Add example DAG for task group (#293)
Browse files Browse the repository at this point in the history
This PR adds an example for the task group. 
Depend on PR: #292

<img width="1678" alt="Screenshot 2024-11-19 at 6 23 42 PM"
src="https://github.com/user-attachments/assets/7203fc6a-940b-4a99-9fc2-c189b0fb70b6">
  • Loading branch information
pankajastro authored Nov 20, 2024
1 parent 7626fa6 commit 831a741
Show file tree
Hide file tree
Showing 2 changed files with 47 additions and 0 deletions.
17 changes: 17 additions & 0 deletions dev/dags/example_task_group.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import os
from pathlib import Path

# The following import is here so Airflow parses this file
# from airflow import DAG
import dagfactory

DEFAULT_CONFIG_ROOT_DIR = "/usr/local/airflow/dags/"
CONFIG_ROOT_DIR = Path(os.getenv("CONFIG_ROOT_DIR", DEFAULT_CONFIG_ROOT_DIR))

config_file = str(CONFIG_ROOT_DIR / "example_task_group.yml")

example_dag_factory = dagfactory.DagFactory(config_file)

# Creating task dependencies
example_dag_factory.clean_dags(globals())
example_dag_factory.generate_dags(globals())
30 changes: 30 additions & 0 deletions dev/dags/example_task_group.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
default:
default_args:
owner: default_owner
retries: 1
retry_delay_sec: 300
start_date: 2024-01-01
default_view: tree
max_active_runs: 1
schedule_interval: 0 1 * * *
example_task_group:
description: "this dag uses task groups"
task_groups:
task_group_1:
tooltip: "this is a task group"
dependencies: [task_1]
task_group_2:
tooltip: "this is a task group"
parent_group_name: task_group_1
tasks:
task_1:
operator: airflow.operators.bash_operator.BashOperator
bash_command: "echo 1"
task_2:
operator: airflow.operators.bash_operator.BashOperator
bash_command: "echo 2"
task_group_name: task_group_1
task_4:
operator: airflow.operators.bash_operator.BashOperator
bash_command: "echo 4"
task_group_name: task_group_2

0 comments on commit 831a741

Please sign in to comment.