Skip to content

Commit

Permalink
Improve snakemake mindset
Browse files Browse the repository at this point in the history
  • Loading branch information
johnbradley committed Mar 8, 2023
1 parent 2ff9d95 commit 5fcf317
Show file tree
Hide file tree
Showing 2 changed files with 28 additions and 6 deletions.
29 changes: 26 additions & 3 deletions episodes/02_snakemake_mindset.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ teaching: 10
- What are the parts of a snakemake workflow?
- What is a snakemake rule?
- How does snakemake combine rules into jobs to run?
- How is a Snakemake Rule different from a Snakemake Job?

::::::::::::::::::::::::::::::::::::::::::::::::
::::::::::::::::::::::::::::::::::::: objectives
Expand All @@ -25,9 +26,6 @@ teaching: 10
- Provides support for specifying containers for each rule
- Provides support for many HPC Clusters including SLURM

TODO
Explain DAG

## What is a Snakemake Rule?

Main parts of a Snakemake Rule used in this workshop
Expand All @@ -51,6 +49,31 @@ rule <name>:
```

## How Snakemake combines rules
### Example workflow

Create checksums for each image and zip up the results
```
fish1.jpg
fish2.jpg
```
rule checksum1:
input: "fish1.jpg"
output: "fish1.checksum"
...

rule checksum2:
input: "fish2.jpg"
output: "fish2.checksum"
...

rule zip:
input: "fish1.jpg", "fish1.checksum", "fish2.jpg", "fish2.checksum"
ouput: "result.zip"
...
```
Explain DAG
DAG (directed acyclic graph)
Rules vs Jobs
Job is running a rule to create a specific set of output files.
5 changes: 2 additions & 3 deletions episodes/12_run_on_slurm.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Create a script name `run-workflow.sh` to run snakemake in the background.
#SBATCH --time=00:30:00
. Scripts/setup_env.sh
JOBS=10
snakemake --jobs $JOBS --use-singularity --profile slurm/ "$@"
snakemake --jobs $JOBS --use-singularity --profile slurm/
```

## Delete All Outputs
Expand All @@ -66,7 +66,7 @@ snakemake c1 --delete-all-output
## Run Background job and monitor progress
Run snakemake in the background scaling up
```bash
sbatch run-workflow.sh -c10
sbatch run-workflow.sh
```

## Monitor job
Expand All @@ -78,6 +78,5 @@ squeue -u $LOGNAME

Where did my logs go?
```bash
ls logs
ls logs/
```

0 comments on commit 5fcf317

Please sign in to comment.