mem setting in resource is not working due to default mem_mb argument is generated in latest version of snakemake #107
Description
The files to reproduce this bug is as follow:
- Snakemake
rule all:
input:
"out.txt",
rule run:
input:
"test.txt",
output:
"out.txt",
threads: 2
resources:
mem="10G",
shell:
"cat {input} > {output}"
- test.txt
any content...
Run snakemake with latest slurm profile,
snakemake --profile slurm -p -n
the log is as follow:
[Wed Nov 16 04:38:53 2022]
rule run:
input: test.txt
output: out.txt
jobid: 1
reason: Missing output files: out.txt
threads: 2
resources: mem_mb=1000, disk_mb=1000, tmpdir=<TBD>, mem=10G
cat test.txt > out.txt
[Wed Nov 16 04:38:53 2022]
localrule all:
input: out.txt
jobid: 0
reason: Input files updated by another job: out.txt
resources: mem_mb=<TBD>, disk_mb=<TBD>, tmpdir=/tmp
We can notice that the mem_mb argument is set as 1000M, because snakemake defines default resources ,'mem_mb=max(2*input.size_mb, 1000)' in the latest version (7.18.2). And slurm profile submit jobs with 1G memory instead of the 10G memory in the customized config. I think this change take place of version 7.10 of even earlier. When switch to snakemake version 7.0. The output is as follow. No default mem_mb is set and slurm job work as expected.
[Wed Nov 16 04:53:08 2022]
rule run:
input: test.txt
output: out.txt
jobid: 1
threads: 2
resources: tmpdir=/tmp, mem=10G
cat test.txt > out.txt
[Wed Nov 16 04:53:08 2022]
localrule all:
input: out.txt
jobid: 0
resources: tmpdir=/tmp