Unexpected interaction between local() inputs, Slurm execution, and fs staging in Snakemake

[jmzhang1911]

Software Versions

snakemake                               9.23.0
snakemake-executor-plugin-slurm         2.7.1
snakemake-executor-plugin-slurm-jobstep 0.6.1
snakemake-interface-common              1.23.0
snakemake-interface-executor-plugins    9.4.0
snakemake-interface-logger-plugins      2.1.0
snakemake-interface-report-plugins      1.3.0
snakemake-interface-scheduler-plugins   2.0.2
snakemake-interface-storage-plugins     4.4.1
snakemake-storage-plugin-fs             1.1.3
slurm-wlm 23.11.4

Describe the bug
when using Slurm together with fs, I observed that if a rule’s input contains local(), the Slurm job is not properly submitted for execution. My use case is that I would like fs to bypass staging for certain large files, since these files are already pre-deployed on the local storage of all compute nodes with identical directory structures, and therefore do not need to be transferred or staged.

Is this the intended and expected behavior, or does local() trigger an unintended change in execution strategy in this combination? I would also appreciate any suggestions on the correct or recommended way to represent this kind of node-local but globally consistent storage setup in Snakemake, in order to avoid unnecessary staging while still ensuring correct and reliable execution on Slurm.

Logs

Minimal example

touch a.txt
touch b.txt

rule all:
    input:
        expand("res/a_{i}.txt", i=range(1)),


rule a:
    input:
        "b.txt",
        local("a.txt"), # assuming that the file is accessible at this path on all compute nodes
    output:
        "res/a_{i}.txt",
    threads: 1
    shell:
        """
        sleep 30
        echo hello >{output}
        """

executor: slurm
default-storage-provider: fs
remote-job-local-storage-prefix: /share/$USER/scratch/$SLURM_JOB_ID
local-storage-prefix: /data/$USER/scratch/$SLURM_JOB_ID
shared-fs-usage:
  - persistence
  - software-deployment
  - sources
  - source-cache

Additional context