snakemake 6.8.0 → CWL

From @sofiakarb

The issue I am currenlty facing has to do with Snakemake (v:6.8.0) - cwltool (v:3.1.20210521105815). More specifically, my Snakefile is the following:

from snakemake.remote.HTTP import RemoteProvider as HTTPRemoteProvider
HTTP = HTTPRemoteProvider()

rule workflow:
    input:
        input_file=HTTP.remote("remote_link", keep_local=True)
    params:
        channels=[0, 1, 2, 3, 4],
        psd_output_file_name="psd.json",
        output_file_name="output.png"
    output:
        output="output.png"
    cwl:
        "file:workflow.cwl"

While executing snakemake -c 1, I get an error related to cwltool :

INFO /../bin/cwltool 3.1.20210521105815
ERROR Workflow error, try again with --debug for more information:
Invalid job input record:
../../../../tmp/tmpw06vyrgf:1:2: the `input_file` field is not valid because
                                   is not a dict

This is because .yml file that holds the information about Class: File is skipped due to the fact that Snakemake only needs the Snakefile (in which all input, parameters, outputs and cwl workflow file are provided). I can see two solutions, one from Snakemake side, where I must specificly provide the Class:File in the input_file field, and the other solution having a way cwltool to not provide errors (maybe just a warning?)

Thank you beforehand for your support.

_Originally posted by @sofiakarb in https://github.com/common-workflow-language/cwltool/issues/758#issuecomment-951760232_

[sofiakarb]

Dear @mr-c, sorry for the late response. Our workflow definition is under an internal gitlab repo so it will not be possible for you to reproduce it -for now. I will try to find some time to work an a simpler example and let you know. For now the solution previously shared seems to work fine

[sofiakarb]

A solution (temporal?) that worked for us is the following:

rule workflow:
    input:
    params:
        input_file={"class": "File", "path": "https://object.cscs.ch/v1/AUTH_25b4e28a742d4987a7b6f84c0c36512e/hbp-01852/data-originals/sub-1/hbp-01852_sub-1_chs-32_hem-RH_ana-ISO000_stim-SPN.smr"},
        channels=[0, 1, 2, 3, 4],
        psd_output_file_name="psd.json",
        output_file_name="output.png"
    output:
        output="output.png"
    cwl:
        "file:workflow.cwl"

We removed the HTTPRemoteProvider, and placed everything inside the params: (including the input_file). This remote location is then taking care of by the cwltool which also sees the correct Class:File and stops the error.