Error installing a patched version of cwltools -- for a Singularity + MPIRequirement fix

[kinow]

Hi,

There is a pull request in the cwltool repository, common-workflow-language/cwltool#2216, that detects when the tool or workflow uses the cwltool:MPIRequirement with Singularity. That pull request is needed to use a wrapper instead of executing Singularity directly.

The wrapper script gets a list of env vars in the environment before, and after launching mpirun, and then passes the different (i.e., the env vars added by MPI) to cwltool. This is necessary as cwltool uses the --cleanenv and other flags to isolate the execution, but without the MPI env vars the MPI server and ranks fail to communicate.

I'm using Toil 9.4.1 on CSC LUMI,

toil-cwl-runner --version
9.4.1

and I tried to simply do a cd cwltool-folder-with-that-pullrequest-branch && pip install -e ..

But running a test workflow I'm getting the following error:

toil-cwl-runner mpich-mpirun-mpi-requirement.cwl
Traceback (most recent call last):
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/bin/cwl-docker-extract", line 6, in <module>
    sys.exit(main())
             ~~~~^^
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/docker_extract.py", line 140, in main
    run(arg_parser().parse_args(sys.argv[1:]))
    ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/docker_extract.py", line 56, in run
    top = cwl.load_document_by_uri(args.input)
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/parser/__init__.py", line 328, in load_document_by_uri
    return load_document_by_string(
        loadingOptions.fetcher.fetch_text(real_uri),
    ...<3 lines>...
        load_all,
    )
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/parser/__init__.py", line 365, in load_document_by_string
    return load_document_by_yaml(result, uri, loadingOptions, id_, load_all)
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/parser/__init__.py", line 390, in load_document_by_yaml
    result = cwl_v1_2.load_document_by_yaml(
        yaml, uri, cast(Optional[cwl_v1_2.LoadingOptions], loadingOptions)
    )
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/parser/cwl_v1_2.py", line 30779, in load_document_by_yaml
    result, metadata = _document_load(
                       ~~~~~~~~~~~~~~^
        union_of_CommandLineToolLoader_or_ExpressionToolLoader_or_WorkflowLoader_or_OperationLoader_or_ProcessGeneratorLoader_or_array_of_union_of_CommandLineToolLoader_or_ExpressionToolLoader_or_WorkflowLoader_or_OperationLoader_or_ProcessGeneratorLoader,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        loadingOptions,
        ^^^^^^^^^^^^^^^
    )
    ^
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/parser/cwl_v1_2.py", line 1040, in _document_load
    loader.load(doc2, baseuri, loadingOptions, docRoot=baseuri),
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/parser/cwl_v1_2.py", line 808, in load
    raise ValidationException("", None, errors, "*")
schema_salad.exceptions.ValidationException: mpich-mpirun-mpi-requirement.cwl:9:1:   Object `mpich-mpirun-mpi-requirement.cwl` is not valid
                                       because:
mpich-mpirun-mpi-requirement.cwl:51:1:   the `requirements` field is not valid because:
                                           Expected a <class 'NoneType'> but got list
Traceback (most recent call last):
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/bin/toil-cwl-runner", line 6, in <module>
    sys.exit(main())
             ~~~~^^
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/toil/cwl/cwltoil.py", line 4466, in main
    try_prepull(uri, runtime_context, expected_config.batchSystem)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/toil/cwl/cwltoil.py", line 289, in try_prepull
    call_command(["cwl-docker-extract", cwl_tool_uri])
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/toil/lib/misc.py", line 218, in call_command
    raise CalledProcessErrorStderr(
        proc.returncode, cmd, output=stdout, stderr=stderr
    )
toil.lib.misc.CalledProcessErrorStderr: Command '['cwl-docker-extract', 'file:///pfs/lustrep4/projappl/project/<USER>/cwl/cwl-mpi/workflows/mpich-sr/mpich-mpirun-mpi-requirement.cwl']' exit status 1: Traceback (most recent call last):
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/bin/cwl-docker-extract", line 6, in <module>
    sys.exit(main())
             ~~~~^^
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/docker_extract.py", line 140, in main
    run(arg_parser().parse_args(sys.argv[1:]))
    ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/docker_extract.py", line 56, in run
    top = cwl.load_document_by_uri(args.input)
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/parser/__init__.py", line 328, in load_document_by_uri
    return load_document_by_string(
        loadingOptions.fetcher.fetch_text(real_uri),
    ...<3 lines>...
        load_all,
    )
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/parser/__init__.py", line 365, in load_document_by_string
    return load_document_by_yaml(result, uri, loadingOptions, id_, load_all)
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/parser/__init__.py", line 390, in load_document_by_yaml
    result = cwl_v1_2.load_document_by_yaml(
        yaml, uri, cast(Optional[cwl_v1_2.LoadingOptions], loadingOptions)
    )
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/parser/cwl_v1_2.py", line 30779, in load_document_by_yaml
    result, metadata = _document_load(
                       ~~~~~~~~~~~~~~^
        union_of_CommandLineToolLoader_or_ExpressionToolLoader_or_WorkflowLoader_or_OperationLoader_or_ProcessGeneratorLoader_or_array_of_union_of_CommandLineToolLoader_or_ExpressionToolLoader_or_WorkflowLoader_or_OperationLoader_or_ProcessGeneratorLoader,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        loadingOptions,
        ^^^^^^^^^^^^^^^
    )
    ^
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/parser/cwl_v1_2.py", line 1040, in _document_load
    loader.load(doc2, baseuri, loadingOptions, docRoot=baseuri),
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/project/<USER>/cwl/cwl-mpi/cwl-conformance-tests/runs/venv-toil/lib/python3.13/site-packages/cwl_utils/parser/cwl_v1_2.py", line 808, in load
    raise ValidationException("", None, errors, "*")
schema_salad.exceptions.ValidationException: mpich-mpirun-mpi-requirement.cwl:9:1:   Object `mpich-mpirun-mpi-requirement.cwl` is not valid
                                       because:
mpich-mpirun-mpi-requirement.cwl:51:1:   the `requirements` field is not valid because:
                                           Expected a <class 'NoneType'> but got list

I think cwltool or cwl_utils may have changes that are not compatible with the current version of Toil?

Cheers

┆Issue is synchronized with this Jira Story
┆Issue Number: TOIL-1838

[kinow]

I'm using these two CWL tools:

https://github.com/kinow/cwl-mpi/blob/7828e3a537d18d1a7cfc9378832f81f3fed2c064/workflows/mpich-sr/mpich-mpicc.cwl

https://github.com/kinow/cwl-mpi/blob/7828e3a537d18d1a7cfc9378832f81f3fed2c064/workflows/mpich-sr/mpich-mpirun-base-command.cwl

With cwltool that works with cwltool --singularity mpich-mpicc.cwl, followed by cwltool --singularity mpich-mpirun-base-command.cwl.

[adamnovak]

I'm also seeing this problem with cwltool==3.2.20260413085819 which Toil depends on.

wget https://raw.githubusercontent.com/kinow/cwl-mpi/7828e3a537d18d1a7cfc9378832f81f3fed2c064/workflows/mpich-sr/mpich-mpirun-mpi-requirement.cwl
# With Toil 5.0 installed, this should do nothing
pip install --upgrade 'cwltool==3.2.20260413085819'
toil-cwl-runner mpich-mpirun-mpi-requirement.cwl

[2026-06-03T12:25:14-0400] [MainThread] [I] [cwltool] Resolved 'mpich-mpirun-mpi-requirement.cwl' to 'file:///Users/anovak/workspace/toil/mpich-mpirun-mpi-requirement.cwl'
[2026-06-03T12:25:14-0400] [MainThread] [I] [toil.cwl.cwltoil] Prepulling the workflow's containers with Docker...
Traceback (most recent call last):
  File "/Users/anovak/workspace/toil/venv/bin/cwl-docker-extract", line 8, in <module>
    sys.exit(main())
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/docker_extract.py", line 140, in main
    run(arg_parser().parse_args(sys.argv[1:]))
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/docker_extract.py", line 56, in run
    top = cwl.load_document_by_uri(args.input)
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/parser/__init__.py", line 328, in load_document_by_uri
    return load_document_by_string(
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/parser/__init__.py", line 365, in load_document_by_string
    return load_document_by_yaml(result, uri, loadingOptions, id_, load_all)
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/parser/__init__.py", line 390, in load_document_by_yaml
    result = cwl_v1_2.load_document_by_yaml(
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/parser/cwl_v1_2.py", line 30779, in load_document_by_yaml
    result, metadata = _document_load(
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/parser/cwl_v1_2.py", line 1040, in _document_load
    loader.load(doc2, baseuri, loadingOptions, docRoot=baseuri),
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/parser/cwl_v1_2.py", line 808, in load
    raise ValidationException("", None, errors, "*")
schema_salad.exceptions.ValidationException: mpich-mpirun-mpi-requirement.cwl:9:1:   Object `mpich-mpirun-mpi-requirement.cwl` is not valid
                                       because:
mpich-mpirun-mpi-requirement.cwl:51:1:   the `requirements` field is not valid because:
                                           Expected a <class 'NoneType'> but got list
Traceback (most recent call last):
  File "/Users/anovak/workspace/toil/venv/bin/toil-cwl-runner", line 6, in <module>
    sys.exit(main())
  File "/Users/anovak/workspace/toil/src/toil/cwl/cwltoil.py", line 4411, in main
    try_prepull(uri, runtime_context, expected_config.batchSystem)
  File "/Users/anovak/workspace/toil/src/toil/cwl/cwltoil.py", line 292, in try_prepull
    call_command(["cwl-docker-extract", cwl_tool_uri])
  File "/Users/anovak/workspace/toil/src/toil/lib/misc.py", line 218, in call_command
    raise CalledProcessErrorStderr(
toil.lib.misc.CalledProcessErrorStderr: Command '['cwl-docker-extract', 'file:///Users/anovak/workspace/toil/mpich-mpirun-mpi-requirement.cwl']' exit status 1: Traceback (most recent call last):
  File "/Users/anovak/workspace/toil/venv/bin/cwl-docker-extract", line 8, in <module>
    sys.exit(main())
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/docker_extract.py", line 140, in main
    run(arg_parser().parse_args(sys.argv[1:]))
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/docker_extract.py", line 56, in run
    top = cwl.load_document_by_uri(args.input)
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/parser/__init__.py", line 328, in load_document_by_uri
    return load_document_by_string(
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/parser/__init__.py", line 365, in load_document_by_string
    return load_document_by_yaml(result, uri, loadingOptions, id_, load_all)
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/parser/__init__.py", line 390, in load_document_by_yaml
    result = cwl_v1_2.load_document_by_yaml(
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/parser/cwl_v1_2.py", line 30779, in load_document_by_yaml
    result, metadata = _document_load(
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/parser/cwl_v1_2.py", line 1040, in _document_load
    loader.load(doc2, baseuri, loadingOptions, docRoot=baseuri),
  File "/Users/anovak/workspace/toil/venv/lib/python3.10/site-packages/cwl_utils/parser/cwl_v1_2.py", line 808, in load
    raise ValidationException("", None, errors, "*")
schema_salad.exceptions.ValidationException: mpich-mpirun-mpi-requirement.cwl:9:1:   Object `mpich-mpirun-mpi-requirement.cwl` is not valid
                                       because:
mpich-mpirun-mpi-requirement.cwl:51:1:   the `requirements` field is not valid because:
                                           Expected a <class 'NoneType'> but got list

[adamnovak]

I'm not able to replicate the patched cwltool understanding the MPIRequirement, either:

pip install "cwltool @ git+https://github.com/common-workflow-language/cwltool.git@detect-mpi-with-singularity"
cwltool mpich-mpirun-mpi-requirement.cwl

INFO /Users/anovak/workspace/toil/venv/bin/cwltool 3.2.20260413085820.dev6+gb776f8414
INFO Resolved 'mpich-mpirun-mpi-requirement.cwl' to 'file:///Users/anovak/workspace/toil/mpich-mpirun-mpi-requirement.cwl'
WARNING URI prefix 'cwltool' of 'cwltool:MPIRequirement' not recognized, are you missing a $namespaces section?
WARNING mpich-mpirun-mpi-requirement.cwl:75:7: Warning: Field 'path' contains undefined reference to
                                       'file:///Users/anovak/workspace/toil/sr'
ERROR Tool definition failed validation:
mpich-mpirun-mpi-requirement.cwl:51:1: checking field 'requirements'
mpich-mpirun-mpi-requirement.cwl:53:3:   checking item
                                           Unsupported scheme 'cwltool' in url:
                                           cwltool:MPIRequirement

[adamnovak]

Even if I fix the missing cwltool namespace definition and install cwltool from that PR, and run on Linux where MPI can actually exist, cwltool still can't parse the MPIRequirement in the CWL file that uses one:

pip install "cwltool @ git+https://github.com/common-workflow-language/cwltool.git@detect-mpi-with-singularity"
wget https://raw.githubusercontent.com/kinow/cwl-mpi/7828e3a537d18d1a7cfc9378832f81f3fed2c064/workflows/mpich-sr/mpich-mpirun-mpi-requirement.cwl
sed -i -e '\!xsd: http://www\.w3\.org/2001/XMLSchema!a\' -e'  cwltool: http://commonwl.org/cwltool#' mpich-mpirun-mpi-requirement.cwl
cwltool mpich-mpirun-mpi-requirement.cwl

INFO /private/home/anovak1/workspace/toil/venv/bin/cwltool 3.2.20260413085820.dev6+gb776f8414
INFO Resolved 'mpich-mpirun-mpi-requirement.cwl' to 'file:///private/home/anovak1/workspace/toil/mpich-mpirun-mpi-requirement.cwl'
WARNING mpich-mpirun-mpi-requirement.cwl:76:7: Warning: Field 'path' contains undefined reference to
                                       'file:///private/home/anovak1/workspace/toil/sr'
ERROR Tool definition failed validation:
mpich-mpirun-mpi-requirement.cwl:52:1: checking field 'requirements'
mpich-mpirun-mpi-requirement.cwl:54:3:   checking item
                                           Field 'class' contains undefined reference to
                                           'http://commonwl.org/cwltool#MPIRequirement'

[adamnovak]

Toil with that cwltool installed and the namespace line added produces the same error:

toil-cwl-runner mpich-mpirun-mpi-requirement.cwl

WARNING: You are using cwltool version 3.2.20260413085820.dev6+gb776f8414, which is not the version Toil is tested against. To install the correct cwltool for Toil, do:

	pip install cwltool==3.2.20260413085819

[2026-06-03T10:00:37-0700] [MainThread] [I] [cwltool] Resolved 'mpich-mpirun-mpi-requirement.cwl' to 'file:///private/home/anovak1/workspace/toil/mpich-mpirun-mpi-requirement.cwl'
[2026-06-03T10:00:37-0700] [MainThread] [I] [toil.cwl.cwltoil] Prepulling the workflow's containers with Docker...
Pulling mfisherman/mpich:4.3.2 with docker...
INFO:cwl_utils.image_puller:Pulling mfisherman/mpich:4.3.2 with docker...
Image successfully pulled: mfisherman/mpich:4.3.2
INFO:cwl_utils.image_puller:Image successfully pulled: mfisherman/mpich:4.3.2
Traceback (most recent call last):
  File "/private/home/anovak1/workspace/toil/venv/bin/toil-cwl-runner", line 8, in <module>
    sys.exit(main())
  File "/private/home/anovak1/workspace/toil/src/toil/cwl/cwltoil.py", line 4435, in main
    loading_context, uri = cwltool.load_tool.resolve_and_validate_document(
  File "/private/home/anovak1/workspace/toil/venv/lib/python3.10/site-packages/cwltool/load_tool.py", line 505, in resolve_and_validate_document
    document_loader.resolve_all(workflowobj, fileuri)
  File "schema_salad/ref_resolver.py", line 988, in resolve_all
  File "schema_salad/ref_resolver.py", line 1194, in validate_links
schema_salad.exceptions.ValidationException: mpich-mpirun-mpi-requirement.cwl:52:1: checking field 'requirements'
mpich-mpirun-mpi-requirement.cwl:54:3:   checking item
                                           Field 'class' contains undefined reference to
                                           'http://commonwl.org/cwltool#MPIRequirement'

It looks like the existence of http://commonwl.org/cwltool#MPIRequirement as a thing that can be loaded is somehow in doubt, and we end up at:
https://github.com/common-workflow-language/schema_salad/blob/d48545d85fbc8e5c468e47c305c134c9dcd0b2fb/src/schema_salad/metaschema.py#L815

We don't even make it to https://github.com/common-workflow-language/cwltool/blob/8949fc2d68efe128ce841739d1190645dbd233bb/cwltool/process.py#L1072 to check if the requirement is on the supported list, because we don't have a loader defined for this class of requirement to actually load it.

@mr-c As probably the resident schemas expert, do you have a sense of what has gone wrong here?

[kinow]

@adamnovak I think you need the --enable-ext in order to be able to use the cwltool:MPIRequirement. This might work,

cwltool --enable-ext mpich-mpirun-mpi-requirement.cwl

I'm just finishing some code in PyCharm, but I can try it as well to see what happens with toil. But it should work with the cwltool command above.

[adamnovak]

OK the problem is that extensions are not enabled by default in either cwltool or toil-cwl-runner. They both take an --enable-ext option to enable the extensions (but don't suggest that you use it when encountering something that might be a disabled extension).

If you do:

python3 -m venv testvenv
. testvenv/bin/activate
pip install 'toil[all]==9.5.0'
pip install "cwltool @ git+https://github.com/common-workflow-language/cwltool.git@detect-mpi-with-singularity"
wget https://raw.githubusercontent.com/kinow/cwl-mpi/7828e3a537d18d1a7cfc9378832f81f3fed2c064/workflows/mpich-sr/mpich-mpirun-mpi-requirement.cwl
sed -i -e '\!xsd: http://www\.w3\.org/2001/XMLSchema!a\' -e'  cwltool: http://commonwl.org/cwltool#' mpich-mpirun-mpi-requirement.cwl

And then either:

cwltool --enable-ext mpich-mpirun-mpi-requirement.cwl

Or:

toil-cwl-runner --enable-ext mpich-mpirun-mpi-requirement.cwl

Then the workflow parses fine and runs into the problem of the sr binary not being available.

I'm not sure why Toil doesn't run into the same problem with http://arvados.org/cwl#UsePreemptible and the tests we have for that. It could be that we only ever test it as a hint and the parser is willing to parse the YAML for hints as it lies rather than insisting everything have a registered class? Or it could be because the parser somehow knows that it ought to assign a class for the requirements known to cwltool, and also that that class is flagged off.

I'm going to close this, since as far as I can tell using the patched cwltool works fine in Toil.

But we should have an issue on cwltool for reporting that extensions need to be enabled to run a workflow, and one on Toil for enabling cwltool extensions by default, since we already support other available extensions by default.

[kinow]

Thank you, @adamnovak !!!