cwltest.utils

Classes

`CWLTestConfig`	Store configuration values for cwltest.
`CWLTestReport`	Encapsulate relevant test result data for a markdown report.
`TestResult`	Encapsulate relevant test result data.

Functions

`generate_badges`(badgedir, ntotal, npassed, nfailures, ...)	Generate badges with conformance levels.
`get_test_number_by_key`(tests, key, value)	Retrieve the test index from its name.
`load_and_validate_tests`(path)	Load and validate the given test file against the cwltest schema.
`parse_results`(results, tests[, suite_name, report])	Parse the results and return statistics and an optional report.
`prepare_test_command`(tool, args, testargs, test, cwd)	Turn the test into a command line.
`prepare_test_paths`(test, cwd)	Determine the test path and the tool path.
`run_test_plain`(config, test[, test_number])	Plain test runner.
`shortname`(name)	Return the short name of a given name.
`absuri`(path)	Return an absolute URI.
`load_optional_fsaccess_plugin`()	Load optional fsaccess plugin.

Module Contents

class cwltest.utils.CWLTestConfig(entry, entry_line, basedir=None, test_baseuri=None, test_basedir=None, outdir=None, classname=None, tool=None, args=None, testargs=None, timeout=None, verbose=None, runner_quiet=None)

Store configuration values for cwltest.

Parameters:

entry (str)
entry_line (str)
basedir (str | None)
test_baseuri (str | None)
test_basedir (str | None)
outdir (str | None)
classname (str | None)
tool (str | None)
args (list[str] | None)
testargs (list[str] | None)
timeout (int | None)
verbose (bool | None)
runner_quiet (bool | None)

basedir: str

test_baseuri: str

test_basedir: str

outdir: str | None = None

classname: str = ''

entry

tool: str = 'cwl-runner'

args: list[str] = []

testargs: list[str] = []

timeout: int | None = None

verbose: bool = False

runner_quiet: bool = True

class cwltest.utils.CWLTestReport(id, category, entry, tool, job)

Encapsulate relevant test result data for a markdown report.

Parameters:

id (int | str)
category (list[str])
entry (str)
tool (str)
job (str | None)

id

category

entry

tool

job

class cwltest.utils.TestResult(return_code, standard_output, error_output, duration, classname, entry, tool, job, message='')

Encapsulate relevant test result data.

Parameters:

return_code (int)
standard_output (str)
error_output (str)
duration (float)
classname (str)
entry (str)
tool (str)
job (str | None)
message (str)

return_code

standard_output

error_output

duration

message = ''

classname

entry

tool

job

create_test_case(test)

Create a jUnit XML test case from this test result.

Parameters:: test (dict[str, Any])
Return type:: junit_xml.TestCase

create_report_entry(test)

Package test result into a CWLTestReport.

Parameters:: test (dict[str, Any])
Return type:: CWLTestReport

cwltest.utils.generate_badges(badgedir, ntotal, npassed, nfailures, nunsupported)

Generate badges with conformance levels.

Parameters:

badgedir (str)
ntotal (dict[str, int])
npassed (dict[str, list[CWLTestReport]])
nfailures (dict[str, list[CWLTestReport]])
nunsupported (dict[str, list[CWLTestReport]])

Return type:

None

cwltest.utils.get_test_number_by_key(tests, key, value)

Retrieve the test index from its name.

Parameters:

tests (list[dict[str, str]])
key (str)
value (str)

Return type:

int | None

cwltest.utils.load_and_validate_tests(path)

Load and validate the given test file against the cwltest schema.

This also processes $import directives.

Parameters:: path (str)
Return type:: tuple[Any, dict[str, Any]]

cwltest.utils.parse_results(results, tests, suite_name=None, report=None)

Parse the results and return statistics and an optional report.

An additional tag named “all” will be computed, containing all the test results.

Returns the total number of tests, dictionary of test counts (total, passed, failed, unsupported) by tag, and a jUnit XML report.

Parameters:

results (collections.abc.Iterable[TestResult])
tests (list[dict[str, Any]])
suite_name (str | None)
report (junit_xml.TestSuite | None)

Return type:

tuple[int, int, int, int, dict[str, int], dict[str, list[CWLTestReport]], dict[str, list[CWLTestReport]], dict[str, list[CWLTestReport]], junit_xml.TestSuite | None]

cwltest.utils.prepare_test_command(tool, args, testargs, test, cwd, quiet=True)

Turn the test into a command line.

Parameters:

tool (str)
args (list[str])
testargs (list[str] | None)
test (dict[str, Any])
cwd (str)
quiet (bool | None)

Return type:

list[str]

cwltest.utils.prepare_test_paths(test, cwd)

Determine the test path and the tool path.

Parameters:

test (dict[str, str])
cwd (str)

Return type:

tuple[str, str | None]

cwltest.utils.run_test_plain(config, test, test_number=None)

Plain test runner.

Parameters:

config (CWLTestConfig)
test (dict[str, str])
test_number (int | None)

Return type:

TestResult

cwltest.utils.shortname(name)

Return the short name of a given name.

It is a workaround of https://github.com/common-workflow-language/schema_salad/issues/511.

Parameters:: name (str)
Return type:: str

cwltest.utils.absuri(path)

Return an absolute URI.

Parameters:: path (str)
Return type:: str

cwltest.utils.load_optional_fsaccess_plugin()

Load optional fsaccess plugin.

Looks for a package with cwltest.fsaccess entry point and if so, use that to get a filesystem access object that will be used for checking test output.

Return type:: None