lit - LLVM Integrated Tester#

SYNOPSIS#

lit [options] [tests]

DESCRIPTION#

lit is a portable tool for executing LLVM and Clang style test suites, summarizing their results, and providing indication of failures. lit is designed to be a lightweight testing tool with as simple a user interface as possible.

lit should be run with one or more tests to run specified on the command line. Tests can be either individual test files or directories to search for tests (see TEST DISCOVERY).

Each specified test will be executed (potentially concurrently) and once all tests have been run lit will print summary information on the number of tests which passed or failed (see TEST STATUS RESULTS). The lit program will execute with a non-zero exit code if any tests fail.

By default lit will use a succinct progress display and will only print summary information for test failures. See OUTPUT OPTIONS for options controlling the lit progress display and output.

lit also includes a number of options for controlling how tests are executed (specific features may depend on the particular test format). See EXECUTION OPTIONS for more information.

Finally, lit also supports additional options for only running a subset of the options specified on the command line, see SELECTION OPTIONS for more information.

lit parses options from the environment variable LIT_OPTS after parsing options from the command line. LIT_OPTS is primarily useful for supplementing or overriding the command-line options supplied to lit by check targets defined by a project’s build system.

lit can also read options from response files which are specified as inputs using the @path/to/file.rsp syntax. Arguments read from a file must be one per line and are treated as if they were in the same place as the original file referencing argument on the command line. A response file can reference other response files.

Users interested in the lit architecture or designing a lit testing implementation should see LIT INFRASTRUCTURE.

GENERAL OPTIONS#

-h, --help#: Show the lit help message and exit.

--version#: Show lit’s version number and exit.

-j N, --workers N#: Run N tests in parallel. By default, this is automatically chosen to match the number of detected available CPUs.

--config-prefix NAME#: Search for NAME.cfg and NAME.site.cfg when searching for test suites, instead of lit.cfg and lit.site.cfg.

-D NAME[=VALUE], --param NAME[=VALUE]#: Add a user defined parameter NAME with the given VALUE (or the empty string if not given). The meaning and use of these parameters is test suite dependent.

OUTPUT OPTIONS#

-q, --quiet#: Suppress any output except for test failures.

-s, --succinct#: Show less output, for example don’t show information on tests that pass. Also show a progress bar, unless --no-progress-bar is specified.

-v, --verbose#

Show more information on test failures, for example the entire test output instead of just the test result.

Each command is printed before it is executed. This can be valuable for debugging test failures, as the last printed command is the one that failed. Moreover, lit inserts 'RUN: at line N' after each command pipeline in the output to help you locate the source line of the failed command.

-vv, --echo-all-commands#: Deprecated alias for -v.

-a, --show-all#: Enable -v, but for all tests not just failed tests.

-o PATH, --output PATH#: Write test results to the provided path.

--no-progress-bar#: Do not use curses based progress bar.

--min-output-interval INTERVAL#: Only output updates to the progress bar and status line at most once per INTERVAL seconds. Has no effect if the curses based progress bar is not used.

--show-excluded#: Show excluded tests.

--show-skipped#: Show skipped tests.

--show-unsupported#: Show unsupported tests.

--show-pass#: Show passed tests.

--show-flakypass#: Show passed with retry tests.

--show-xfail#: Show expectedly failed tests.

EXECUTION OPTIONS#

--gtest-sharding#: Enable sharding for GoogleTest format.

--no-gtest-sharding#: Disable sharding for GoogleTest format.

--path PATH#: Specify an additional PATH to use when searching for executables in tests.

--pass-env NAME#: Pass the environment variable NAME through to the test environment, in addition to the built-in allow-list of variables that are always passed through. May be specified multiple times to pass through several variables.

--vg#

Run individual tests under valgrind (using the memcheck tool). The --error-exitcode argument for valgrind is used so that valgrind failures will cause the program to exit with a non-zero status.

When this option is enabled, lit will also automatically provide a “valgrind” feature that can be used to conditionally disable (or expect failure in) certain tests.

--vg-leak#: When --vg is used, enable memory leak checks. When this option is enabled, lit will also automatically provide a “vg_leak” feature that can be used to conditionally disable (or expect failure in) certain tests.

--vg-arg ARG#: When --vg is used, specify an additional argument to pass to valgrind itself.

--no-execute#: Don’t execute any tests (assume that they pass).

--xunit-xml-output XUNIT_XML_OUTPUT#: Write XUnit-compatible XML test reports to the specified file.

--report-failures-only#: Only include failures (see TEST STATUS RESULTS) in the report.

--resultdb-output RESULTDB_OUTPUT#: Write LuCI ResultDB compatible JSON to the specified file.

--time-trace-output TIME_TRACE_OUTPUT#: Write Chrome tracing compatible JSON to the specified file

--timeout MAXINDIVIDUALTESTTIME#: Maximum time to spend running a single test (in seconds). 0 means no time limit. [Default: 0]

--timeout N