Skip to content
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
9ec0a4e
test_runner: avoid swallowing of asynchronously thrown errors
fossamagna Nov 7, 2022
7201845
test_runner: add reporters
MoLow Dec 19, 2022
a34809a
module: move test reporter loading
GeoffreyBooth Dec 22, 2022
b628161
test_runner: report `file` in test runner events
MoLow Jan 2, 2023
6d9ec6c
test_runner: make built in reporters internal
cjihrig Jan 6, 2023
6cb8f02
test_runner: add initial code coverage support
cjihrig Dec 21, 2022
8ad7668
doc,lib,src,test: rename --test-coverage
cjihrig Jan 8, 2023
95250e1
test: fix tap escaping with and without --test
pulkit-30 Jan 28, 2023
e1f5a3a
test_runner: top-level diagnostics not ommited when running with --test
pulkit-30 Feb 2, 2023
f4264f0
test_runner: fix missing test diagnostics
MoLow Feb 2, 2023
9c49554
test: fix tap parser fails if a test logs a number
pulkit-30 Feb 6, 2023
1502aca
test_runner: allow nesting test within describe
MoLow Feb 17, 2023
f7d8bed
test_runner: flatten TAP output when running using `--test`
MoLow Feb 18, 2023
730f11f
test_runner: parse non-ascii character correctly
mertcanaltin Feb 18, 2023
4a63d24
test_runner: display skipped tests in spec reporter output
richiemccoll Feb 18, 2023
32a0802
test_runner: centralize CLI option handling
cjihrig Feb 20, 2023
b50cfb3
test_runner: emit test-only diagnostic warning
richiemccoll Feb 21, 2023
69100c9
test_runner: bootstrap reporters before running tests
MoLow Feb 19, 2023
6ef4886
test_runner: add `describe.only` and `it.only` shorthands
richiemccoll Feb 21, 2023
b7aefcb
test_runner: better handle async bootstrap errors
cjihrig Feb 24, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
doc,lib,src,test: rename --test-coverage
Add experimental to the name as requested during review.

PR-URL: #46017
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
  • Loading branch information
cjihrig authored and MoLow committed Feb 25, 2023
commit 8ad7668b8c8850b446c047f1fecd4f660c22b6bd
22 changes: 11 additions & 11 deletions doc/api/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -440,6 +440,17 @@ the ability to import a directory that has an index file.

See [customizing ESM specifier resolution][] for example usage.

### `--experimental-test-coverage`

<!-- YAML
added: REPLACEME
-->

When used in conjunction with the `node:test` module, a code coverage report is
generated as part of the test runner output. If no tests are run, a coverage
report is not generated. See the documentation on
[collecting code coverage from tests][] for more details.

### `--experimental-vm-modules`

<!-- YAML
Expand Down Expand Up @@ -1209,17 +1220,6 @@ Starts the Node.js command line test runner. This flag cannot be combined with
See the documentation on [running tests from the command line][]
for more details.

### `--test-coverage`

<!-- YAML
added: REPLACEME
-->

When used in conjunction with the `node:test` module, a code coverage report is
generated as part of the test runner output. If no tests are run, a coverage
report is not generated. See the documentation on
[collecting code coverage from tests][] for more details.

### `--test-name-pattern`

<!-- YAML
Expand Down
20 changes: 11 additions & 9 deletions doc/api/test.md
Original file line number Diff line number Diff line change
Expand Up @@ -370,13 +370,14 @@ internally.

## Collecting code coverage

When Node.js is started with the [`--test-coverage`][] command-line flag, code
coverage is collected and statistics are reported once all tests have completed.
If the [`NODE_V8_COVERAGE`][] environment variable is used to specify a
code coverage directory, the generated V8 coverage files are written to that
directory. Node.js core modules and files within `node_modules/` directories
are not included in the coverage report. If coverage is enabled, the coverage
report is sent to any [test reporters][] via the `'test:coverage'` event.
When Node.js is started with the [`--experimental-test-coverage`][]
command-line flag, code coverage is collected and statistics are reported once
all tests have completed. If the [`NODE_V8_COVERAGE`][] environment variable is
used to specify a code coverage directory, the generated V8 coverage files are
written to that directory. Node.js core modules and files within
`node_modules/` directories are not included in the coverage report. If
coverage is enabled, the coverage report is sent to any [test reporters][] via
the `'test:coverage'` event.

Coverage can be disabled on a series of lines using the following
comment syntax:
Expand Down Expand Up @@ -411,7 +412,8 @@ which will be addressed in a future Node.js release:

* Although coverage data is collected for child processes, this information is
not included in the coverage report. Because the command line test runner uses
child processes to execute test files, it cannot be used with `--test-coverage`.
child processes to execute test files, it cannot be used with
`--experimental-test-coverage`.
* Source maps are not supported.
* Excluding specific files or directories from the coverage report is not
supported.
Expand Down Expand Up @@ -1634,7 +1636,7 @@ added: v18.7.0
aborted.

[TAP]: https://testanything.org/
[`--test-coverage`]: cli.md#--test-coverage
[`--experimental-test-coverage`]: cli.md#--experimental-test-coverage
[`--test-name-pattern`]: cli.md#--test-name-pattern
[`--test-only`]: cli.md#--test-only
[`--test-reporter-destination`]: cli.md#--test-reporter-destination
Expand Down
6 changes: 3 additions & 3 deletions doc/node.1
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,9 @@ Use the specified file as a security policy.
.It Fl -experimental-shadow-realm
Use this flag to enable ShadowRealm support.
.
.It Fl -experimental-test-coverage
Enable code coverage in the test runner.
.
.It Fl -no-experimental-fetch
Disable experimental support for the Fetch API.
.
Expand Down Expand Up @@ -391,9 +394,6 @@ Specify the minimum allocation from the OpenSSL secure heap. The default is 2. T
.It Fl -test
Starts the Node.js command line test runner.
.
.It Fl -test-coverage
Enable code coverage in the test runner.
.
.It Fl -test-name-pattern
A regular expression that configures the test runner to only execute tests
whose name matches the provided pattern.
Expand Down
6 changes: 4 additions & 2 deletions lib/internal/process/pre_execution.js
Original file line number Diff line number Diff line change
Expand Up @@ -285,8 +285,10 @@ function setupCodeCoverage() {
// Resolve the coverage directory to an absolute path, and
// overwrite process.env so that the original path gets passed
// to child processes even when they switch cwd. Don't do anything if the
// --test-coverage flag is present, as the test runner will handle coverage.
if (process.env.NODE_V8_COVERAGE && !getOptionValue('--test-coverage')) {
// --experimental-test-coverage flag is present, as the test runner will
// handle coverage.
if (process.env.NODE_V8_COVERAGE &&
!getOptionValue('--experimental-test-coverage')) {
process.env.NODE_V8_COVERAGE =
setupCoverageHooks(process.env.NODE_V8_COVERAGE);
}
Expand Down
2 changes: 1 addition & 1 deletion lib/internal/test_runner/harness.js
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ function createProcessEventHandler(eventName, rootTest) {
}

function configureCoverage(rootTest) {
if (!getOptionValue('--test-coverage')) {
if (!getOptionValue('--experimental-test-coverage')) {
return null;
}

Expand Down
2 changes: 1 addition & 1 deletion lib/internal/test_runner/runner.js
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ const {
triggerUncaughtException,
} = internalBinding('errors');

const kFilterArgs = ['--test', '--test-coverage', '--watch'];
const kFilterArgs = ['--test', '--experimental-test-coverage', '--watch'];
const kFilterArgValues = ['--test-reporter', '--test-reporter-destination'];

// TODO(cjihrig): Replace this with recursive readdir once it lands.
Expand Down
5 changes: 3 additions & 2 deletions src/node_options.cc
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,8 @@ void EnvironmentOptions::CheckOptions(std::vector<std::string>* errors,
if (test_runner_coverage) {
// TODO(cjihrig): This restriction can be removed once multi-process
// code coverage is supported.
errors->push_back("--test-coverage cannot be used with --test");
errors->push_back(
"--experimental-test-coverage cannot be used with --test");
}

if (syntax_check_only) {
Expand Down Expand Up @@ -561,7 +562,7 @@ EnvironmentOptionsParser::EnvironmentOptionsParser() {
AddOption("--test",
"launch test runner on startup",
&EnvironmentOptions::test_runner);
AddOption("--test-coverage",
AddOption("--experimental-test-coverage",
"enable code coverage in the test runner",
&EnvironmentOptions::test_runner_coverage);
AddOption("--test-name-pattern",
Expand Down
14 changes: 8 additions & 6 deletions test/parallel/test-runner-coverage.js
Original file line number Diff line number Diff line change
Expand Up @@ -37,15 +37,17 @@ function getCoverageFixtureReport() {
return report;
}

test('--test-coverage and --test cannot be combined', () => {
test('--experimental-test-coverage and --test cannot be combined', () => {
// TODO(cjihrig): This test can be removed once multi-process code coverage
// is supported.
const result = spawnSync(process.execPath, ['--test', '--test-coverage']);
const args = ['--test', '--experimental-test-coverage'];
const result = spawnSync(process.execPath, args);

// 9 is the documented exit code for an invalid CLI argument.
assert.strictEqual(result.status, 9);
assert.match(
result.stderr.toString(), /--test-coverage cannot be used with --test/
result.stderr.toString(),
/--experimental-test-coverage cannot be used with --test/
);
});

Expand All @@ -55,7 +57,7 @@ test('handles the inspector not being available', (t) => {
}

const fixture = fixtures.path('test-runner', 'coverage.js');
const args = ['--test-coverage', fixture];
const args = ['--experimental-test-coverage', fixture];
const result = spawnSync(process.execPath, args);

assert(!result.stdout.toString().includes('# start of coverage report'));
Expand All @@ -70,7 +72,7 @@ test('coverage is reported and dumped to NODE_V8_COVERAGE if present', (t) => {
}

const fixture = fixtures.path('test-runner', 'coverage.js');
const args = ['--test-coverage', fixture];
const args = ['--experimental-test-coverage', fixture];
const options = { env: { ...process.env, NODE_V8_COVERAGE: tmpdir.path } };
const result = spawnSync(process.execPath, args, options);
const report = getCoverageFixtureReport();
Expand All @@ -87,7 +89,7 @@ test('coverage is reported without NODE_V8_COVERAGE present', (t) => {
}

const fixture = fixtures.path('test-runner', 'coverage.js');
const args = ['--test-coverage', fixture];
const args = ['--experimental-test-coverage', fixture];
const result = spawnSync(process.execPath, args);
const report = getCoverageFixtureReport();

Expand Down