Skip to content

build: do not run benchmark tests on 'make test' #34434

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Trott merged 8 commits into from
Jul 22, 2020
Merged

build: do not run benchmark tests on 'make test' #34434

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
b0b52b2
test: fix flaky test-watch-file
Trott Jul 18, 2020
f8eaeb0
zlib: switch to lazy init for zlib streams
puzpuzpuz Jun 23, 2020
07b3aae
doc: add HarshithaKP to collaborators
HarshithaKP Jul 18, 2020
f4f191b
build: define NODE_EXPERIMENTAL_QUIC in mkcodecache and node_mksnapshot
joyeecheung Jul 21, 2020
7603c7e
worker: set trackUnmanagedFds to true by default
addaleax Jul 16, 2020
bfef588
doc: add author-ready label ref to onboarding doc
ruyadorno Jul 15, 2020
90ee98a
2020-07-22, Version 12.18.3 'Erbium' (LTS)
codebytere Jul 13, 2020
3caa2e2
build: do not run benchmark tests on 'make test'
Trott Jul 20, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
build: do not run benchmark tests on 'make test' 
Fixes: #34427

PR-URL: #34434
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
  • Loading branch information
@Trott
Trott committed Jul 22, 2020
commit 3caa2e256bd403e0fb06cfbefc0338462f2c60d9
14 changes: 8 additions & 6 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -298,8 +298,8 @@ v8:
jstest: build-addons build-js-native-api-tests build-node-api-tests ## Runs addon tests and JS tests
$(PYTHON) tools/test.py $(PARALLEL_ARGS) --mode=$(BUILDTYPE_LOWER) \
--skip-tests=$(CI_SKIP_TESTS) \
$(CI_JS_SUITES) \
$(CI_NATIVE_SUITES)
$(JS_SUITES) \
$(NATIVE_SUITES)
Comment on lines 301 to 302

@richardlau richardlau Jul 20, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW the reason we split tests into CI_JS_SUITES and CI_NATIVE_SUITES was that to accommodate the arm-fanned setup we split the tests into test-ci-native (requires a build toolchain set up to build the native addons) and test-ci-js (which "should not use a native compiler at all" (see comment in Makefile)).

For testing outside of the CI we don't have the same split in Makefile test targets, so you could optionally have a single make variable instead of two.


.PHONY: tooltest
tooltest:
Expand Down Expand Up @@ -492,9 +492,11 @@ test-all-valgrind: test-build
test-all-suites: | clear-stalled test-build bench-addons-build doc-only ## Run all test suites.
$(PYTHON) tools/test.py $(PARALLEL_ARGS) --mode=$(BUILDTYPE_LOWER) test/*

JS_SUITES ?= default
NATIVE_SUITES ?= addons js-native-api node-api
# CI_* variables should be kept synchronized with the ones in vcbuild.bat
CI_NATIVE_SUITES ?= addons js-native-api node-api
CI_JS_SUITES ?= default benchmark
CI_NATIVE_SUITES ?= $(NATIVE_SUITES)
CI_JS_SUITES ?= $(JS_SUITES) benchmark
ifeq ($(node_use_openssl), false)
CI_DOC := doctool
else
Expand Down Expand Up @@ -654,8 +656,8 @@ test-with-async-hooks:
$(MAKE) build-node-api-tests
$(MAKE) cctest
NODE_TEST_WITH_ASYNC_HOOKS=1 $(PYTHON) tools/test.py $(PARALLEL_ARGS) --mode=$(BUILDTYPE_LOWER) \
$(CI_JS_SUITES) \
$(CI_NATIVE_SUITES)
$(JS_SUITES) \
$(NATIVE_SUITES)


.PHONY: test-v8
Expand Down
8 changes: 5 additions & 3 deletions vcbuild.bat
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,14 @@ if /i "%1"=="/?" goto help

cd %~dp0

set JS_SUITES=default
set NATIVE_SUITES=addons js-native-api node-api
@rem CI_* variables should be kept synchronized with the ones in Makefile
set CI_NATIVE_SUITES=addons js-native-api node-api
set CI_JS_SUITES=default benchmark
set "CI_NATIVE_SUITES=%NATIVE_SUITES%"
set "CI_JS_SUITES=%JS_SUITES% benchmark"
set CI_DOC=doctool
@rem Same as the test-ci target in Makefile
set "common_test_suites=%CI_JS_SUITES% %CI_NATIVE_SUITES% %CI_DOC%&set build_addons=1&set build_js_native_api_tests=1&set build_node_api_tests=1"
set "common_test_suites=%JS_SUITES% %NATIVE_SUITES%&set build_addons=1&set build_js_native_api_tests=1&set build_node_api_tests=1"

@rem Process arguments.
set config=Release
Expand Down