Skip to content

test_runner: add shards support #48639

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Jul 5, 2023
Merged

test_runner: add shards support #48639

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
a93fa41
test_runner: add shards support
rluvaton Jul 2, 2023
3bc36b8
test_runner: add validations
rluvaton Jul 2, 2023
bf9d2e5
test_runner: add tests for the shards
rluvaton Jul 2, 2023
3663fb2
reset unwanted changes
rluvaton Jul 2, 2023
09c8734
do not mutate array
rluvaton Jul 2, 2023
ed19709
update validation
rluvaton Jul 3, 2023
5e39d75
allow passing shards in CLI
rluvaton Jul 3, 2023
5c6ab88
test_runner: add cli support
rluvaton Jul 3, 2023
d293da5
test_runner: added docs
rluvaton Jul 3, 2023
51f7697
remove comment
rluvaton Jul 3, 2023
0625b75
format
rluvaton Jul 3, 2023
2e0040b
fix example
rluvaton Jul 3, 2023
602f767
test_runner: replace shards option with shard and --test-shard
rluvaton Jul 3, 2023
4ce4d76
test_runner: used stricter shard validation
rluvaton Jul 3, 2023
b7009d7
update node man page
rluvaton Jul 3, 2023
76cfcb6
test_runner: update based on CR
rluvaton Jul 3, 2023
5a846d2
test_runner: update based on the CR
rluvaton Jul 3, 2023
d4f9168
fix test
rluvaton Jul 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions doc/node.1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -424,6 +424,9 @@ The destination for the corresponding test reporter.
Configures the test runner to only execute top level tests that have the `only`
option set.
.
.It Fl -test-shard
Test suite shard to execute in a format of <index>/<total>.
.
.It Fl -throw-deprecation
Throw errors for deprecations.
.
Expand Down
12 changes: 7 additions & 5 deletions lib/internal/main/test_runner.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ const {
ERR_INVALID_ARG_VALUE,
},
} = require('internal/errors');
const { NumberParseInt } = primordials;
const { NumberParseInt, RegExpPrototypeExec, StringPrototypeSplit } =
primordials;

prepareMainThreadExecution(false);
markBootstrapComplete();
Expand All @@ -31,7 +32,7 @@ if (isUsingInspector()) {
let shard;
const shardOption = getOptionValue('--test-shard');
if (shardOption) {
if (!/^[1-9]([0-9]+)?\/[1-9]([0-9]+)?$/.test(shardOption)) {
if (!RegExpPrototypeExec(/^\d+\/\d+$/, shardOption)) {
process.exitCode = kGenericUserError;

throw new ERR_INVALID_ARG_VALUE(
Expand All @@ -41,12 +42,13 @@ if (shardOption) {
);
}

const { 0: indexStr, 1: totalStr } = shardOption.split('/');
const { 0: indexStr, 1: totalStr } = StringPrototypeSplit(shardOption, '/');

const index = NumberParseInt(indexStr);
const total = NumberParseInt(totalStr);
const index = NumberParseInt(indexStr, 10);
const total = NumberParseInt(totalStr, 10);
Comment on lines +50 to +51
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fwiw if it's NumberParseInt, the , 10 isn't required - only on the global one.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on the global one it's not required as far as I know

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's required in the conceptual sense, because parseInt will try to guess the radix if you omit it.


shard = {
__proto__: null,
index,
total,
};
Expand Down
15 changes: 9 additions & 6 deletions lib/internal/test_runner/runner.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ const {
validateBoolean,
validateFunction,
validateObject,
validateNumber,
validateInteger,
} = require('internal/validators');
const { getInspectPort, isUsingInspector, isInspectorMessage } = require('internal/util/inspector');
const { isRegExp } = require('internal/util/types');
Expand Down Expand Up @@ -424,8 +424,8 @@ function run(options) {
if (options === null || typeof options !== 'object') {
options = kEmptyObject;
}
let { testNamePatterns } = options;
const { concurrency, timeout, signal, files, inspectPort, watch, setup, shard } = options;
let { testNamePatterns, shard } = options;
const { concurrency, timeout, signal, files, inspectPort, watch, setup } = options;

if (files != null) {
validateArray(files, 'options.files');
Expand All @@ -435,8 +435,11 @@ function run(options) {
}
if (shard != null) {
validateObject(shard, 'options.shard');
validateNumber(shard.total, 'options.shard.total', 1);
validateNumber(shard.index, 'options.shard.index');
// Avoid re-evaluating the shard object in case it's a getter
shard = { index: shard.index, total: shard.total };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
shard = { index: shard.index, total: shard.total };
shard = { __proto__: null, index: shard.index, total: shard.total };

Copy link
Member Author

@rluvaton rluvaton Jul 3, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a reason there is no eslint rule for that or there is some limitation prevents from implementing it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think anybody's taken the time to make one. There will be a few places where objects shouldn't have a null prototype - mostly when they're returned to userland - but those could have override comments.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll make one

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#48646 need to answer some questions before changing all files


validateInteger(shard.total, 'options.shard.total', 1);
validateInteger(shard.index, 'options.shard.index');

if (shard.index <= 0 || shard.total < shard.index) {
throw new ERR_OUT_OF_RANGE('options.shard.index', `>= 1 && <= ${shard.total} ("options.shard.total")`, shard.index);
Expand Down Expand Up @@ -470,7 +473,7 @@ function run(options) {
let testFiles = files ?? createTestFileList();

if (shard) {
testFiles = testFiles.filter((_, index) => (index % shard.total) === (shard.index - 1));
testFiles = ArrayPrototypeFilter(testFiles, (_, index) => index % shard.total === shard.index - 1);
}

let postRun = () => root.postRun();
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-runner-cli.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ const testFixtures = fixtures.path('test-runner');

assert.strictEqual(child.status, 1);
assert.strictEqual(child.signal, null);
assert.match(child.stderr.toString(), /The argument '--test-shard' must be in the form of <index>\/<total>\. Received '0\/3'/);
assert.match(child.stderr.toString(), /The value of "options\.shard\.index" is out of range\. It must be >= 1 && <= 3 \("options\.shard\.total"\)\. Received 0/);
const stdout = child.stdout.toString();
assert.strictEqual(stdout, '');
}
Expand Down
5 changes: 3 additions & 2 deletions test/parallel/test-runner-run.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ describe('require(\'node:test\').run', { concurrency: true }, () => {
});

describe('sharding', () => {
const shardsTestsFixtures = fixtures.path('test-runner/shards');
const shardsTestsFixtures = fixtures.path('test-runner', 'shards');
const shardsTestsFiles = [
'a.cjs',
'b.cjs',
Expand Down Expand Up @@ -190,7 +190,8 @@ describe('require(\'node:test\').run', { concurrency: true }, () => {
}), {
name: 'RangeError',
code: 'ERR_OUT_OF_RANGE',
message: 'The value of "options.shard.total" is out of range. It must be >= 1. Received 0'
message:
'The value of "options.shard.total" is out of range. It must be >= 1 && <= 9007199254740991. Received 0'
});
});

Expand Down