Skip to content

Commit 1b8d2eb

Browse files
bterlsonbterlson
committed
Update readme
1 parent b07680b commit 1b8d2eb

File tree

1 file changed

+11
-146
lines changed

1 file changed

+11
-146
lines changed

‎README.md‎

Lines changed: 11 additions & 146 deletions
Original file line numberDiff line numberDiff line change
@@ -2,157 +2,22 @@
22
An experimental Node-based test262 harness. Once this harness has stabilized, I plan to push to include it by default in official test262.
33

44
## Quick Start
5-
1. `npm install test262-harness`
6-
2. `./node_modules/.bin/test262-harness glob/of/test262/tests/to/run`
7-
8-
If you need the official test262 collateral:
9-
10-
`git clone https://github.com/tc39/test262.git --depth 1`
11-
12-
### Examples
13-
Run language tests in the default runner (Node):
14-
15-
`> test262-harness "../test262/test/language/**/*.js"`
16-
17-
Using -T, collateral paths are relative to the test262 root directory:
18-
19-
`> test262-harness -T ~/test262 language/**/*.js`
20-
21-
When a glob matches a directory, the default behavior will run all .js files under it recursively:
22-
23-
`> test262-harness -T ~/test262 language`
24-
25-
Run local tests in jsshell:
26-
27-
`> test262-harness -r jsshell -e jsshell/js -b ./tests`
28-
29-
Run promise tests on a promise polyfill:
30-
31-
`> test262-harness --prelude promise.js "./tests/es6/ch25/**/*.js"`
5+
1. `git clone https://github.com/tc39/test262.git --depth 1`
6+
2. `npm install -g test262-harness`
7+
3. `test262-harness test262\test\**\*.js --hostType node --hostPath <path to node.exe>`
328

9+
Run `test262-harness --help` for details on the various configuration options.
3310

3411
## Options
35-
These options may be passed on the command line or passed to useConfig in your config file (see below).
3612

3713
| Name | Action |
3814
|------------|---------------|
39-
| -r, --runner | Selects a runner to use. Currently available are `node`, `node-ip`, `jsshell`, and `console`. Config files may also pass a runner constructor whose instances implement the runner API described below.
40-
| -c, --config | Load a config.js file
41-
| -C, --compile | Save compiled tests so they can be run directly in the host without the harness. Can specify either "all" or "failures". Default is "all".
42-
| -o, --outputDir | Output directory for compiled test collateral
43-
| -e, --consoleCommand | For console runner, sets the command to invoke. Must be in PATH.
44-
| -p, --consolePrintCommand | For console runner, sets the command to write to console. Used for reporting errors to the harness.
45-
| -t, --threads | Run this many tests in parallel.
46-
| -b, --batch | How many tests to batch together. Only supported by some runners (currently just jsshell)
47-
| --testStrict | Tests both strict and non-strict mode (note: many tests need fixing for this to work)
48-
| -R, --reporter | Selects test case result format. Currently either `json`, `tap`, or `simple`. Default `simple`.
49-
| --prelude | Appends specified file to the top of each test file.
50-
| -T, --test262Dir | Root test262 directory. When used, globs are relative to the test262Dir/test.
51-
52-
53-
## Config File
54-
You can store configuration options or implement a custom runner in a config file. For example, given the following `t262.js` file:
55-
56-
```javascript
57-
var t262 = require('test262-harness');
58-
t262.useConfig({
59-
batch: 50,
60-
consoleCommand: 'js.exe',
61-
runner: 'jsshell'
62-
})
63-
```
64-
65-
the command `test262-harness -c t262.js ../test262/test/suite/**/*.js` will run all of test262 in jsshell.
66-
67-
You may also pass in an array of test globs you wish to exclude from running under 'exclude' property in config. For example, this will exclude all tests in chapter 8 from running:
68-
69-
```javascript
70-
t262.useConfig({
71-
exclude: ["../test262/test/suite/ch08/**/*.js"]
72-
})
73-
```
74-
75-
## Runners
76-
This harness is capable of running tests out of the box on a number of different hosts. Today these include Node, jsshell, and generic console hosts. You can also subclass any of these runners to provide custom behavior, for example to support transpilation tools. See the Runner API below for more details on how to do this.
77-
78-
Different runners may execute tests in different ways. The two basic methods are "normal" mode and "batch" mode. In normal mode, only one test is sent to the runner at a time, while in batch mode a configurable number is sent at once. Batch mode is signficantly more efficient in certain cases (for example, with the jsshell runner). Either mode may run tests in-process (the node-ip runner) or out-of-proc (every other runner). The following table describes the various runners and their capabilities.
79-
80-
| Runner | Name | Description |
81-
|--------|------|-------------|
82-
| out-of-proc node | `node` | Uses child\_process.fork() to run tests out-of-proc. The out-of-proc host uses vm.runInNewContext to provide isolation between tests. Default runner.
83-
| in-proc node | `node-ip` | Runs the test in the current process using vm.runInNewContext.
84-
| console | `console` | Runs tests out-of-proc in a generic console host. Works with Node, JSShell, and probably others. You will have to provide -p for normal mode runs (defaults to console.log). Can enable batch mode behavior by providing the createEnv and runBatched configuration options.
85-
| jsshell | `jsshell` | Subclass of the console runner with defaults for jsshell
86-
87-
## Debugging Test Failures
88-
Because test262-harness makes significant modifications to a test before ultimately running it, even stack traces are little help when debugging test262 failures. In these cases, the --compile flag is your friend. By passing "--compile failures -o output" to the harness, fully compiled failing tests will be written to disk under the output directory. You can run these files directly in the host you are testing.
89-
90-
## API
91-
92-
### Test262
93-
94-
#### useConfig(config)
95-
Adds the provided configuration object to the current configuration.
96-
97-
```js
98-
var t262 = require('test262-harness');
99-
t262.useConfig({batch: 50});
100-
t262.useConfig({runner: 'jsshell'}); // may be called many times
101-
```
102-
103-
#### Runner(config)
104-
The runner base constructor. Provides default implementations for much of the compilation pipeline. See below for details.
105-
106-
```js
107-
var t262 = t262.Runner
108-
```
109-
110-
#### ConsoleRunner(config)
111-
The console runner base constructor. Provides default implementations for suitable for most console hosts.
112-
113-
#### NodeRunner(config)
114-
The out-of-proc node runner constructor
115-
116-
#### JSShellRunner(config)
117-
118-
The JSShell runner constructor.
119-
120-
### Runners
121-
All runners have the same basic interface. Supporting new hosts, transpilers, or test mutations can be accomplished by writing a new runner subclass.
122-
123-
#### Runner(args)
124-
Runners are constructed by passing the current configuration object.
125-
126-
#### Runner.prototype.needsCtrlFlow
127-
Boolean flag indicating whether control-flow-related code should be injected during test compilation (see `Runner.prototype.compile`).
128-
129-
#### Runner.prototype.compile(test)
130-
Modifies the test contents to run in the target host. By default, it will append a call to $DONE if not already present, append the appropriate environment dependencies (eg. `$DONE`, `$LOG`, `$ERROR`, etc), append helpers, and add "use strict" if required.
131-
132-
#### Runner.prototype.link(test)
133-
Recursively appends helpers required in the front-matter of the test.
134-
135-
#### Runner.prototype.validateResult(test, result)
136-
Sets test.pass to true or false depending on if the result is expected.
137-
138-
Result can have the following keys:
139-
* **errorString**: Error of the form ErrorName: ErrorMessage.
140-
* **log**: An array of log strings.
141-
* **doneCalled**: boolean indicating whether $DONE was called by the test.
142-
* **errorName**: name of error thrown (if any)
143-
* **errorMessage**: message from error thrown (used for debugging purposes)
144-
* **errorStack**: stack trace of error thrown (used for debugging purposes)
145-
146-
You will not specify all of the result keys at once. If running in-proc, you will likely pass doneCalled, errorName, errorMessage, and errorStack as you will have the actual error object and you can pull these off. If running out-of-proc, you will be serializing and deserializing test results, so you may just have log strings collected from stdout and maybe an errorString parsed from stderr depending on the test is run.
147-
148-
#### Runner.prototype.run(test, done)
149-
Compiles and executes the test. Call done when the test has finished running.
150-
151-
#### Runner.prototype.runBatch(tests, batchDone)
152-
Takes an array of tests to run and compiles and executes each. Call batchDone once the entire batch is completed.
15+
| --hostType | Type of host to run tests in. See [eshost's supported hosts](https://github.com/bterlson/eshost#supported-hosts) for available options.
16+
| --hostPath | Path to the host executable.
17+
| --hostArgs | Any additional arguments to pass to the host when invoking it (eg. --harmony, --es6all, etc).
18+
| -t, --threads | Run this many tests in parallel. Note that the browser runners don't work great with t > 1.
19+
| -r, --reporter | Selects test case result format. Currently either `json` or `simple`. Default `simple`.
20+
|--test262Dir | Optional. Root test262 directory and is used to locate the includes directory.
21+
|--includesDir | Includes directory. By default inferred from test262Dir or else detected by walking upward from the first test found.
15322

154-
#### Runner.prototype.execute(test, done)
155-
Takes a fully compiled test ready to be executed in the target host, executes it, and validates the results. Call done when the test has finished executing.
15623

157-
#### Runner.prototype.executeBatch(batch, done)
158-
Takes a fully compiled batch of tests and executes them in the target host, validating each result. Call done when the entire batch has finished executing.

0 commit comments

Comments
 (0)