Skip to content

Commit 0fd44ff

Browse files
done with getting-started section (other than vscode)
1 parent 6793b60 commit 0fd44ff

File tree

5 files changed

+56
-63
lines changed

5 files changed

+56
-63
lines changed

docs/getting-started/ci.mdx

Lines changed: 39 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -9,14 +9,20 @@ This guide explains how to set up TestDriver in CI/CD pipelines, using **GitHub
99

1010
## Prerequisites
1111

12-
1. **TestDriver API Key**: Obtain your API key from TestDriver and store it as a CI/CD secret (for example, `TD_API_KEY` in GitHub Actions).
12+
1. **Add API key to secrets**: Obtain your API key from TestDriver and store it as a [CI/CD secret](https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/use-secrets) as `TD_API_KEY`.
13+
Also make sure to add the `TD_WEBSITE` secret to the repository with the value of the website you are testing if its a web based application.
1314
2. **Test Files**: Ensure your test files are saved in the `testdriver/` directory of your repository.
1415

1516
## Create a CI/CD workflow
1617

1718
This example uses GitHub Actions, but the same approach can be adapted for other CI/CD platforms:
1819

1920
1. Create a new file in your repository: `.github/workflows/testdriver.yaml`.
21+
22+
<Note>
23+
If you have already [initialized the project](/getting-started/setup) with `npx testdriverai@latest init`, the workflow file will already be created.
24+
</Note>
25+
2026
2. Add the following workflow configuration:
2127

2228
### Example GitHub Actions workflow
@@ -33,64 +39,23 @@ on:
3339
- cron: '0 13 * * *' # Every day at 1 PM UTC (adjust if needed for timezone)
3440

3541
permissions:
36-
actions: read
3742
contents: read
38-
statuses: write
39-
pull-requests: write
43+
pull-requests: read
4044

4145
jobs:
42-
gather-test-files:
43-
name: Setup Test Matrix (./testdriver/*.yaml)
44-
runs-on: ubuntu-latest
45-
outputs:
46-
test_files: ${{ steps.test_list.outputs.files }}
47-
steps:
48-
- name: Check out repository
49-
uses: actions/checkout@v4
50-
with:
51-
ref: ${{ github.event.ref }}
52-
- name: Find all test files and extract filenames
53-
id: test_list
54-
run: |
55-
FILES=$(ls ./testdriver/*.yaml)
56-
FILENAMES=$(basename -a $FILES)
57-
FILES_JSON=$(echo "$FILENAMES" | jq -R -s -c 'split("\n")[:-1]')
58-
echo "files=$FILES_JSON" >> $GITHUB_OUTPUT
5946

6047
test:
61-
needs: gather-test-files
62-
strategy:
63-
matrix:
64-
test: ${{ fromJson(needs.gather-test-files.outputs.test_files) }}
65-
fail-fast: false
66-
name: ${{ matrix.test }}
48+
name: TestDriver
49+
runs-on: ubuntu-latest
6750
steps:
68-
- name: Check out repository
51+
- name: Checkout repository
6952
uses: actions/checkout@v4
70-
with:
71-
ref: ${{ github.event.ref }}
72-
fetch-depth: 0
73-
74-
- name: Set up Node.js
75-
uses: actions/setup-node@v4
76-
with:
77-
node-version: "20"
78-
cache: "npm"
79-
80-
- name: Install dependencies
81-
run: NODE_ENV=production npm ci
82-
83-
- name: Display filename being tested
84-
run: echo "Running job for file: ${{ matrix.test }}"
8553

8654
- name: Run TestDriver
87-
run: npx testdriverai@latest run testdriver/${{ matrix.test }} --headless
55+
run: npx testdriverai@latest run testdriver/testdriver.yaml --headless
8856
env:
8957
TD_API_KEY: ${{ secrets.TD_API_KEY }}
90-
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
91-
FORCE_COLOR: "3"
92-
TD_WEBSITE: ${{ vars.TD_WEBSITE || 'https://your-app.com' }}
93-
TD_THIS_FILE: ${{ matrix.test }}
58+
TD_WEBSITE: ${{ secrets.TD_WEBSITE }}
9459
```
9560
9661
## Trigger the workflow
@@ -102,7 +67,11 @@ jobs:
10267
## View results
10368

10469
1. Navigate to the **Actions** tab in your GitHub repository (or equivalent CI/CD dashboard).
105-
2. Select the workflow run to view the test results.
70+
2. Select the workflow run to view the test results. You will also find the link to the test recording at the end of the run, the url is of the format: `https://app.dashcam.io/replay/1234567890`
71+
72+
<Tip>
73+
You can also intregate our supported [reporting platform](/exporting/junit) to get more currated results.
74+
</Tip>
10675

10776
<Note>
10877
For other CI/CD platforms, adapt the workflow syntax to match your platform's
@@ -153,3 +122,24 @@ TestDriver provides a powerful debugging interface through its platform. This in
153122

154123
- Share test results with your team by generating a shareable link.
155124
- Add comments or annotations to specific steps to facilitate discussions and debugging.
125+
126+
---
127+
128+
## How it works
129+
130+
1. **Trigger**: The GitHub workflow is triggered based on the conditions defined in the `on` section.
131+
2. **Fetch the test files**: The test files are fetched from the repository based off the current branch.
132+
3. **Running the tests**: The TestDriver CLI is installed and the tests are run via the `run` command.
133+
4. **Authentication**: The `TD_API_KEY` environment variable authenticates your account.
134+
5. **Dashcam Recording**: Dashcam begins recording the test execution.
135+
6. **Test Execution**: The TestDriver CLI executes your commands on the cloud VM.
136+
8. **Test Summary**: TestDriver summarizes the test and sets the exit code based on the pass or fail state.
137+
9. **Cleanup**: The recording is uploaded to the TestDriver platform and the VM is destroyed, and all data is wiped.
138+
139+
---
140+
141+
## Additional features
142+
143+
- **Parallel testing**: You can run multiple tests in [parallel](/features/parallel-testing#setting-up-parallel-testing-with-a-matrix-strategy). The number of parallel instances depends on the plan you are on.
144+
- **Environment Variables**: Pass [custom environment variables](guide/authentication#step-2%3A-pass-credentials-to-the-workflow) to configure credentials that are required for the tests.
145+
- **Chaining Workflows**: A common workflow involves waiting for staging to deploy and testing against the staging environment by passing in the deployed url as `TD_WEBSITE` or the staging build's installation url depending on the kind of application you are testing.

docs/getting-started/editing.mdx

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,8 @@ description: "Edit previously generated tests."
55
icon: "file-pen"
66
---
77

8-
import YmlWarning from "/snippets/yml-warning.mdx";
9-
108
Test scripts are written in YAML format, making them easy to read and modify manually if you need to.
119

12-
<YmlWarning />
13-
1410
<Card title="Commands Reference" icon="link" href="/commands/assert">
1511
Explore the available commands and their parameters.
1612
</Card>
@@ -27,12 +23,12 @@ Use your preferred text editor or IDE (for example, Visual Studio Code) to open
2723

2824
Each test file consists of:
2925

30-
- **Version**: Specifies the TestDriver version.
26+
- **Version**: Specifies the TestDriver version that was used to generate the test file. This is used to lock the version of TestDriver that is used to run the test.
3127
- **Session**: A unique identifier for the test session.
3228
- **Steps**: A list of prompts and commands to execute.
3329

34-
```yaml testfile.yaml
35-
version: 4.2.18
30+
```yaml test-file.yaml
31+
version: 6.0.0
3632
session: abc1234
3733
steps:
3834
- prompt: Open Google Chrome and navigate to Airbnb
@@ -46,7 +42,8 @@ steps:
4642
- command: type
4743
text: airbnb.com
4844
- command: press-keys
49-
keys: [enter]
45+
keys:
46+
- enter
5047
```
5148
5249
<Tip>
@@ -61,7 +58,7 @@ steps:
6158
You can add, edit, or remove steps as needed. Each step contains:
6259

6360
- **Prompt**: A description of the action.
64-
- **Commands**: The specific actions to perform.
61+
- **Commands**: The specific actions to perform. It is an **optional** field, meaning, if there are no `commands` the AI considers the `prompt` and generates and executes the yaml on-the-go using the [exploratory mode](/interactive/explore)
6562

6663
#### Example edits:
6764

@@ -86,7 +83,11 @@ Ensure the YAML file is properly formatted:
8683

8784
- Use consistent indentation (2 spaces recommended).
8885
- Avoid trailing spaces or tabs.
89-
- Validate the file using a YAML linter if needed.
86+
- TestDriver will validate the yaml syntax and provide suggestions to fix any errors when you run the test.
87+
88+
<Tip>
89+
It is recommended to use our [VS Code extension](/getting-started/vscode) while you are editing the yaml. It will provide you with a rich experience with auto-completion, syntax highlighting, and validation.
90+
</Tip>
9091

9192
### Test the changes
9293

docs/getting-started/running.mdx

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,20 +17,24 @@ npx testdriverai@latest run path/to/test.yaml
1717

1818
This will execute the test script located at `path/to/test.yaml`. If the test passes, the command will exit with a status code of `0`. If it fails, the command will exit with a status code of `1`.
1919

20+
<Note>
21+
The path to the test file provided should be relative to the current working directory.
22+
</Note>
23+
2024
### Interactive mode
2125

2226
To run a test interactively:
2327

2428
1. Launch the TestDriver CLI:
2529

2630
```bash
27-
npx testdriverai@latest
31+
npx testdriverai@latest edit
2832
```
2933

3034
2. At the `>` prompt, type:
3135

3236
```bash
33-
/run path/to/test.yaml
37+
> /run path/to/test.yaml
3438
```
3539

3640
## Notes

docs/getting-started/setup.mdx

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,6 @@ icon: "square-terminal"
6767
- creating new sandbox...
6868
(this can take between 10 - 240 seconds)
6969
- connecting...
70-
7170
>
7271
```
7372
You can now start writing your prompts after the `>`

docs/snippets/yml-warning.mdx

Lines changed: 0 additions & 1 deletion
This file was deleted.

0 commit comments

Comments
 (0)