You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/docs/cli/creating-test-outputs.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,10 +1,10 @@
1
1
# Defining Test Outputs in Text Files
2
2
3
-
Outputs are really useful when running [Transactions](../concepts/transactions). They allow for exporting values from a test so they become available in the [Environment Variables](../concepts/variable-sets.md) of the current transaction.
3
+
Outputs are really useful when running [Transactions](../concepts/transactions). They allow for exporting values from a test so they become available in the [Variable Sets](../concepts/variable-sets.md) of the current transaction.
4
4
5
5
## Outputs are Expression Results
6
6
7
-
An output exports the result of an [Expression](../concepts/expressions) and assigns it to a name, so it can be injected into the environment variables of a running transaction.
7
+
An output exports the result of an [Expression](../concepts/expressions) and assigns it to a name, so it can be injected into the variable set of a running transaction.
8
8
A `selector` is needed only if the provided expression refers to a/some span/s attribute or meta attributes.
9
9
10
10
It can be defined using the following YAML definition:
This page showcases how to create and edit variable sets with the CLI.
4
+
5
+
:::tip
6
+
[To read more about variable sets check out variable sets concepts.](../concepts/variable-sets.md)
7
+
:::
8
+
9
+
Just like Data Stores, you can also manage your variable sets using the CLI and definition files.
10
+
11
+
A definition file looks like the following:
12
+
13
+
```yaml
14
+
type: VariableSet
15
+
spec:
16
+
name: Production
17
+
description: Production env variables
18
+
values:
19
+
- key: URL
20
+
value: https://app-production.company.com
21
+
- key: API_KEY
22
+
value: mysecret
23
+
```
24
+
25
+
In order to apply this configuration to your Tracetest instance, make sure to have your [CLI configured](./configuring-your-cli.md) and run:
26
+
27
+
```sh
28
+
tracetest apply variableset -f <variableset.yaml>
29
+
```
30
+
31
+
> If the file contains the property `spec.id`, the operation will be considered a variable set update. If you try to apply a variable set and you get the error: `could not apply variableset: 404 Not Found`, it means the provided ID doesn't exist. Either update the ID to reference an existing variable set or remove the property from the file. Tracetest will create a new variable set and a new ID.
We can run a test and specify that environment with this command:
197
+
We can run a test and specify that variable set with this command:
198
198
199
199
```sh
200
200
tracetest run test -f path/to/test.yaml -e testenv
201
201
```
202
202
203
-
You can also reference an environment resource file which will be used to create a new environment or update an existing one. For example, if you have a file named local.env with this content:
203
+
You can also reference a variable set resource file which will be used to create a new variable set or update an existing one. For example, if you have a file named local.env with this content:
204
204
205
205
```yaml
206
-
type: Environment
206
+
type: VariableSet
207
207
spec:
208
208
id: local.env
209
209
name: local.env
@@ -215,9 +215,9 @@ spec:
215
215
```
216
216
217
217
```sh
218
-
tracetest run test -f path/to/test.yaml -e path/to/local.env
218
+
tracetest run test -f path/to/test.yaml --vars path/to/local.env
219
219
```
220
220
221
-
If you use the environment resource approach, a new environment will be created in Tracetest.
221
+
If you use the variable set resource approach, a new variable set will be created in Tracetest.
222
222
223
223
The second approach is very useful if you are running tests from a CI pipeline.
We can run a transaction and specify that environment with this command:
32
+
We can run a transaction and specify that variable set with this command:
33
33
34
34
```sh
35
-
tracetest run transaction -f path/to/transaction.yaml -e testenv
35
+
tracetest run transaction -f path/to/transaction.yaml --vars testenv
36
36
```
37
37
38
-
You can also reference an environment resource file which will be used to create a new environment or update an existing one. For example, if you have a file named `local.env` with this content:
38
+
You can also reference a variable set resource file which will be used to create a new variable set or update an existing one. For example, if you have a file named `local.env` with this content:
39
39
40
40
```yaml
41
-
type: Environment
41
+
type: VariableSet
42
42
spec:
43
43
id: local.env
44
44
name: local.env
@@ -50,9 +50,9 @@ spec:
50
50
```
51
51
52
52
```sh
53
-
tracetest run test -f path/to/transaction.yaml -e path/to/local.env
53
+
tracetest run test -f path/to/transaction.yaml --vars path/to/local.env
54
54
```
55
55
56
-
If you use the environment resource approach, a new environment will be created in Tracetest.
56
+
If you use the variable set resource approach, a new variable set will be created in Tracetest.
57
57
58
58
The second approach is very useful if you are running tests from a CI pipeline.
Copy file name to clipboardExpand all lines: docs/docs/cli/undefined-variables.md
+9-9Lines changed: 9 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,29 +13,29 @@ POKEID:
13
13
POKENAME:
14
14
```
15
15
16
-
Undefined variables are dependent on the environment selected and whether or not the variable is defined in the current environment. Select the environment to run the test or transaction by passing it into the test run command.
16
+
Undefined variables are dependent on the variable set selected and whether or not the variable is defined in the current variable set. Select the variable set to run the test or transaction by passing it into the test run command.
17
17
18
18
```sh
19
-
tracetest list environment
19
+
tracetest list variableset
20
20
```
21
21
22
22
```text title="Output:"
23
23
ID NAME DESCRIPTION
24
24
--------- --------- -------------
25
-
testenvtestenvtestenv
25
+
testvarstestvarstestvars
26
26
```
27
27
28
28
```sh
29
-
tracetest get environment --id testenv
29
+
tracetest get variableset --id testvars
30
30
```
31
31
32
32
```text title="Output:"
33
33
---
34
-
type: Environment
34
+
type: VariableSet
35
35
spec:
36
-
id: testenv
37
-
name: testenv
38
-
description: testenv
36
+
id: testvars
37
+
name: testvars
38
+
description: testvars
39
39
values:
40
40
- key: POKEID
41
41
value: "42"
@@ -44,7 +44,7 @@ spec:
44
44
```
45
45
46
46
```sh
47
-
tracetest run test -f path/to/test.yaml -e testenv
47
+
tracetest run test -f path/to/test.yaml --vars testvars
Copy file name to clipboardExpand all lines: docs/docs/concepts/ad-hoc-testing.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
# Ad-hoc Testing
2
2
3
-
This page showcases use-cases for undefined variables and how to enable ad-hoc testing by utilizing environments and undefined variabled.
3
+
This page showcases use-cases for undefined variables and how to enable ad-hoc testing by utilizing variable sets and undefined variables.
4
4
5
5
:::tip
6
6
[Check out how to configure ad-hoc testing with undefined variables with the **Web UI** here.](../web-ui/undefined-variables.md)
@@ -24,15 +24,15 @@ In Tracetest, undefined variables can be used in both the UI and CLI.
24
24
25
25
## **Undefined Variables Transaction with Multiple Tests Example**
26
26
27
-
1. Create an HTTP Pokemon list test that uses environment variables for hostname and the SKIP query parameter:
27
+
1. Create an HTTP Pokemon list test that uses variables for hostname and the SKIP query parameter:
28
28
29
29

30
30
31
-
2. Within the test, create test spec assertions that use environment variables for comparators, something like: `http.status_code = "${env:STATUS_CODE}"`:
31
+
1. Within the test, create test spec assertions that use variables for comparators, something like: `http.status_code = "${env:STATUS_CODE}"`:
32
32
33
33

34
34
35
-
3. Create a GRPC Pokemon add test that uses environment variables for hostname and Pokemon name:
35
+
1. Create a GRPC Pokemon add test that uses variables for hostname and Pokemon name:
Some tests might require strings to be compared, but maybe you need to generate a dynamic string that relies on a dynamic value. This might be used in an assertion or even in the request body referencing an environment variable.
35
+
Some tests might require strings to be compared, but maybe you need to generate a dynamic string that relies on a dynamic value. This might be used in an assertion or even in the request body referencing a variable.
36
36
37
37
```css
38
38
attr:error.message = "Could not withdraw ${attr:withdraw.amount}, your balance is insufficient."
Copy file name to clipboardExpand all lines: docs/docs/concepts/transactions.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,13 +9,13 @@ A transaction is defined as a group of steps that are executed in the defined or
9
9
The main benefit of using transactions is to chain tests together and use values obtained from a test in a subsequent test.
10
10
11
11
### How Values are Shared by Tests
12
-
When a transaction is run, a context object is created with information about that specific run. One of those pieces of information is an `environment variables` object, which is empty by default. If the transaction is run when referencing an [environment](./environments), all values from the selected environments will be copied to the `environment variables` object.
12
+
When a transaction is run, a context object is created with information about that specific run. One of those pieces of information is a `variable set` object, which is empty by default. If the transaction is run when referencing an [variable set](./variable-sets.md), all values from the selected variable sets will be copied to the `variable set` object.
13
13
14
-
When a test is executed within a transaction, if it generates any outputs, its outputs will be injected into the transaction context environment variables object. After the outputs are injected, all subsequent tests to be run within the transaction will be able to reference those values.
14
+
When a test is executed within a transaction, if it generates any outputs, its outputs will be injected into the transaction context variable set object. After the outputs are injected, all subsequent tests to be run within the transaction will be able to reference those values.
15
15
16
-
> :information_source: Outputs generated by steps don't modify the selected [environment](./environments). It only modifies the transaction run context object.
16
+
> :information_source: Outputs generated by steps don't modify the selected [variable set](./variable-sets.md). It only modifies the transaction run context object.
17
17
18
-
Consider you have 3 tests within a transaction: A, B, and C. Tests A and B generate outputs called A_OUTPUT and B_OUTPUT, respectively. When running the transaction, we provide an environment which contains a `HOST`environment variable. The execution of test A would only be able to reference `env:HOST`. B would be able to reference `env:HOST`, and `env:A_OUTPUT`. While C would be able to reference all three environment variables: `env:HOST`, `env:A_OUTPUT`, `env:B_OUTPUT`.
18
+
Consider you have 3 tests within a transaction: A, B, and C. Tests A and B generate outputs called A_OUTPUT and B_OUTPUT, respectively. When running the transaction, we provide a variable set which contains a `HOST` variable. The execution of test A would only be able to reference `env:HOST`. B would be able to reference `env:HOST`, and `env:A_OUTPUT`. While C would be able to reference all three variables: `env:HOST`, `env:A_OUTPUT`, `env:B_OUTPUT`.
19
19
20
20
> :information_source: A single test can contain as many outputs as you like.
21
21
@@ -29,7 +29,7 @@ outputs:
29
29
expression: attr:tracetest.time.start
30
30
```
31
31
32
-
This would create an output called `TIME_CANCEL_SUBSCRIPTION_MESSAGE_OBTAINED` that is obtained by reading the attribute `tracetest.time.start` from the span with `name` equal to `Process request from cancel subscription queue`. This value would would then be injected into the environment variables of that transaction to be accessed by other tests within the same transaction run.
32
+
This would create an output called `TIME_CANCEL_SUBSCRIPTION_MESSAGE_OBTAINED` that is obtained by reading the attribute `tracetest.time.start` from the span with `name` equal to `Process request from cancel subscription queue`. This value would then be injected into the variables of that transaction to be accessed by other tests within the same transaction run.
0 commit comments