Skip to content

[Docs] Add telemetry section to chaos strategies documentation pages #2071

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Apr 27, 2024
Merged

[Docs] Add telemetry section to chaos strategies documentation pages #2071

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
445c51d
Use relative links instead of absolute
peter-csala Apr 23, 2024
fa3070c
Add telemetry section to Behavior Chaos
peter-csala Apr 23, 2024
c4d520e
Add telemetry section to Fault Chaos
peter-csala Apr 23, 2024
48f421c
Add telemetry to Latency Chaos
peter-csala Apr 24, 2024
0f9e52f
Revise defaults descriptions for chaos strategies
peter-csala Apr 24, 2024
c5db33f
Add telemetry section to Outcome
peter-csala Apr 24, 2024
b64f75c
Replace mandatory to required
peter-csala Apr 24, 2024
800be64
Fix linting issue
peter-csala Apr 24, 2024
0038671
Fix linting issue
peter-csala Apr 24, 2024
4d1e9e5
Update chaos index page
peter-csala Apr 24, 2024
b8bbfdf
Update telemetry page
peter-csala Apr 24, 2024
3e062eb
Rename static to fixed
peter-csala Apr 24, 2024
616fdc7
Replace table to bullet list
peter-csala Apr 24, 2024
975a8f3
Replace table to bullet list
peter-csala Apr 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 15 additions & 9 deletions docs/advanced/telemetry.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Telemetry

Starting with version 8, Polly provides telemetry for all built-in resilience strategies.
Starting with version 8, Polly provides telemetry for all built-in standard and chaos resilience strategies.

## Usage

Expand Down Expand Up @@ -140,14 +140,20 @@ Tags:

The `event.name` tag is reported by individual resilience strategies. The built-in strategies report the following events:

- `OnRetry`
- `OnFallback`
- `OnHedging`
- `OnTimeout`
- `OnCircuitClosed`
- `OnCircuitOpened`
- `OnCircuitHalfOpened`
- `OnRateLimiterRejected`
| Event | Details |
|-------------------------|----------------------------------------------------|
| `OnRetry` | [Link](../strategies/retry.md#telemetry) |
| `OnFallback` | [Link](../strategies/fallback.md#telemetry) |
| `OnHedging` | [Link](../strategies/hedging.md#telemetry) |
| `OnTimeout` | [Link](../strategies/timeout.md#telemetry) |
| `OnCircuitClosed` | [Link](../strategies/circuit-breaker.md#telemetry) |
| `OnCircuitOpened` | [Link](../strategies/circuit-breaker.md#telemetry) |
| `OnCircuitHalfOpened` | [Link](../strategies/circuit-breaker.md#telemetry) |
| `OnRateLimiterRejected` | [Link](../strategies/rate-limiter.md#telemetry) |
| `Chaos.OnFault` | [Link](../chaos/fault.md#telemetry) |
| `Chaos.OnOutcome` | [Link](../chaos/outcome.md#telemetry) |
| `Chaos.OnLatency` | [Link](../chaos/latency.md#telemetry) |
| `Chaos.OnBehavior` | [Link](../chaos/behavior.md#telemetry) |

### Instrument: `resilience.polly.strategy.attempt.duration`

Expand Down
43 changes: 35 additions & 8 deletions docs/chaos/behavior.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@

## About

- **Options**: [`ChaosBehaviorStrategyOptions`](xref:Polly.Simmy.Behavior.ChaosBehaviorStrategyOptions)
- **Extensions**: `AddChaosBehavior`
- **Strategy Type**: Proactive
- **Option(s)**:
- [`ChaosBehaviorStrategyOptions`](xref:Polly.Simmy.Behavior.ChaosBehaviorStrategyOptions)
- **Extension(s)**:
- `AddChaosBehavior`
- **Exception(s)**: -

---

The behavior chaos strategy is designed to inject custom behaviors into system operations right before such an operation is invoked. This strategy is flexible, allowing users to define specific behaviors such as altering the input, simulating resource exhaustion, putting the system in a given state before the actual operation is called, or other operational variations to simulate real-world scenarios.
The behavior **proactive** chaos strategy is designed to inject custom behaviors into system operations right before such an operation is invoked. This strategy is flexible, allowing users to define specific behaviors such as altering the input, simulating resource exhaustion, putting the system in a given state before the actual operation is called, or other operational variations to simulate real-world scenarios.

## Usage

Expand Down Expand Up @@ -66,10 +68,35 @@ var pipeline = new ResiliencePipelineBuilder()

## Defaults

| Property | Default Value | Description |
|----------------------|---------------|------------------------------------------------|
| `OnBehaviorInjected` | `null` | Action executed when the behavior is injected. |
| `BehaviorGenerator` | `null` | Custom behavior to be injected. |
| Property | Default Value | Description |
|----------------------|---------------|------------------------------------------------------------------------------------------------------------------------------|
| `BehaviorGenerator` | `null` | This required delegate allows you to inject custom behavior by utilizing information that is only available at runtime. |
| `OnBehaviorInjected` | `null` | If provided then it will be invoked after the behavior injection occurred. |

## Telemetry

The behavior chaos strategy reports the following telemetry events:

| Event Name | Event Severity | When? |
|--------------------|----------------|------------------------------------------------------------------|
| `Chaos.OnBehavior` | `Information` | Just before the strategy calls the `OnBehaviorInjected` delegate |

Here are some sample events:

```none
Resilience event occurred. EventName: 'Chaos.OnBehavior', Source: '(null)/(null)/Chaos.Behavior', Operation Key: '', Result: ''

Resilience event occurred. EventName: 'Chaos.OnBehavior', Source: 'MyPipeline/MyPipelineInstance/MyChaosBehaviorStrategy', Operation Key: 'MyBehaviorInjectedOperation', Result: ''
```

> [!NOTE]
> Please note that the `Chaos.OnBehavior` telemetry event will be reported **only if** the behavior chaos strategy injects a custom behavior which does not throw exception.
>
> So, if the behavior is either not injected or injected and throws an exception then there will be no telemetry emitted.
>
> Also remember that the `Result` will be **always empty** for the `Chaos.OnBehavior` telemetry event.

For further information please check out the [telemetry page](../advanced/telemetry.md).

## Diagrams

Expand Down
43 changes: 35 additions & 8 deletions docs/chaos/fault.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@

## About

- **Options**: [`ChaosFaultStrategyOptions`](xref:Polly.Simmy.Fault.ChaosFaultStrategyOptions)
- **Extensions**: `AddChaosFault`
- **Strategy Type**: Proactive
- **Option(s)**:
- [`ChaosFaultStrategyOptions`](xref:Polly.Simmy.Fault.ChaosFaultStrategyOptions)
- **Extensions**:
- `AddChaosFault`
- **Exception(s)**: -

---

The fault chaos strategy is designed to introduce faults (exceptions) into the system, simulating real-world scenarios where operations might fail unexpectedly. It is configurable to inject specific types of exceptions or use custom logic to generate faults dynamically.
The fault **proactive** chaos strategy is designed to introduce faults (exceptions) into the system, simulating real-world scenarios where operations might fail unexpectedly. It is configurable to inject specific types of exceptions or use custom logic to generate faults dynamically.

## Usage

Expand Down Expand Up @@ -87,10 +89,35 @@ var pipeline = new ResiliencePipelineBuilder()

## Defaults

| Property | Default Value | Description |
|-------------------|---------------|------------------------------------------------------|
| `OnFaultInjected` | `null` | Action executed when the fault is injected. |
| `FaultGenerator` | `null` | Generates the fault to inject for a given execution. |
| Property | Default Value | Description |
|-------------------|---------------|------------------------------------------------------------------------------------------------------------------------|
| `FaultGenerator` | `null` | This required delegate allows you to inject exception by utilizing information that is only available at runtime. |
| `OnFaultInjected` | `null` | If provided then it will be invoked after the fault injection occurred. |

## Telemetry

The fault chaos strategy reports the following telemetry events:

| Event Name | Event Severity | When? |
|-----------------|----------------|---------------------------------------------------------------|
| `Chaos.OnFault` | `Information` | Just before the strategy calls the `OnFaultInjected` delegate |

Here are some sample events:

```none
Resilience event occurred. EventName: 'Chaos.OnFault', Source: '(null)/(null)/Chaos.Fault', Operation Key: '', Result: ''

Resilience event occurred. EventName: 'Chaos.OnFault', Source: 'MyPipeline/MyPipelineInstance/MyChaosFaultStrategy', Operation Key: 'MyFaultInjectedOperation', Result: ''
```

> [!NOTE]
> Please note that the `Chaos.OnFault` telemetry event will be reported **only if** the fault chaos strategy injects an exception which is wrapped into a `ValueTask`.
>
> So, if the fault is either not injected or injected and throws an exception then there will be no telemetry emitted.
>
> Also remember that the `Result` will be **always empty** for the `Chaos.OnFault` telemetry event.

For further information please check out the [telemetry page](../advanced/telemetry.md).

## Diagrams

Expand Down
36 changes: 21 additions & 15 deletions docs/chaos/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,14 @@ This section highlights the major differences compared to the [`Polly.Contrib.Si
- **Unified configuration options**: The `InjectOptionsBase` and `InjectOptionsAsyncBase` are now consolidated into `ChaosStrategyOptions`. This change brings Simmy in line with the Polly v8 API, offering built-in support for options-based configuration and seamless integration of synchronous and asynchronous executions.
- **Chaos strategies enabled by default**: Adding a chaos strategy (previously known as monkey policy) now means it's active right away. This is a departure from earlier versions, where the monkey policy had to be explicitly enabled.
- **API changes**: The new version of Simmy introduces several API updates. While this list isn't complete, it includes key changes like renaming `Inject` to `AddChaos` and switching from `Result` to `Outcome`. Here are some specific renames:
- `InjectException` is now `AddChaosFault`
- `InjectResult` is now `AddChaosOutcome`
- `InjectBehavior` is now `AddChaosBehavior`
- `InjectLatency` is now `AddChaosLatency`

| From | To |
|-------------------|--------------------|
| `InjectException` | `AddChaosFault` |
| `InjectResult` | `AddChaosOutcome` |
| `InjectBehavior` | `AddChaosBehavior` |
| `InjectLatency` | `AddChaosLatency` |

- **Sync and async unification**: Before, Simmy had various methods to set policies like `InjectLatency`, `InjectLatencyAsync`, `InjectLatency<T>`, and `InjectLatencyAsync<T>`. With the new version based on Polly v8, these methods have been combined into a single `AddChaosLatency` extension that works for both `ResiliencePipelineBuilder` and `ResiliencePipelineBuilder<T>`. These rules are covering all types of chaos strategies (Outcome, Fault, Latency, and Behavior).

## Motivation
Expand All @@ -82,12 +86,12 @@ Chaos strategies (formerly known as Monkey strategies) are in essence a [Resilie

### Built-in strategies

| Strategy | Reactive | What does the strategy do? |
|-------------------------|----------|----------------------------------------------------------------------|
| [Fault](fault.md) | No | Injects exceptions in your system. |
| [Outcome](outcome.md) | Yes | Injects fake outcomes (results or exceptions) in your system. |
| [Latency](latency.md) | No | Injects latency into executions before the calls are made. |
| [Behavior](behavior.md) | No | Allows you to inject *any* extra behavior, before a call is placed. |
| Strategy | Type | What does the strategy do? |
|-------------------------|-----------|---------------------------------------------------------------------|
| [Fault](fault.md) | Proactive | Injects exceptions in your system. |
| [Outcome](outcome.md) | Reactive | Injects fake outcomes (results or exceptions) in your system. |
| [Latency](latency.md) | Proactive | Injects latency into executions before the calls are made. |
| [Behavior](behavior.md) | Proactive | Allows you to inject *any* extra behavior, before a call is placed. |

## Common options across strategies

Expand All @@ -114,12 +118,14 @@ All the strategies' options implement the [`ChaosStrategyOptions`](xref:Polly.Si

## Telemetry

The telemetry of chaos strategies is seamlessly integrated with Polly [telemetry infrastructure](../advanced/telemetry.md). The chaos strategies produce the following events:
The telemetry of chaos strategies is seamlessly integrated with Polly [telemetry infrastructure](../advanced/telemetry.md). The chaos strategies produce the following information events:

- `Chaos.OnFault`
- `Chaos.OnOutcome`
- `Chaos.OnLatency`
- `Chaos.OnBehavior`
| Event | Details |
|--------------------|-------------------------------|
| `Chaos.OnFault` | [Link](fault.md#telemetry) |
| `Chaos.OnOutcome` | [Link](outcome.md#telemetry) |
| `Chaos.OnLatency` | [Link](latency.md#telemetry) |
| `Chaos.OnBehavior` | [Link](behavior.md#telemetry) |

## Patterns

Expand Down
50 changes: 41 additions & 9 deletions docs/chaos/latency.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@

## About

- **Options**: [`ChaosLatencyStrategyOptions`](xref:Polly.Simmy.Latency.ChaosLatencyStrategyOptions)
- **Extensions**: `AddChaosLatency`
- **Strategy Type**: Proactive
- **Option(s)**:
- [`ChaosLatencyStrategyOptions`](xref:Polly.Simmy.Latency.ChaosLatencyStrategyOptions)
- **Extension(s)**:
- `AddChaosLatency`
- **Exception(s)**: -

---

The latency chaos strategy is designed to introduce controlled delays into system operations, simulating network latency or slow processing times. This strategy helps in assessing and improving the resilience of applications against increased response times.
The latency **proactive** chaos strategy is designed to introduce controlled delays into system operations, simulating network latency or slow processing times. This strategy helps in assessing and improving the resilience of applications against increased response times.

## Usage

Expand Down Expand Up @@ -90,15 +92,45 @@ var pipeline = new ResiliencePipelineBuilder()

## Defaults

| Property | Default Value | Description |
|---------------------|---------------|--------------------------------------------------------|
| `Latency` | `30 seconds` | A `TimeSpan` indicating the delay to be injected. |
| `LatencyGenerator` | `null` | Generates the latency to inject for a given execution. |
| `OnLatencyInjected` | `null` | Action executed when latency is injected. |
| Property | Default Value | Description |
|---------------------|---------------|----------------------------------------------------------------------------------------------------------------------|
| `Latency` | 30 seconds | Defines a **fixed** delay to be injected. |
| `LatencyGenerator` | `null` | This delegate allows you to **dynamically** inject delay by utilizing information that is only available at runtime. |
| `OnLatencyInjected` | `null` | If provided then it will be invoked after the latency injection occurred. |

> [!NOTE]
> If both `Latency` and `LatencyGenerator` are specified then `Latency` will be ignored.

---

> [!IMPORTANT]
> Please note that if the calculated latency is negative (regardless if it's fixed or dynamic) then it will not be injected.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍


## Telemetry

The latency chaos strategy reports the following telemetry events:

| Event Name | Event Severity | When? |
|-------------------|----------------|-----------------------------------------------------------------|
| `Chaos.OnLatency` | `Information` | Just before the strategy calls the `OnLatencyInjected` delegate |

Here are some sample events:

```none
Resilience event occurred. EventName: 'Chaos.OnLatency', Source: '(null)/(null)/Chaos.Latency', Operation Key: '', Result: ''

Resilience event occurred. EventName: 'Chaos.OnLatency', Source: 'MyPipeline/MyPipelineInstance/MyLatencyStrategy', Operation Key: 'MyLatencyInjectedOperation', Result: ''
```

> [!NOTE]
> Please note that the `Chaos.OnLatency` telemetry event will be reported **only if** the latency chaos strategy injects a positive delay.
>
> So, if the latency is not injected then there will be no telemetry emitted. Also if injected but the latency is negative or the `LatencyGenerator` throws an exception then there will be no telemetry emitted.
>
> Also remember that the `Result` will be **always empty** for the `Chaos.OnLatency` telemetry event.

For further information please check out the [telemetry page](../advanced/telemetry.md).

## Diagrams

### Normal 🐵 sequence diagram
Expand Down
45 changes: 37 additions & 8 deletions docs/chaos/outcome.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,15 @@

## About

- **Options**:
- **Option(s)**:
- [`ChaosOutcomeStrategyOptions<T>`](xref:Polly.Simmy.Outcomes.ChaosOutcomeStrategyOptions`1)
- **Extensions**: `AddChaosOutcome`
- **Strategy Type**: Reactive
- **Extension(s)**:
- `AddChaosOutcome`
- **Exception(s)**: -

---

The outcome chaos strategy is designed to inject or substitute fake results into system operations. This allows testing how an application behaves when it receives different types of responses, like successful results, errors, or exceptions.
The outcome **reactive** chaos strategy is designed to inject or substitute fake results into system operations. This allows testing how an application behaves when it receives different types of responses, like successful results, errors, or exceptions.

## Usage

Expand Down Expand Up @@ -79,10 +80,38 @@ var pipeline = new ResiliencePipelineBuilder<HttpResponseMessage>()

## Defaults

| Property | Default Value | Description |
|---------------------|---------------|---------------------------------------------------------|
| `OutcomeGenerator` | `null` | Function to generate the outcome for a given execution. |
| `OnOutcomeInjected` | `null` | Action executed when the outcome is injected. |
| Property | Default Value | Description |
|---------------------|---------------|-----------------------------------------------------------------------------------------------------------------------------|
| `OutcomeGenerator` | `null` | This required delegate allows you to inject custom outcome by utilizing information that is only available at runtime. |
| `OnOutcomeInjected` | `null` | If provided then it will be invoked after the outcome injection occurred. |

> [!NOTE]
> Please note this strategy is a reactive chaos strategy, but it does not have a `ShouldHandle` delegate.

## Telemetry

The outcome chaos strategy reports the following telemetry events:

| Event Name | Event Severity | When? |
|-------------------|----------------|-----------------------------------------------------------------|
| `Chaos.OnOutcome` | `Information` | Just before the strategy calls the `OnOutcomeInjected` delegate |

Here are some sample events:

```none
Resilience event occurred. EventName: 'Chaos.OnOutcome', Source: '(null)/(null)/Chaos.Outcome', Operation Key: '', Result: ''

Resilience event occurred. EventName: 'Chaos.OnOutcome', Source: 'MyPipeline/MyPipelineInstance/MyOutcomeStrategy', Operation Key: 'MyOutcomeInjectedOperation', Result: ''
```

> [!NOTE]
> Please note that the `Chaos.OnOutcome` telemetry event will be reported **only if** the outcome chaos strategy injects an outcome object.
>
> So, if the outcome is not injected or injected but the generator delegate throws an exception then there will be no telemetry emitted.
>
> Also remember that the `Result` will be **always empty** for the `Chaos.OnOutcome` telemetry event.

For further information please check out the [telemetry page](../advanced/telemetry.md).

## Diagrams

Expand Down
Loading