Skip to content

Add Thresholds support #746

@AntyaDev

Description

@AntyaDev

Discussion reference: #707
Documentation link
AC:

  • Thresholds can be defined via code
  • Thresholds can be defined via JsonConfig
  • Thresholds should be displayed in the HTML report
  • Thresholds should support "StartCheck from specific time". StartCheckFrom: TimeSpan
  • Thresholds should support AbortScenario after racing specified error count. AbortWhenErrorCount: int

Code example:

Scenario
    .Create("scn", ...)
    .WithThresholds(
        // threshold for Scenario
        Threshold.Create(scnStats => scnStats.Ok.Request.RPS < 100),

        // threshold for Step
        Threshold.Create("step_name", stepStats => stepStats.Ok.Request.RPS < 100),

        // will abort scenario when ErrorCount = 2
        Threshold.Create(scnStats => scnStats.Ok.Request.RPS < 100, abortWhenErrorCount: 2),
    
        // will start check threshold after 1 minute of Scenario execution
        Threshold.Create(scnStats => scnStats.Ok.Request.RPS < 100, startCheckFrom: TimeSpan.FromMinutes(1))
    )

JsonConfig

"GlobalSettings": {

        "ScenariosSettings": [
            {
                "ScenarioName": "scenario",
                
                "ThresholdSettings": [

                    { "OkRequest": "RPS < 100" },

                    { "FailRequest": "Percent < 20" },
                    
                    { "StepName":  "logout", "OkLatency": "p50 <= 30" },

                    { "FailRequest": "Percent < 20", "AbortWhenErrorCount": 10 },

                    { "FailRequest": "Percent < 20", "StartCheckFrom": "00:00:20" },
                   
                ]
            }
        ]
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions