rate_limit on Cisco, Arista, Sonic, Dell OS 10

[juztas]

Dell OS 9 and SiteRM publishes information right now for qos (#189) - Need to look on how to do this on different devices which SiteRM Supports at given moment.

• Arista
• Sonic
• Dell OS 10
• Cisco
• Juniper

[juztas]

Arista:

qos map traffic-class 1 to uc-tx-queue 1
qos map traffic-class 4 to uc-tx-queue 4
qos map traffic-class 7 to uc-tx-queue 7

class-map match-any VLAN3618
  match vlan 3618
class-map match-any VLAN3615
  match vlan 3615
class-map match-any VLAN3619
  match vlan 3619

policy-map type quality-of-service SENSE_QOS
  class VLAN3615
    #comment  Up to 240 characters, comment for this mode
    #police cir 5000000000 bc 625000 bytes
    set traffic-class 1
  class VLAN3618
    #comment  Up to 240 characters, comment for this mode
    set traffic-class 4
  class VLAN3619
    #comment  Up to 240 characters, comment for this mode
    set traffic-class 7

interface Ethernet5/1
  service-policy type qos input SENSE_QOS

DELETE:
policy-map type quality-of-service SENSE_QOS
  no class VLAN3615
no class-map VLAN3615
ADDNEW:
class-map match-any VLAN4000
  match vlan 4000
policy-map type quality-of-service SENSE_QOS
  class VLAN4000
    set traffic-class 1

[juztas]

Dell OS 10

DELL OS10 (> 10.5.3 Release)
class-map match-any VLAN3618
  # police cir 4000000 bc 200000 pir 400000
  match vlan 3618
class-map match-any VLAN3615
  match vlan 3615
class-map match-any VLAN3619
  match vlan 3619

policy-map type qos SENSE_QOS
  class VLAN3615
    set qos-group 1
  class VLAN3618
    set qos-group 4
  class VLAN3619
    set qos-group 7
 interface ethernet 1/1/21
service-policy input type qos SENSE_QOS
 interface ethernet 1/1/22
service-policy input type qos SENSE_QOS
 interface ethernet 1/1/23
service-policy input type qos SENSE_QOS
 interface ethernet 1/1/24
service-policy input type qos SENSE_QOS

[juztas]

Dell OS 9 is limited in its capabilities and mainly expects DSCP (which would require rewriting on the client side). It will not be supported moving forward.

[juztas]

1 - bestEffort;
4 - softCapped;
7 - guaranteedCapped;
Allow this to be configured per site configuration. (Is this something we should expose to SENSE?)

[juztas]

Allow control of QoS Classes via Siteconfiguration: #882

[juztas]

LA Arista changes done here:
sdn-sense/sense-aristaeos-collection@cc69c0b - mainly to send commands in batch (to speed up apply, this is a minimal improvement, but helps if the switch/router is far away)
sdn-sense/ansible-templates@02148fc - Jinja template update for QoS on Arista;

SiteRM changes (general for all devices): https://github.com/sdn-sense/siterm/compare/qoscalculateremain?expand=1
SiteRM changes include: Move traffic_classes inside qos_policy (under switch). Allow to specify max_policy_rate and burst_size.

[juztas]

Here is how QoS works on SiteRM (on Arista).
The default configuration specifies the following parameters, which can be overridden by the RM config:

"qos_policy": {
    "traffic_classes": {
        "default": 1,
        "bestEffort": 2,
        "softCapped": 4,
        "guaranteedCapped": 7
    },
    "max_policy_rate": "268000",
    "burst_size": "256"
}

Arista has certain limits. For example:

• max_policy_rate is the maximum police rate that can be set for a class.
• burst_size is the maximum burst size.

These parameters are not provided in the request. (In the future, it would be beneficial if the Orchestrator calculated the burst size based on the RTT and MTU.)

SENSE can request one of the following bandwidth types: guaranteedCapped, softCapped, or bestEffort.

When SiteRM receives a request, it calculates the bandwidth for all requests based on the following parameters:

1. Available port capacity (can be overridden by RM config if the site wants to reserve a certain percentage of bandwidth for other purposes).

2. If the request is guaranteedCapped, SiteRM will add the following configuration on the Arista device:

   policy-map type quality-of-service SENSE_QOS
      class VLAN{VLAN_ID}
         set traffic-class {TRAFFICCLASSGC}
         police rate {GC_REQUEST_BW} mbps burst-size {BURST_SIZE} mbytes
   

3. After all guaranteedCapped allocations are configured, SiteRM calculates the remaining bandwidth:

   {NewRemainingCapacity} = {AvailablePortCapacity} - {ActiveGuaranteedCapped}
   

4. For softCapped, SiteRM will configure:

      class VLAN{VLAN_ID}
         set traffic-class {TRAFFICCLASSSC}
         police rate {SC_REQUEST_BW} mbps burst-size {BURST_SIZE} mbytes \
            action set drop-precedence rate {NewRemainingCapacity} mbps burst-size {BURST_SIZE} mbytes
   

5. For bestEffort, SiteRM will configure (always allowing at least 100 Mbps, up to the maximum remaining capacity):

      class VLAN{VLAN_ID}
         set traffic-class {TRAFFICCLASSBE}
         police rate 100 mbps burst-size {BURST_SIZE} mbytes \
            action set drop-precedence rate {NewRemainingCapacity} mbps burst-size {BURST_SIZE} mbytes
   !
   

Note: SoftCapped and BestEffort are very similar in that both can reach the maximum remaining capacity of the link. However, if both compete for bandwidth simultaneously, traffic class prioritization applies (2 vs 4), and BestEffort has lower priority than SoftCapped.

[xi-yang]

Three questions:

1. Does the committed packets of softCapped have lower priority than guaranteedCapped (4 vs 7)? So there is a chance the committed packets of softCapped will be dropped when competing with the guaranteedCapped if the accounting of total bandwidth isn't done correctly.
2. The maximum commit rate is 288G per flow, correct?
3. What is the point of having 100mbps committed for bestEffort if it has the lowest priority that would get dropped anyways when competing with others?

[juztas]

1. Correct, guaranteedCapped has a higher priority than softCapped/bestEffort. SoftCapped will not be dropped, as it will not compete with guaranteedCapped in the current implementation. SiteRM knows the total available capacity (let's say 100Gbps), and if the request is 20Gbps guaranteed, the remaining BW for softQoS is 80Gbps max. Unless SiteRM fails to compute it correctly (and fails to apply the correct policy rate), then it is up to the arista to prioritize based on traffic-class (guaranteed - 7, softCapped -4, bestEffort 1). If that happens, then Guaranteed Capped could easily take it over everything. But that is where SENSE shines, that we also have traffic shaping at the host level, and it tries to be a "good citizen", not pushing too much.
2. No, on Arista EOS, max_policy_rate is not per flow; it is applied per traffic class and match criteria in the policy map (in our case, per vlan)
3. Just my thinking of implementation. Even though it is a bestEffort, it does allocate a minimum of 100 Mbps. This might change as we start some initial testing.

[xi-yang]

For 2, if our definition of a flow is per vlan, we cannot get a bandwidth reservation for like 300gbps or more?