Skip to content

[Smartswitch] Adapt ENI Based forwarding design to FNIC #2081

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

[Smartswitch] Adapt ENI Based forwarding design to FNIC #2081

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d8c2fb4
[Smartswitch] Adapt ENI Based forwarding design to FNIC
vivekrnv Oct 3, 2025
d4881d0
Move VNI into the key of DASH_ENI_FWD table
vivekrnv Oct 6, 2025
373bcc2
Remove Tunnel VNI matching for ENI based forwarding
vivekrnv Oct 7, 2025
8cfbd23
Handle comments
vivekrnv Oct 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 18 additions & 48 deletions doc/smart-switch/high-availability/eni-based-forwarding.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
| Rev | Date | Author | Change Description |
| --- | ---- | ------ | ------------------ |
| 0.1 | 10/05/2024 | Vivek Reddy Karri | Initial version |
| 0.2 | 10/05/2025 | Vivek Reddy Karri | Floating NIC support |

## Scope ##

Expand All @@ -46,6 +47,8 @@ This document provides a high-level design for Smart Switch ENI based Packet For
| NH | Next Hop |
| HA | High Availability |
| ENI | Elastic Network Interface |
| FNIC | Floating NIC |
| PL | Private Link |

## Overview ##

Expand Down Expand Up @@ -88,16 +91,17 @@ ENI based forwarding requires the switch to understand the relationship between
* Every ENI should be a part of T1 cluster
* Each packet can be identified as belonging to that switch using VIP and VNI
* Forwarding can be to local DPU or remote DPU over L3 VxLAN
* Scale:
- [# of ENIs hosted] * 2 (inbound and outbound) * 2 (with/without Tunnel Termination) + [# of ENIs not hosted] * 2 (Inbound and outbound)
* Scale Example:
* Only support Floating NIC scenario
* Scale for FNIC + PL:
- [# of ENIs hosted] * 2 (with/without Tunnel Termination) + [# of ENIs not hosted] * 1 (without Tunnel Termination)
* Scale Example for FNIC + PL:
- T1 per cluster: 8
- DPU per T1: 4
- ENI per DPU: 64
- HA Scaling factor: 2
- Total ENI's in this Cluster: (8 * 4 * 64) / 2 = 1024
- ENI's hosted on a T1: 256
- Number of ACL Rules: 256 * (2 * 2) + (1024 - 256) * 2 = 2560
- Number of ACL Rules: 256 * 2 + (1024 - 256) = 1280

### Phase 1 ###

Expand All @@ -106,7 +110,7 @@ ENI based forwarding requires the switch to understand the relationship between
- Orchagent should also program ACL Rules with Tunnel termination entries
- No BFD sessions are created to local DPU or the remote DPU.

DASH_ENI_FORWARD_TABLE schema is available here https://github.com/r12f/SONiC/blob/user/r12f/ha2/doc/smart-switch/high-availability/smart-switch-ha-detailed-design.md#2321-dash_eni_forward_table
DASH_ENI_FORWARD_TABLE schema is available here https://github.com/sonic-net/SONiC/blob/master/doc/smart-switch/high-availability/smart-switch-ha-detailed-design.md#2321-dash_eni_forward_table

### Phase 2 ###

Expand All @@ -122,10 +126,8 @@ DASH_ENI_FORWARD_TABLE schema is available here https://github.com/r12f/SONiC/bl
"ACL_TABLE_TYPE": {
"ENI_REDIRECT": {
"MATCHES": [
"TUNNEL_VNI",
"DST_IP",
"DST_IPV6",
"INNER_SRC_MAC",
"INNER_DST_MAC",
"TUNNEL_TERM"
],
Expand Down Expand Up @@ -158,32 +160,17 @@ VIP: 1.1.1.1/32
VNET: Vnet1000
```

**ACL Rule for outbound traffic**
**ACL Rule**

MacDirection for outbound rules depends on the outbound_eni_mac_lookup field in the DASH_ENI_FORWARD_TABLE
Note:
- outbound_eni_mac_lookup is not relevant for FNIC cases since we always match on INNER_DST_MAC
- outbound_vni is not relevant for FNIC cases since we don't need to match on Tunnel VNI
- VIP is common across the T1 cluster. It is not clear on where to consume it. It is currently read from a temporary config DB table "VIP_TABLE".

```
{
"ACL_RULE": {
"ENI:Vnet1000_AABBCCDDEEFF_OUT": {
"PRIORITY": "9997",
"TUNNEL_VNI": "4000",
"DST_IP": "1.1.1.1/32",
"INNER_SRC_MAC/INNER_DST_MAC": "aa:bb:cc:dd:ee:ff"
"REDIRECT": "<local/tunnel nexthop>"
}
}
}
```

**ACL Rule for inbound traffic**

Inbound Traffic can have any ENI expect the outbound VNI, no need to match on TUNNEL_VNI

```
{
"ACL_RULE": {
"ENI:Vnet1000_AABBCCDDEEFF_IN": {
"ENI:Vnet1000_AABBCCDDEEFF": {
"PRIORITY": "9996",
"DST_IP": "1.1.1.1/32",
"INNER_DST_MAC": "aa:bb:cc:dd:ee:ff"
Expand All @@ -207,30 +194,13 @@ To solve this, ACL rules with high priority are added and the redirect should al

![Tunnel Termination Solution](./images/tunn_term_solution.png)

**ACL Rule for outbound traffic with Tunnel Termination**
**ACL Rule with Tunnel Termination**

```
{
"ACL_RULE": {
"ENI:Vnet1000_AABBCCDDEEFF_OUT_TERM": {
"PRIORITY": "9999",
"TUNNEL_VNI": "4000",
"DST_IP": "1.1.1.1/32",
"INNER_SRC_MAC/INNER_DST_MAC": "aa:bb:cc:dd:ee:ff",
"TUNN_TERM": "true",
"REDIRECT": "<local nexthop oid>"
}
}
}
```

**ACL Rule for inbound traffic with Tunnel Termination**

```
{
"ACL_RULE": {
"ENI:Vnet1000_AABBCCDDEEFF_IN_TERM": {
"PRIORITY": "9998",
"ENI:Vnet1000_AABBCCDDEEFF_TERM": {
"PRIORITY": "9997",
"DST_IP": "1.1.1.1/32",
"INNER_DST_MAC": "aa:bb:cc:dd:ee:ff",
"TUNN_TERM": "true",
Expand Down