arp traffic through bridge

[msyretis]

Hello,

I am trying to migrate a few Nokia vSRs from EVE-NG to containerlab.
Aside the issue with the default password of admin causing the deployment to hang my biggest problem is that I cannot get arp replies to passthrough a bridge.

OS

networks@containerlab1:~/labs/nwby-pe-5v$ cat /etc/os-release 
PRETTY_NAME="Ubuntu 24.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.1 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
networks@containerlab1:~/labs/nwby-pe-5v$ 

containerlab

networks@containerlab1:~/labs/nwby-pe-5v$ containerlab version
  ____ ___  _   _ _____  _    ___ _   _ _____ ____  _       _     
 / ___/ _ \| \ | |_   _|/ \  |_ _| \ | | ____|  _ \| | __ _| |__  
| |  | | | |  \| | | | / _ \  | ||  \| |  _| | |_) | |/ _` | '_ \ 
| |__| |_| | |\  | | |/ ___ \ | || |\  | |___|  _ <| | (_| | |_) |
 \____\___/|_| \_| |_/_/   \_\___|_| \_|_____|_| \_\_|\__,_|_.__/ 

    version: 0.61.0
     commit: 3065a43b
       date: 2025-01-03T14:28:31Z
     source: https://github.com/srl-labs/containerlab
 rel. notes: https://containerlab.dev/rn/0.61/
networks@containerlab1:~/labs/nwby-pe-5v$ 

topology

name: containerlab1-nwby-pe-5v
topology:
  nodes:
    nwby-pe-5v:
      kind: nokia_sros
      image: vrnetlab/nokia_sros:20.10.R1
      license: license-31-40.txt
      cpu-set: 0-7
      type: >-
        cp: cpu=2 min_ram=4 slot=A chassis=SR-12 card=cpm5 sfm=m-sfm6-7/12 system-base-mac=50:09:02:01:00:06 ___
        lc: cpu=2 min_ram=4 slot=1 max_nics=10 chassis=SR-12 sfm=m-sfm6-7/12 card=iom5-e mda/1=me16-25gb-sfp28+2-100gb-qsfp28 mda/2=me16-25gb-sfp28+2-100gb-qsfp28 ___
        lc: cpu=2 min_ram=4 slot=2 max_nics=10 chassis=SR-12 sfm=m-sfm6-7/12 card=iom5-e mda/1=me16-25gb-sfp28+2-100gb-qsfp28 mda/2=me16-25gb-sfp28+2-100gb-qsfp28 ___
        lc: cpu=2 min_ram=4 slot=3 max_nics=10 chassis=SR-12 sfm=m-sfm6-7/12 card=iom5-e mda/1=me16-25gb-sfp28+2-100gb-qsfp28 mda/2=me16-25gb-sfp28+2-100gb-qsfp28
      startup-config: mwby-pe-5v.txt
    br-vlan1202:
      kind: bridge
  links:
    - endpoints: ["nwby-pe-5v:1/1/1", "br-vlan1202:eth1"]
#    - endpoints: ["nwby-pe-5v:1/1/1", "macvlan:ens224"]
      mtu: 9000
networks@containerlab1:~/labs/nwby-pe-5v$ 

bridge

netplan

    vlan1202:
      id: 1202
      link: ens192
      mtu: 9000

    br-vlan1202:
      interfaces:
        - vlan1202
      dhcp4: false
      dhcp6: false
      mtu: 9000

networks@containerlab1:~/labs/nwby-pe-5v$ brctl show br-vlan1202
bridge name	bridge id		STP enabled	interfaces
br-vlan1202		8000.86b94582106c	no		eth1
							vlan1202
networks@containerlab1:~/labs/nwby-pe-5v$ brctl showmacs br-vlan1202
port no	mac addr		is local?	ageing timer
  1	00:50:56:8f:21:41	no		  38.37
  1	00:50:56:8f:af:b8	yes		   0.00
  1	00:50:56:8f:af:b8	yes		   0.00
  1	0c:00:65:10:50:01	no		   2.55
  1	34:1b:2d:a8:eb:5a	no		   1.84
  1	6c:4e:f6:79:f8:7a	no		   1.31
  2	aa:c1:ab:7a:5f:e9	yes		   0.00
  2	aa:c1:ab:7a:5f:e9	yes		   0.00
  1	b4:14:89:d2:32:b9	no		   2.93
networks@containerlab1:~/labs/nwby-pe-5v$ 

iptables

networks@containerlab1:~/labs/nwby-pe-5v$ sudo iptables -L -v --line-numbers
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 DOCKER-USER  all  --  any    any     anywhere             anywhere            
2        0     0 DOCKER-ISOLATION-STAGE-1  all  --  any    any     anywhere             anywhere            
3        0     0 ACCEPT     all  --  any    br-2453c0dce5c3  anywhere             anywhere             ctstate RELATED,ESTABLISHED
4        0     0 DOCKER     all  --  any    br-2453c0dce5c3  anywhere             anywhere            
5        0     0 ACCEPT     all  --  br-2453c0dce5c3 !br-2453c0dce5c3  anywhere             anywhere            
6        0     0 ACCEPT     all  --  br-2453c0dce5c3 br-2453c0dce5c3  anywhere             anywhere            
7        0     0 ACCEPT     all  --  br-vlan1202 any     anywhere             anywhere            
8        0     0 ACCEPT     all  --  any    docker0  anywhere             anywhere             ctstate RELATED,ESTABLISHED
9        0     0 DOCKER     all  --  any    docker0  anywhere             anywhere            
10       0     0 ACCEPT     all  --  docker0 !docker0  anywhere             anywhere            
11       0     0 ACCEPT     all  --  docker0 docker0  anywhere             anywhere            

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (2 references)
num   pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 DOCKER-ISOLATION-STAGE-2  all  --  br-2453c0dce5c3 !br-2453c0dce5c3  anywhere             anywhere            
2        0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  anywhere             anywhere            
3        3   246 RETURN     all  --  any    any     anywhere             anywhere            

Chain DOCKER-ISOLATION-STAGE-2 (2 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 DROP       all  --  any    br-2453c0dce5c3  anywhere             anywhere            
2        0     0 DROP       all  --  any    docker0  anywhere             anywhere            
3        3   246 RETURN     all  --  any    any     anywhere             anywhere            

Chain DOCKER-USER (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 ACCEPT     all  --  any    br-2453c0dce5c3  anywhere             anywhere             /* set by containerlab */
2        3   246 RETURN     all  --  any    any     anywhere             anywhere            
networks@containerlab1:~/labs/nwby-pe-5v$ 

capture on bridge's link

networks@containerlab1:~/labs/nwby-pe-5v$ sudo tcpdump -i br-vlan1202 -nn -e arp
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on br-vlan1202, link-type EN10MB (Ethernet), snapshot length 262144 bytes
15:52:59.108096 0c:00:65:10:50:01 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 194.70.105.85 tell 194.70.105.86, length 46
15:52:59.108332 0c:00:65:10:50:01 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 194.70.105.85 tell 194.70.105.86, length 46
15:52:59.109051 b4:14:89:d2:32:b9 > 0c:00:65:10:50:01, ethertype ARP (0x0806), length 60: Reply 194.70.105.85 is-at b4:14:89:d2:32:b9, length 46
15:53:03.814436 0c:00:65:10:50:01 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 194.70.105.85 tell 194.70.105.86, length 46
15:53:03.814672 0c:00:65:10:50:01 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 194.70.105.85 tell 194.70.105.86, length 46
15:53:03.815409 b4:14:89:d2:32:b9 > 0c:00:65:10:50:01, ethertype ARP (0x0806), length 60: Reply 194.70.105.85 is-at b4:14:89:d2:32:b9, length 46
15:53:08.814671 0c:00:65:10:50:01 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 194.70.105.85 tell 194.70.105.86, length 46
15:53:08.814728 0c:00:65:10:50:01 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 194.70.105.85 tell 194.70.105.86, length 46

capture on the link to vSR --- the reply is missing/filtered

networks@containerlab1:~$ sudo tcpdump -i eth1 -nn -e arp
[sudo] password for networks: 
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
15:53:18.815261 0c:00:65:10:50:01 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 194.70.105.85 tell 194.70.105.86, length 46
15:53:18.815383 0c:00:65:10:50:01 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 194.70.105.85 tell 194.70.105.86, length 46
15:53:24.338852 0c:00:65:10:50:01 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 194.70.105.85 tell 194.70.105.86, length 46

This is between a cisco ASR9K and nokia vSR

ISIS adjacency forms

NWBY-PE-05v    BE100.1202               *PtoP*         Up    28   00:12:41 No  None None

RP/0/RSP0/CPU0:ASRPE3#show isis neighbors BE100.1202
Thu Jan  9 16:04:28.581 UTC

IS-IS CORE neighbors:
System Id      Interface        SNPA           State Holdtime Type IETF-NSF
NWBY-PE-05v    BE100.1202       *PtoP*         Up    21       L2   Unable

Total neighbor count: 1
RP/0/RSP0/CPU0:ASRPE3#

vSR

A:NWBY-PE-05v# show router isis adjacency 

===============================================================================
Rtr Base ISIS Instance 0 Adjacency 
===============================================================================
System ID                Usage State Hold Interface                     MT-ID
-------------------------------------------------------------------------------
ASRPE3                   L2    Up    25   port-1/1/c1/1                 0
-------------------------------------------------------------------------------
Adjacencies : 1
===============================================================================
A:NWBY-PE-05v# 

**But ISIS is using ISO family, anything with ipv4 fails. ARP entry is missing on the vSR

ASR9K**

RP/0/RSP0/CPU0:ASRPE3#show arp be100.1202 location 0/3/CPU0
Thu Jan  9 16:06:01.140 UTC

Address         Age        Hardware Addr   State      Type  Interface
194.70.105.85   -          b414.89d2.32b9  Interface  ARPA  Bundle-Ether100.1202
194.70.105.86   00:00:02   0c00.6510.5001  Dynamic    ARPA  Bundle-Ether100.1202
RP/0/RSP0/CPU0:ASRPE3#

Nokia

A:NWBY-PE-05v# show router arp 

===============================================================================
ARP Table (Router: Base)
===============================================================================
IP Address      MAC Address       Expiry    Type   Interface
-------------------------------------------------------------------------------
63.130.124.190  0c:00:cf:be:27:00 00h00m00s Oth    system
194.70.105.85                     04h00m00s Dyn    port-1/1/c1/1
194.70.105.86   0c:00:65:10:50:01 00h00m00s Oth[I] port-1/1/c1/1
-------------------------------------------------------------------------------
No. of ARP Entries: 3
===============================================================================
A:NWBY-PE-05v# 

I have been trying to figure this out, so far, no joy. Any insight you can offer is appreciated.

BR,
Mel

[hellt]

Hi @msyretis
so far, to me it looks more like a linux networking detail, hence I moved this into a discussion
maybe you would also find more attention to it in our discord

[msyretis]

Hello hellt, so nothing seems off on the containerlab side of things? I dont think there are many things to go wrong. I tried with and without the startup config, even tried partial, same behaviour.

I am not holding my breath on discord.

[marcj04]

Did you ever figure this out? I’m running into the exact same issue

[msyretis]

Sadly no. All I did was add a static arp entry on the Nokia side so that ipv4 unicast starts working. This is just a workaround. Are you running containerlab hosts on baremetal or you are on VMs? If you are on VMs which hypervisor are you using?

[marcj04]

I have it running on an Ubuntu vm on top of Hyper-V. I have been troubleshooting this the last two days and I’ve been losing my mind haha.

[msyretis]

I have tried on ubuntu and rocky, same exact behavior, my hypervisor is esxi 7u3. I am not sure what is up. I see the same behavior on an older eve-ng instance, running again as a vm on esxi.

[marcj04]

So the weird thing on my issue is I can get ARP messages inbound to the containers. Just nothing outbound. So it seems I’ll have to hardcode ARP on both sides for every device which is a pain

[msyretis]

Mine is exactly the opposite. Outbound from the container, the arp is sent and the physical device learns about the container, but inbound from the physical device to the container the arp is never passed to the container, I can see it on the bridge uplink, but not on the port the container attaches, which means that is bridge related, but I was not able to solve it.