Archive for March, 2014

1GB SFP on Cisco Nexus 5000

March 19th, 2014 Comments off

I’m setting up a new Nexus 5000 vpc cluster. For the keep alive links, I used the old 1GB SFP modules because the links don’t send much traffic. However, it didn’t work

1 sfp

Now that’s interesting because from the Cisco Datasheet, it says

Supported Tranceiver Options:
Cisco SFP-10G-SR 10GBASE-SR SFP+ module (MMF)
Cisco SFP-10G-LR 10GBASE-LR SFP+ module (SMF)
Cisco SFP-H10GB-CU1M 10GBASE-CU SFP+ cable 1m (Twinax cable)
Cisco SFP-H10GB-CU3M 10GBASE-CU SFP+ cable 3m (Twinax cable)
Cisco SFP-H10GB-CU5M 10GBASE-CU SFP+ cable 5m (Twinax cable)
Cisco GLC-T 1000BASE-T SFP
Cisco GLC-SX-MM GE SFP, LC connector SX transceiver (MMF)
Cisco GLC-LH-SM GE SFP, LC connector LX/LH transceiver (SMF)
Cisco SFP-GE-T 1000BASE-T SFP, extended temperature range
Cisco SFP-GE-S GE SFP, LC connector SX transceiver (MMF), extended temperature range and digital optical monitoring (DOM)
Cisco-SFP-GE-L GE SFP, LC connector LX/LH transceiver (SMF), extended temperature range and DOM
Cisco DS-SFP-FC4G-SW 4-Gbps Fibre Channel SW SFP, LC (for Fibre Channel expansion module ports)
Cisco DS-SFP-FC4G-LW 4-Gbps Fibre Channel LW SFP, LC (for Fibre Channel expansion module ports)
Cisco DS-SFP-FC8G-SW 8-Gbps Fibre Channel SW SFP+, LC (for Fibre Channel expansion module ports)
Cisco DS-SFP-FC8G-LW 8-Gbps Fibre Channel LW SFP+, LC (for Fibre Channel expansion module ports)

So I spent a little bit of time playing with the config. It turned out that you have to tell the switch to run at 1Gb on the SFP

2 sfp

I’m on the newest version of Nexus, ver 7.0 N1(1). Not sure why it doesn’t auto-detect the speed. The NX-OS is still evolving, stay tuned for more tips and tricks.

Categories: Uncategorized

Cisco UCS : Tracing packet paths with a MAC address

March 14th, 2014 Comments off

In the UCS world where a virtual NIC on a virtual server is connected to a virtual port on a virtual switch by a virtual cable, it is not surprising that there can be confusion about what path packets are actually taking through the UCS infrastructure.

Similarly knowing the full data path through the UCS infrastructure is essential to understanding troubleshooting and testing failover.

In this post I will demonstrate how to trace the paths of the packets in a Cisco UCS Data Center.

The diagram below shows a Half width blade with a vNIC called eth0 created on a Cisco VIC (M81KR) with its primary path mapped to Fabric A. For simplicity only one IO Module to Fabric Interconnect link is shown in the diagram, as well as only one of the Host Interfaces (HIFs / Server facing ports) on the IO module.

1 overview

With the MAC address, you first need to find out the virtual circuit number with the following commands. Note that it will show nothing if you are in the wrong FI.

2 mac to veth

With the Veth#, now we can find the Chassis/Server ID with this command

3 veth to chassis-server

We can go further and find the Uplink/Border Interface where the Fabric Interconnect connects to the LAN with this command

4 veth to uplink

Next , we will find the FI port (Server port) that connect to this virtual circuit with the following command

Where Ethernet #/#/# is the “Bound Interface” you found above with the “show int veth #” command
3 veth to chassis-server

Now you should have the server port (Fabric-if), to find the
FEX Network Port
5 F-if to FEX-uplink

The steps above should help you identify the paths of the packets. For in depth network troubleshooting , see the following Cisco slide