Saturday 16 March 2019

How to use 25G ports in 1/10G mode on Cisco NCS (540, 55A2, etc)

I've recently been test driving a Cisco NCS 55A2, which has 24 x 1/10 Gbps and 16 x 1/10/25 Gbps ports. However, I ran into a problem where the 1/10/25 Gbps ports would not rate adapt down to 10 Gbps. There is no "speed" configuration under the port and seemingly nothing useful under controllers relevant to 25G ports.

I eventually got the answer from Cisco engineering but since I couldn't find this documented anywhere on the Internet I thought it might be worth preserving it here for prosperity!

Theory


Unlike 1/10G operation where the device just detects what optic is inserted then presents the appropriate GigabitEthernet or TenGigabitEthernet interface, 1/10/25G ports need to be hard set into either 1/10G or 25G mode. Frustratingly, when it comes to 25 Gbps ports, the NCS platforms set this in groups of four ports, each of which is referred to as a"quad". In another confusing move, Cisco has decided to put the quad config in a completely different place to very similar things such as the config to break out 40/100G ports into 10/25G members.

Configuration


As we can see here, the 1/10/25G ports default to 25G mode, reflected in their TFx/x/x/x naming:

RP/0/RP0/CPU0:55A2(config)#do show interfaces description | begin 0/0/0/24
TF0/0/0/24 admin-down admin-down
TF0/0/0/25 admin-down admin-down
TF0/0/0/26 admin-down admin-down
TF0/0/0/27 admin-down admin-down
TF0/0/0/28 admin-down admin-down
TF0/0/0/29 admin-down admin-down
TF0/0/0/30 admin-down admin-down
TF0/0/0/31 admin-down admin-down
TF0/0/0/32 admin-down admin-down
TF0/0/0/33 admin-down admin-down
TF0/0/0/34 admin-down admin-down
TF0/0/0/35 admin-down admin-down
TF0/0/0/36 admin-down admin-down
TF0/0/0/37 admin-down admin-down
TF0/0/0/38 admin-down admin-down
TF0/0/0/39 admin-down admin-down
Mg0/RP0/CPU0/0 admin-down admin-down

Now, let's configure the first four 25G ports into 1/10G mode. The configuration lives under the "hw-module" branch, and quads are numbered from 0 (up to 3 on this platform). To set the first four ports into 1/10G mode:

RP/0/RP0/CPU0:55A2(config)#hw-module quad 0 location 0/0/CPU0 mode 10g
RP/0/RP0/CPU0:55A2(config)#commit

As we can see, ports 24-27 have transformed into 1/10G ports and now appear with the Tex/x/x/x naming convention:

RP/0/RP0/CPU0:55A2(config)#do show interfaces description | begin 0/0/0/24
Te0/0/0/24 admin-down admin-down
Te0/0/0/25 admin-down admin-down
Te0/0/0/26 admin-down admin-down
Te0/0/0/27 admin-down admin-down
TF0/0/0/28 admin-down admin-down
TF0/0/0/29 admin-down admin-down
TF0/0/0/30 admin-down admin-down
TF0/0/0/31 admin-down admin-down
TF0/0/0/32 admin-down admin-down
TF0/0/0/33 admin-down admin-down
TF0/0/0/34 admin-down admin-down
TF0/0/0/35 admin-down admin-down
TF0/0/0/36 admin-down admin-down
TF0/0/0/37 admin-down admin-down
TF0/0/0/38 admin-down admin-down
TF0/0/0/39 admin-down admin-down
Mg0/RP0/CPU0/0 admin-down admin-down

Let's set the rest to 10G mode, because 25G optics are expensive and I don't have any in the lab:

RP/0/RP0/CPU0:55A2(config)#hw-module quad 1 location 0/0/CPU0 mode 10g
RP/0/RP0/CPU0:55A2(config)#hw-module quad 2 location 0/0/CPU0 mode 10g
RP/0/RP0/CPU0:55A2(config)#hw-module quad 3 location 0/0/CPU0 mode 10g
RP/0/RP0/CPU0:55A2(config)#commit
RP/0/RP0/CPU0:55A2(config)#do show interfaces description | begin 0/0/0/24
Te0/0/0/24 admin-down admin-down
Te0/0/0/25 admin-down admin-down
Te0/0/0/26 admin-down admin-down
Te0/0/0/27 admin-down admin-down
Te0/0/0/28 admin-down admin-down
Te0/0/0/29 admin-down admin-down
Te0/0/0/30 admin-down admin-down
Te0/0/0/31 admin-down admin-down
Te0/0/0/32 admin-down admin-down
Te0/0/0/33 admin-down admin-down
Te0/0/0/34 admin-down admin-down
Te0/0/0/35 admin-down admin-down
Te0/0/0/36 admin-down admin-down
Te0/0/0/37 admin-down admin-down
Te0/0/0/38 admin-down admin-down
Te0/0/0/39 admin-down admin-down
Mg0/RP0/CPU0/0 admin-down admin-down

Easy (when you know how)!