Saturday 16 March 2019

How to use 25G ports in 1/10G mode on Cisco NCS (540, 55A2, etc)

I've recently been test driving a Cisco NCS 55A2, which has 24 x 1/10 Gbps and 16 x 1/10/25 Gbps ports. However, I ran into a problem where the 1/10/25 Gbps ports would not rate adapt down to 10 Gbps. There is no "speed" configuration under the port and seemingly nothing useful under controllers relevant to 25G ports.

I eventually got the answer from Cisco engineering but since I couldn't find this documented anywhere on the Internet I thought it might be worth preserving it here for prosperity!

Theory


Unlike 1/10G operation where the device just detects what optic is inserted then presents the appropriate GigabitEthernet or TenGigabitEthernet interface, 1/10/25G ports need to be hard set into either 1/10G or 25G mode. Frustratingly, when it comes to 25 Gbps ports, the NCS platforms set this in groups of four ports, each of which is referred to as a"quad". In another confusing move, Cisco has decided to put the quad config in a completely different place to very similar things such as the config to break out 40/100G ports into 10/25G members.

Configuration


As we can see here, the 1/10/25G ports default to 25G mode, reflected in their TFx/x/x/x naming:

RP/0/RP0/CPU0:55A2(config)#do show interfaces description | begin 0/0/0/24
TF0/0/0/24 admin-down admin-down
TF0/0/0/25 admin-down admin-down
TF0/0/0/26 admin-down admin-down
TF0/0/0/27 admin-down admin-down
TF0/0/0/28 admin-down admin-down
TF0/0/0/29 admin-down admin-down
TF0/0/0/30 admin-down admin-down
TF0/0/0/31 admin-down admin-down
TF0/0/0/32 admin-down admin-down
TF0/0/0/33 admin-down admin-down
TF0/0/0/34 admin-down admin-down
TF0/0/0/35 admin-down admin-down
TF0/0/0/36 admin-down admin-down
TF0/0/0/37 admin-down admin-down
TF0/0/0/38 admin-down admin-down
TF0/0/0/39 admin-down admin-down
Mg0/RP0/CPU0/0 admin-down admin-down

Now, let's configure the first four 25G ports into 1/10G mode. The configuration lives under the "hw-module" branch, and quads are numbered from 0 (up to 3 on this platform). To set the first four ports into 1/10G mode:

RP/0/RP0/CPU0:55A2(config)#hw-module quad 0 location 0/0/CPU0 mode 10g
RP/0/RP0/CPU0:55A2(config)#commit

As we can see, ports 24-27 have transformed into 1/10G ports and now appear with the Tex/x/x/x naming convention:

RP/0/RP0/CPU0:55A2(config)#do show interfaces description | begin 0/0/0/24
Te0/0/0/24 admin-down admin-down
Te0/0/0/25 admin-down admin-down
Te0/0/0/26 admin-down admin-down
Te0/0/0/27 admin-down admin-down
TF0/0/0/28 admin-down admin-down
TF0/0/0/29 admin-down admin-down
TF0/0/0/30 admin-down admin-down
TF0/0/0/31 admin-down admin-down
TF0/0/0/32 admin-down admin-down
TF0/0/0/33 admin-down admin-down
TF0/0/0/34 admin-down admin-down
TF0/0/0/35 admin-down admin-down
TF0/0/0/36 admin-down admin-down
TF0/0/0/37 admin-down admin-down
TF0/0/0/38 admin-down admin-down
TF0/0/0/39 admin-down admin-down
Mg0/RP0/CPU0/0 admin-down admin-down

Let's set the rest to 10G mode, because 25G optics are expensive and I don't have any in the lab:

RP/0/RP0/CPU0:55A2(config)#hw-module quad 1 location 0/0/CPU0 mode 10g
RP/0/RP0/CPU0:55A2(config)#hw-module quad 2 location 0/0/CPU0 mode 10g
RP/0/RP0/CPU0:55A2(config)#hw-module quad 3 location 0/0/CPU0 mode 10g
RP/0/RP0/CPU0:55A2(config)#commit
RP/0/RP0/CPU0:55A2(config)#do show interfaces description | begin 0/0/0/24
Te0/0/0/24 admin-down admin-down
Te0/0/0/25 admin-down admin-down
Te0/0/0/26 admin-down admin-down
Te0/0/0/27 admin-down admin-down
Te0/0/0/28 admin-down admin-down
Te0/0/0/29 admin-down admin-down
Te0/0/0/30 admin-down admin-down
Te0/0/0/31 admin-down admin-down
Te0/0/0/32 admin-down admin-down
Te0/0/0/33 admin-down admin-down
Te0/0/0/34 admin-down admin-down
Te0/0/0/35 admin-down admin-down
Te0/0/0/36 admin-down admin-down
Te0/0/0/37 admin-down admin-down
Te0/0/0/38 admin-down admin-down
Te0/0/0/39 admin-down admin-down
Mg0/RP0/CPU0/0 admin-down admin-down

Easy (when you know how)!

6 comments:

  1. hi,
    thanks you for your post.
    ok for change the interface 25 to1/10g
    but when I insert a SFP 1g in interface 24 to 31 the inteface don't up.
    it's ok in 1 to 23.
    are you a idea?

    ReplyDelete
    Replies
    1. Sorry it took me so long to reply but for anyone reading in the future... in general the 25G capable ports do not support 1G auto negotiate clause 37 for fibre so you will need to disable autonegotiate on the switch end for this to come up. Yes, even on fibre.

      Delete
    2. Hello!
      Thank you for detailed description.
      I have a question: there is an SFP card which supports all modes 1/10/25. Do you know, is it possible to switch quad group mode from 1/10 to 25 and from 25 again to 1/10 having such multi-mode SFP inserted into slot of quad group?
      Thank you in advance!

      Delete
    3. So, if you knock those ports down to 10 Gbps, can you the use a SFP-10G-SR (or -LR) SFP in them (a.k.a. an SFP+) or do you still need to use an SFP-H25G-CU5M (a.k.a. an SFP 28)?

      Delete
    4. You use normal SFP+ such as SFP-10G-SR or SFP-10G-LR. DAC also work but I'm not sure if they are officially supported.

      Delete
  2. The ncs is connected with cisco 2960

    ReplyDelete