Minimal SNMP View for Solarwinds Management of a Cisco Device

Some time ago, I was asked to provide a customer read-only SNMP access to a router so that they could monitor bandwidth utilisation. Unfortunately the device had some sensitive configuration which needed to be protected, so giving access to the full MIB was not an option. With that in mind, I set up an SNMP view which permitted access to the ifMIB and nothing else, thinking that would be sufficient to monitor interface usage.

The customer tested and soon came back to advise me that, although he could successfully run an SNMP walk, his Solarwinds NMS could not discover the device and therefore he couldn't get the stats he needed.

It turns out that Solarwinds needs a particular set of OIDs to be visible before it will allow a device to be discovered and brought under management. There are a few articles on the knowledge base about which OIDs it uses for various purposes, e.g:

http://knowledgebase.solarwinds.com/kb/questions/1196/What+object+IDs+(OIDs)+does+Orion+NPM+poll+for+interface+information%3F+What+types+of+interface+information+does+Orion+NPM+poll%3F

http://knowledgebase.solarwinds.com/kb/questions/1196/

However, there doesn't seem to be one directly addressing the question of which OIDs are required just to bring a device under management.

Eventually I think I gave up and sniffed a discovery off the wire and looked at which OIDs it used -  I don't exactly remember any more. Anyway, here is a list that seems to do the trick. The following CLI should configure a minimal set of OIDs on a Cisco device:

snmp-server view STATS iso excluded
snmp-server view STATS mib-2 excluded
snmp-server view STATS cisco excluded
snmp-server view STATS system.1.0 included
snmp-server view STATS system.2.0 included
snmp-server view STATS system.4.0 included
snmp-server view STATS system.5.0 included
snmp-server view STATS system.6.0 included
snmp-server view STATS ifIndex included
snmp-server view STATS ifDescr included
snmp-server view STATS ifSpeed included
snmp-server view STATS ifOperStatus included
snmp-server view STATS ipAddrEntry.2 included
snmp-server view STATS lsystem.8 included
snmp-server view STATS lsystem.58 included
snmp-server view STATS chassis.6 included
snmp-server view STATS ifName included
snmp-server view STATS ifHCInOctets included
snmp-server view STATS ifHCInUcastPkts included
snmp-server view STATS ifHCInMulticastPkts included
snmp-server view STATS ifHCInBroadcastPkts included
snmp-server view STATS ifHCOutOctets included
snmp-server view STATS ifHCOutUcastPkts included
snmp-server view STATS ifHCOutMulticastPkts included
snmp-server view STATS ifHCOutBroadcastPkts included
snmp-server view STATS ifHighSpeed included
snmp-server view STATS ifAlias included
snmp-server view STATS ciscoMemoryPoolEntry.5 included
snmp-server view STATS ciscoMemoryPoolEntry.6 included
snmp-server view STATS cpmCPUTotalTable.1.5 included
snmp-server view STATS cpmCPUTotalTable.1.8 included

Once the view is defined, you just need to apply it to the particular community string for SNMPv2:

snmp-server community mycomm view STATS RO

Or the user for SNMPv3:

snmp-server group Monitoring v3 priv read STATS write STATS

This config allowed SolarWinds to discover the device and bring it under management. It also allowed basic port stats to be collected and general up/down alarms to be raised.

Of course, you can add more OIDs into the "included" list as needed for your particular use case, however these should be enough for SolarWinds to discover the device.

Quiet Mode on ME3x00 Platforms

The Cisco ME3x00 range of devices comes with automatic lockout for SSH as default. If too many bad login attempts are made within a short period of time, the device will go into a locked down state called "quiet mode" which blocks any new management connections.

In a default configuration, entering quiet mode causes the device to completely refuse any and all new telnet, SSH and port 80 sessions directed towards it. Existing (open) sessions are not affected. If you've ever tried a couple of credentials out, then suddenly started getting "connection refused", you have probably run into this feature!

The default threshold and timer values are as follows:

• An artificial 1 second delay is added to each login
• 5 bad logins within 60 seconds triggers a lockout
• lockout lasts 5 minutes

If that's all you needed to know then I suppose you can go now :)

If you're interested in checking / tweaking the settings, read on.

The commands to view and edit settings relevant to this feature all centre around "login", for example:

ME3x00#show login
     A default login delay of 1 seconds is applied.
     No Quiet-Mode access list has been configured.

     Router enabled to watch for login Attacks.
     If more than 5 login failures occur in 60 seconds or less,
     logins will be disabled for 300 seconds.

     Router presently in Normal-Mode.
     Current Watch Window remaining time 37 seconds.
     Present login failure count 1.

ME3x00#

One thing to note here is that if you try to authenticate with public keys (like openssh and others do by default) it will refuse and count that as a failure. If you bail out at the password prompt (i.e. ctrl-C or leave it time out) then that also counts as a failure. It doesn't take long to get to 5! Attempts blocked by the VTY ACLs don't count.

If you would like to force a longer delay between login attempts, you can adjust the value (between 1 and 10 seconds) under config mode as follows:

ME3x00(config)# login delay 5

The lockout thresholds can easily be changed using the following command:

ME3x00(config)# login block-for 120 attempts 10 within 60

This example would cause the device to trigger a two minute lockdown into quiet mode if it saw 10 failed logins within 60 seconds.

Now the 5 minute default logout is quite a long time and being able to force a lockdown could be advantageous to attackers - if you can prevent an administrator from being able to log into a device then it makes it far more difficult for him to detect and / or mitigate attacks on the network. Luckily there is a feature available to effectively whitelist management traffic that should always be allowed through, even when the device goes into lockdown. This comes in the form of the "login quiet-mode access-class", which basically decides what access controls are put in place when the device enters quiet mode. By default, the quiet-mode access-class is set to "sl_def_acl", which is an omnipresent ACL which looks like this:

ME3x00#show ip access-list sl_def_acl
Extended IP access list sl_def_acl
    10 deny tcp any any eq telnet (7 matches)
    20 deny tcp any any eq www
    30 deny tcp any any eq 22 (72 matches)
    40 permit ip any any

ME3x00#

In order to provide back door access when the device enters quiet mode, simply define an ACL which permits the desired traffic but blocks all other management traffic, such as:

ME3x00(config)#ip access-list extended quiet_mode_access

ME3x00(config-ext-nacl)#permit tcp host 10.1.1.1 any eq 22
ME3x00(config-ext-nacl)#deny tcp any any eq telnet
ME3x00(config-ext-nacl)#deny tcp any any eq www
ME3x00(config-ext-nacl)#deny tcp any any eq 22 
ME3x00(config-ext-nacl)#exit

Then apply it to the box using:

ME3x00(config)#login quiet-mode access-class quiet_mode_access 

That way a dedicated management box (10.1.1.1) will always be able to connect and manage the device, even if it is in quiet mode, while everything else will be locked out.

Logging

The device also generates some helpful syslog messages to tell you what it's doing, for example here is my device going into quiet mode:

%SEC_LOGIN-1-QUIET_MODE_ON: Still timeleft for watching failures is 22 secs, [user: billybob] [Source: 10.10.10.10] [localport: 22] [Reason: Login Authentication Failed] [ACL: sl_def_acl] at 16:30:00 BST Sun Oct 5 2014

And coming out again:

%SEC_LOGIN-5-QUIET_MODE_OFF: Quiet Mode is OFF, because block period timed out at 16:35:00 BST Sun Oct 5 2014

References

Cisco IOS Security Configuration Guide