err-disbaled cause loopback

One of our Port channel ports came up as err-disbaled with the cause stated as "loopback". This in turn put the portchannel into the err-disbaled state and caused the spanning tree backup port to become active.

(config)#do sh inter status err
Port      Name               Status       Reason               Err-disabled Vlans
Gi0/22    PORT channel 1 to  err-disabled loopback
Port channel config which shows the interface gig0/22 is in the down state.

#sh ether sum
Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        H - Hot-standby (LACP only)
        R - Layer3      S - Layer2
        U - in use      f - failed to allocate aggregator
        M - not in use, minimum links not met
        u - unsuitable for bundling
        w - waiting to be aggregated
        d - default port

Number of channel-groups in use: 1
Number of aggregators:           1
Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
1      Po1(SU)         LACP      Gi0/21(P)   Gi0/22(D)

Here is the spanning tree output that shows the alternate interface gig0/23 is now the prefered path to the root. The cost of the link for the port channel has also increased to 20000, normally it is 10000, which means it will be chosen as the root as the cost is lower.

MST1
  Spanning tree enabled protocol mstp
  Root ID    Priority    4097
             Address     0021.915e.1900
             Cost        20000
             Port        23 (GigabitEthernet0/23)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
  Bridge ID  Priority    32769  (priority 32768 sys-id-ext 1)
             Address     a40c.c315.ba80
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi0/23              Root FWD 20000     128.23   P2p
Po1                 Altn BLK 20000     128.56   P2p
The reason for the lookback error to disable the interface is because the interface sends out periodic keepalive messages and if the interface recieves one of its own keepalives back it will put the interface in a err-disabled state and assume that there is a loop in the network.

This is not the case in our situation as the layer 2 layout and final active topology is well understood. The issue may be occuring due to the compatibility between the DLINK and Cisco switches. More investigation will need to be done to verify this.

So how do we re-enable the interface that is in the err-diabled state and restore the layer 2 topology.
You can set the err-recovery cause for the loopback err-disbaled state by using

(config)#errdisable recovery cause loopback

and then setting the interval time to wait before re-enabling the interface, (seconds)

(config)#errdisable recovery interval 200

In our case though I decided to disable keepalives on both interfaces in the port channel so this error will not occur and the layer 2 topology will remain contsant. Port channel will be the root port for spanning tree and will be in a forwarding state.

(config)#interface gig0/21
(config-int)#no keepalives
(config)#interface gig0/22
(config-int)#no keepalives

Then you have to shudown the err-diabled port and then renable is with the no shutdown command. This will bring the port channel group out of the err-disabled state and resume the appropriate layer 2 topology.

#sh span mst 1
##### MST1    vlans mapped:   2-3,500,600,605,900
Bridge        address a40c.c315.ba80  priority      32769 (32768 sysid 1)
Root          address 0021.915e.1900  priority      4097  (4096 sysid 1)
              port    Po1             cost          10000     rem hops 19
Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Gi0/23           Altn BLK 20000     128.23   P2p
Po1              Root FWD 10000     128.56   P2p

Link:
http://www.cisco.com/en/US/tech/tk389/tk621/technologies_tech_note09186a00806cd87b.shtml

Spanning Tree MST

Spanning Tree MST works on the principal that your infrastructure doesn't really physically change, you simply setup a layer 2 path then associate vlans with that path.

You create regions/instances and associate vlans with the region.

Check your current VLANs.

Switch#
sh vlan brief

You will need to setup each switch individually with the following code:

Switch(config)#
spanning-tree mode mst
spanning-tree mst configuration
name CISCO
revision 1
instance 1 vlan 10-50,60
instance 2 vlan 70-100,110,130
show current ! this will show the current (blank) mst configuration
show pending ! this will show the changes to be commited on exit
exit


This will create two MST instances and put the named vlans in them.

Switch#
show span mst config

To set the root bridge on each MST instance use:

SwitchALPHA(config)#
spanning-tree mst 1 root primary
spanning-tree mst 2 root secondary

SwitchBETA(config)#
spanning-tree mst 1 root secondary
spanning-tree mst 2 root primary


Other commands

Switch#
show spanning-tree mst 1 detail
show spanning-tree interface vlan 20 detail

Troubleshooting

Some old IOS may have pre-ieee mst standard which when you do a 'sh spann' the type field is shown as
- Bound PVST
- Pre-STD-Rx

The normal/default is p2p.

Bound PVST means the switch has fallen back to slow legacy PVST.
Pre-STD-Rx means the neibour is running a pre-ieee cisco propriotary version of mst.

Upgrade the older switch or hard code on the interface configure:
spanning-tree mst pre-standard
This will turn off the automatic/detection which is not 100% reliable.

Ideally all mst switch interface types should be: p2p.

Clear Spanning Tree

Shutdown interfaces or clear the detected-protocols on EVERY switch.

Switch(config)#
interface range g0/1-2
shut
no shut

Switch#
clear spanning-tree detected-protocol

Spanning Tree Tips

Show blocked ports STP
show spanning-tree blockedports

Port Priorities
Set your root bridge on vlan 1 with:
Switch1(config)# spanning-tree vlan 1 root primary
This will set the priority to 24576

On the secondary switch use:
Switch2(config)# spanning-tree vlan 1 root secondary

This will set the priority to 28672

Say you have two trunk links bettween switches and you want to use one port over the other, for example:

Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- ----------------------------
Fa0/7 Desg FWD 19 128.9 P2p
Fa0/8 Desg FWD 19 128.10 P2p

The current Port priority is 128, this is the default.

To change individual priorities use:
Switch1(config-if)# spanning-tree port-priority 112

Now when you show spanning-tree:

Switch1# sh span

Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- ----------------------------
Fa0/7 Desg FWD 19 128.9 P2p
Fa0/8 Desg FWD 19 112.10 P2p

This will then send out a topology change. The remote switch connected to fa0/8 will now use the port with the lower priority instead of the default fa0/7.

Switch2# sh span

Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- ----------------------------
Fa0/7 Altn BLK 19 128.13 P2p
Fa0/8 Root FWD 19 128.14 P2p

The process of choosing which links to use and which to turn off include, lowest root priority, lowest port priority, lowest switchport/mac address.

PortFast
When configuring switchports for a host you can use:

Switch(config)# int range fa0/1-22
Switch(config-if-range)# switch mode access
Switch(config-if-range)# spanning-tree portfast

This will staticly make the port an access port and will not negotiate trunk links etc, which is the default.

If you use switchport host this will do access and portfast for you.

Switch(config)# int range fa0/1-22
Switch(config)# switch host

To verify the port fast configuration use the command

show spanning-tree interface Fa0/2 portfast

Trunk Links
To turn off DTP (dynamic trunking protocol) on your trunk link use:
Switch(config)# switch nonegotiate

If you trunk a Cisco switch with a non-Cisco switch, this is best practise as the other switch does not understand the DTP messages.
This will also speed up convergence time to up to 2 seconds on boot. This is recommended on all 'stable' trunk links, cisco-to-cisco or cisco-to-other.

Alias
Switch1(config)# alias configure fa int range fa
Switch1(config)# fa 0/1-24
Switch1(config-if-range)#

Portfast on trunks

Portfast can be enable on trunk links, this is useful when connect to a server that needs VLAN's configured. (Vmware). This can only used on switchport trunks that do not connect to other switches. as this may cause a layer 2 loop
Switch1(config)# interface gig0/1
Switch1(config int)#spanningtree portfast trunk

BPDU Guard
This should be set on access layer ports or ports that should not go to another switch or device that creates BPDU's. I will shutdown the port in err-disabled state.