Cisco Wireless Controllers: High Availability

A few gotcha's from bringing up a High Availability pair of Cisco 5508s.

1. Do not try to straight up follow Cisco's directions of upgrading from the boot up wizard
2. Bring up new interfaces in the same subnets as per directions etc. etc. 
3. First you have to upgrade both controllers to the same image (makes sense)
4. upgrade both controllers to the same FUS image
5. Manually set your HA Sku'd controller as secondary, even though cisco says it isn't necessary...
6. If you like the GUI, you won't be able to use it on the service port anymore, so delete your network route, and use it from the regular LAG interface management port.  If you are good with CLI then you can still use the service port
7. Enable SSO on your PRIMARY controller and let it reboot   (all of cisco's directions imply you should do this at the same time on both controllers, but that doesn't work)
8. Once the Primary controller is back up in SSO state, then you can enable SSO on the secondary controller, wait for it to reboot a couple times, and look for "ACTIVE HOT"
9. Tell your customers they now have sub-second failover and see them not care


Hopefully these can save someone a bit of troubleshooting


 

STP Enhancements Part 1

STP Enhancements Part 1 - BDPU Guard, BPDU Filter, BPDU Root Guard

BPDU Guard - BPDU is guarding your switch against BPDUs.  Wait, I thought BPDUs were a good thing?  Yes but only on ports uplinked to other switches, ports you expect to have BPDUs.

Why:  When you configure your switch to connect to hosts (PCs and Servers) you are probably using portfast.  If a user ever connected a switch, or some wires got crossed and a switch got plugged in to a host port, or a linux user enabled a software bridge etc. etc.... you could have a network loop 

What:  BPDU guard is a port setting that will shut down a port if it ever sees any BPDU

How:

Globally:

spanning-tree portfast edge bpduguard default

*make sure you disable bpduguard on the uplink ports you expect to have switches plug into: the global command affects all ports with portfast enabled*

Per Interface:

spanning-tree bpduguard enable

Recovery:  if BPDU guard triggers (sees a BPDU) it places the port in an error-disabled state, and the port is effectively shut down.  You can recover by (first removing the offending device), then logging into the switch, and issuing a shut command, followed by no shut.

If you would like the switch to be able to re-enable the port by itself, you can use error-disabled's regular methods:

errdisable recovery cause bpduguard

errdisable recovery interval 400

This will cause the switch to shut down the port for 400 seconds, then if it isn't receiving BPDUs, re-enable the interface.

*Note that BPDU Guard is triggered by BPDUs, so if a user plugs in a home/residential switch, it will not trigger as small home switches do not send BPDUs or participate in spanning tree at all.  This will not have any effect on home switches or hubs. (unless there is a loop and the cisco switch see's it's own BPDU sent back to itself, but then you already have a problem!)

BPDU Filter:

Why: Filter is slightly trickier as the global command and the interface command perform two different functions.  

1. First the interface method.  If you apply BPDU filtering to a specific interface, it effectively turns off spanning-tree.  

2. The global method is intended to save on bandwidth/proc overhead by not sending BPDUs out host ports.

What:

1. Interface Method: When turned on for a specific interface, that interface will ignore BPDUs sent to it, and it will not send any BPDUs.  It will continue forwarding traffic and not participate in spanning-tree.

2. In global mode: all interfaces in portfast mode will be put in filtering mode.  They will send out 10 BPDUs when the port comes up, and assuming no incoming BPDUs the switch will stop sending any BPDUs.  If a port ever receives a BPDU at any time, it will lose it's BPDU filter mode, removes the portfast status, and begins regular spanning tree listening/learning process.

How:

global mode

config)# spanning-tree portfast bpdufilter default

interface mode

(config-if)# spanning-tree bpdufilter enable

*note - enabling both BPDU guard and BPDU filtering on the same interface - BPDU filtering takes precedence and BPDU guard has no effect

Root Guard:

Why: ensure an interface will never be used as a root port, increasing STP stability.  Now at first it is tempting to think you should add root guard to all non-root ports but don't forget, you have a redundant topology for a reason.  You need your STP domain to be able to reconverge in the case of a failure and elect a new root bridge/use a different root port.  When you choose what ports get root guard, take into consideration any paths you might ever want your traffic to traverse and do not implement root guard on them.  The easiest and preferred way might be to just use this on edge/access ports and leave it off for all switch-to-switch connections, but perhaps your topology has a "leaf loop" off of your primary STP loop that you never want to use as an STP root, and would rather stop forwarding traffic than allow a switch out there to become root... maybe.

Anyway, root guard is designed so that if anyone plugs a new or non-corporate switch into your network, and it happens to have a low priority, it won't become root.

What:  If the root guard port receives a BPDU with superior priority to what the LOCAL switch is seeing (not the root's own priority) the port will move to a "root-inconsistent state" and stop forwarding traffic.  If the switch is removed or the priority is raised, the port will move out of root inconsistent state on it's own and begin forwarding traffic (after moving through listening/learning states).  No user intervention or config changes needed to unblock the port.

This feature does not have a global function, only a per-interface function.

config# int fa0/2

config-if)# spanning-tree guard root

*note - if you are just enabling this on host ports to protect your STP topology, just use BPDU guard instead.  The only useful time I can think of using root guard: if you want to create a policy that all un-used uplink ports have root guard configured to protect the topology.  When a new switch is added the root is protected, and once the switch is up and operational you can remove the root guard feature.

Spanning Tree Portfast

Portfast - didn't think I would have enough for a whole post, but there are a few items worth mentioning.


What is it - Typically, if a host (server, PC, etc.) plugs into a port, spanning tree will run to ensure there is not a network loop before allowing the host to talk.  If you are running the default, Cisco per-vlan spanning tree 802.1d, this takes 30 seconds for both listening and learning mode to run.

The problem - Thirty seconds is a long time to wait, especially when it means you are not getting DHCP responses and your computer is deciding that it must not be on a real network so it doesn't need to run all the network startup scripts.

The solution - configuring portfast on an interface tells that switch that only a host (nothing that could cause a loop) is connected.  The switch then allows the port to immediately transition to forwarding (but will loose it's portfast status if a BPDU is recieved)

The dangers - if you mis-configure this, and turn portfast on for an uplink to another switch, hub, etc. a loop could form and crash your network before spanning tree ever has a chance to prevent it.

The terms:  in rapid spanning tree protocol (802.1w) they are called "edge ports" but are still configured with the same portfast command. 

The commands:

interface fa0/1
spanning-tree portfast
exit

or

interface fa0/1
switchport host
 *This turns on portfast and also disables channeling and trunk negotiation*

or globally from privileged exec mode

#spanning-tree portfast default
*In the global form of portfast, immediate forwarding is enabled for all access ports (NOT ANY TRUNK PORTS) but you should still manually input the command no spanning-tree portfast for any ports that may be connected to other switches*


spanning-tree portfast trunk
*This command is to turn portfast on for a trunk.  You would typically have a trunk enabled for a host such as a server that needs multiple vlans, or a VoIP phone*


If a BPDU is ever recieved on an edge/portfast port, it loses it's portfast status AND sends a Topology Change Notification to all other switches in the STP domain.

Too Long

Sorry about no posts in forever, life/work have been crazy.

I did pass my CCNP-Route so I might have a few posts from notes to bring, but will probably start focusing on SWITCH topics.

Let me know if anyone has a topic request.

dt-lacp Distributed Trunking

Distributed Trunking -

Now everyone understands how two switches connected logically (stacked) can balance an aggregate link between them with a shared logical backplane, as cisco does, but what about HP who doesn't have stacked switches, where just a few higher end switches have the ability to do distributed trunking. A server (as it is not supported for switch to switch) can connect to two separate switches and create a single LACP trunk to both.

*note - the two switches must be directly connected, and it is a little unclear but it sounds like they have to have a dedicated link between them as the dt-trunk link, apart from the regular link. (now this does not have to be aggregated, could be a single gig connection, but as you will see in a moment it should be of equivalent bandwidth to each switch's lacp link.

Now this seems simple but what happens under the hood? With HP's usual lack of documentation it took me a while to dig up, but basically, create a non-negotiating trunk. (eg. lacp auto will not work), and connect the server. The server sends a frame to switch A which learns the mac and forwards it to the destination. Switch B also learns the mac as being on switch A. Switch B receives a frame and, having learned the mac as being on switch A, forwards the frame to switch A to send to destination.

So in essence, you are creating redundancy, but as for doubling bandwidth, ALL traffic will proceed through a single switch so both the inter-switch link and EACH switch's core uplink must be able to support the full server lacp bandwidth, for all it's links.

Eg. Server has 2gigs to switch A and 2gigs to switch B. Switch A needs at minimum 2gigs to switch B. Switch A and Switch B both need at minimum 4 gigs up to the uplink/core. (assuming you want the server to have 4gigs of bandwidth to the core) This is of course on top of any other traffic the switches will be forwarding from other devices.

Of course spanning tree is going to have to block a link somewhere anyway, so all traffic forwarding through a single switch may be the expected outcome anyway.

Cheers!

Oh ya, the config:

ProCurve Switch 1(config)# switch-interconnect a1
ProCurve Switch 2(config)# switch-interconnect a1
 ProCurve Switch 2(config)# trunk b20-b21 trk99 dt-lacp
ProCurve Switch 1(config)# trunk c10-c11 trk99 dt-lacp

 and of course if you needed an aggregate link between switches, replace a1 with trk5 and add whatever ports you needed to trk5 just like a regular trunk port.

  show lacp distributed is your show command to see the lacp status of BOTH switches