After several hours of troubleshooting wireless and looking through obscure bugs, I had to come back to the Keep It Simple Stupid idea. So I dig through the log files and find a few errors that maybe I will help someone else if they see them.
%APF-4-REGISTER_IPADD_ON_MSCB_FAILED: apf_foreignap.c:1281 Could not Register IP Add on MSCB. MSCB still in init state.
The reason you will see this error is pretty simple. You are getting packets for or from a client (such as ARP requests) and the client doesn't have an IP address yet. Without an IP address, the "MSCB" is said to be in the init state.
%APF-1-CHANGE_ORPHAN_PKT_IP: apf_foreignap.c:
The registered IP address changed leaving packets "orphaned". Note, I saw this in regard to clients not being able to get DHCP and the changing IP address was 169. addresses. This is also a common error with mac clients trying to hold on to old IP addresses.
%DHCP-3-BIND_SRPORT_ERR: dhcp_support.c:374 Binding service port failed.
There is no info on cisco about this error (other than check the bug tool, which has nothing) and I thought this was where my problem lied. The controller was having problems binding the DHCP addresses to the clients. Well low and behold, I found it wasn't that the controller was unable the DHCP response, this message means that there was either no response or an error response. In my case I had run out of addresses, the entire /22 was full. So now I get to enlarge the network and DHCP pool.
So in conclusion, if your clients aren't getting IP addresses, before you dig through obscure bugs and error messages, make sure you pool isn't used up!
Thursday, September 30, 2010
Tuesday, September 28, 2010
P@ssw0rd
You know you have been typing too many passwords when you accidently type 3x1t and l0g0ut when you try to exit your network gear...
Monday, September 27, 2010
Use twitter with Geotagging?
I personally don't like the idea of telling the whole internet exactly where I am at any given time, but Josh O'Brian at StaticNat blog points out some business dangers. Do you use geotagging with your tweets or posts? Do you tweet about problems or issues you are having with the network or software to get help? See how that could be a problem?
Click here for Josh's article.
Click here for Josh's article.
New Internet Wiretap legislation
The Obama administration attempts to send "sweeping new legislation" demanding "all services that enable communications" including blackberry, facebook, peer to peer messaging and Skype have the ability to allow government wiretaps, unencrypting all communications to plain text. http://www.nytimes.com/2010/09/27/us/27wiretap.html
Well first off, this would appear to be a terrible idea. Forcing companies to create backdoors and master private keys is just adding new avenues for attack.
Perhaps this is just one sided journalism, but I noticed they were not able to find any tech or industry professionals to quote in support of this bill, only against. All the "for" quotes are by politicians. This is also an amazingly poor time to bring up something so fiercly unpopular just before November Elections.
Anyway, looking at the technical aspect:
Facebook, I'm sure they store every keystroke ever entered on their site for advertising purposes anyway, so they shouldn't have a problem.
Blackberry, I assume this only includes the encryption built into the default mail program. Users should be able to still encrypt their own messages with a key system if they have the technical knowhow, if blackberry was forced to ban personal encryption from their devices, well I would suspect every major corporation would ditch blackberry. Do you want any joe smoe at an ISP or blackberry corporate being able to view defense data being e-mailed between people at Lockheed Martin or Boeing? I assume this would mean government oversite committees would have to be created (and funded with our tax dollars) for every company dealing with government contracts and want to use a blackberry?
Skype: Ok, so how would this work. Skype is not like vonage or a regular phone company that sends everything through a central office, skype is peer to peer. After the call creation the users are talking to each other, not a skype server. So do wiretapped phones then get re-routed to skype servers? Wouldn't this be noticed and affect service? I'm sure people would make programs to alert them if Skype wasn't being routed directly to the end user.
Peer2Peer Messaging: Well, there is certainly a variety here, from IRC to ICQ to MSN Messenger to google to .... Microsoft and google might be able to keep track of messages fairly easy, but IRC? I can't imagine what it would take to get all the IRC servers around the world to listen to the FBI/NSA. IRC has been part of wars, coups, and rebellions without anyone being able to stop the flow of information. I doubt a US wiretap request for "fear someone might be a terrorist" is going to change that.
My personal oppinion (if you couldn't tell from the post) terrible idea, unpopular politically and socially, and a nightmare of new costs, both trying to implement something like this as well as the lawsuits/profit loss that would come from any new security holes that are created and used by malicious users. This could also destroy consumer confidence in online usages. (yes I just wrote usages because I was too lazy to think of a real word)
The governments only defense was: it worked out with cell phone companies despite fears. Well, lets just say that is comparing apples and oranges.
Well first off, this would appear to be a terrible idea. Forcing companies to create backdoors and master private keys is just adding new avenues for attack.
Perhaps this is just one sided journalism, but I noticed they were not able to find any tech or industry professionals to quote in support of this bill, only against. All the "for" quotes are by politicians. This is also an amazingly poor time to bring up something so fiercly unpopular just before November Elections.
Anyway, looking at the technical aspect:
Facebook, I'm sure they store every keystroke ever entered on their site for advertising purposes anyway, so they shouldn't have a problem.
Blackberry, I assume this only includes the encryption built into the default mail program. Users should be able to still encrypt their own messages with a key system if they have the technical knowhow, if blackberry was forced to ban personal encryption from their devices, well I would suspect every major corporation would ditch blackberry. Do you want any joe smoe at an ISP or blackberry corporate being able to view defense data being e-mailed between people at Lockheed Martin or Boeing? I assume this would mean government oversite committees would have to be created (and funded with our tax dollars) for every company dealing with government contracts and want to use a blackberry?
Skype: Ok, so how would this work. Skype is not like vonage or a regular phone company that sends everything through a central office, skype is peer to peer. After the call creation the users are talking to each other, not a skype server. So do wiretapped phones then get re-routed to skype servers? Wouldn't this be noticed and affect service? I'm sure people would make programs to alert them if Skype wasn't being routed directly to the end user.
Peer2Peer Messaging: Well, there is certainly a variety here, from IRC to ICQ to MSN Messenger to google to .... Microsoft and google might be able to keep track of messages fairly easy, but IRC? I can't imagine what it would take to get all the IRC servers around the world to listen to the FBI/NSA. IRC has been part of wars, coups, and rebellions without anyone being able to stop the flow of information. I doubt a US wiretap request for "fear someone might be a terrorist" is going to change that.
My personal oppinion (if you couldn't tell from the post) terrible idea, unpopular politically and socially, and a nightmare of new costs, both trying to implement something like this as well as the lawsuits/profit loss that would come from any new security holes that are created and used by malicious users. This could also destroy consumer confidence in online usages. (yes I just wrote usages because I was too lazy to think of a real word)
The governments only defense was: it worked out with cell phone companies despite fears. Well, lets just say that is comparing apples and oranges.
Cisco IOS Hints and Tricks: EIGRP myths debunked
A very clear explanation of why EIGRP is in no way a Link State Routing Protocol, or a Hybrid.
Cisco IOS Hints and Tricks: EIGRP myths debunked: "Matthew Norwood performed a really thorough EIGRP research and unearthed a lot of myths around it, some of them coming from official documen..."
Cisco IOS Hints and Tricks: EIGRP myths debunked: "Matthew Norwood performed a really thorough EIGRP research and unearthed a lot of myths around it, some of them coming from official documen..."
Thursday, September 23, 2010
Wireless Lan Controllers SNMP Traps
Well, it is nice to know after the fact that cisco wireless lan controllers do not send the same information in syslog entrees as they do in SNMP traps. Like User Authentication notices, not in syslog.
So I have an IP address and need to track the username to see what user snuck some bittorrent past my packetshaper block. I grep through all my syslog files and there are no IP addresses, even at the debug level. So now I had to set up an SNMP trap receiver on my linux box as well as syslog server to catch all the needed info.
I also might mention the SNMP traps are horribly ugly, probably purposely so that you will buy WCS or a cisco solution to make them readable. Oh well, it will be 100 years before we get the money for that!
Now if only I could get my logs to rotate properly...
So I have an IP address and need to track the username to see what user snuck some bittorrent past my packetshaper block. I grep through all my syslog files and there are no IP addresses, even at the debug level. So now I had to set up an SNMP trap receiver on my linux box as well as syslog server to catch all the needed info.
I also might mention the SNMP traps are horribly ugly, probably purposely so that you will buy WCS or a cisco solution to make them readable. Oh well, it will be 100 years before we get the money for that!
Now if only I could get my logs to rotate properly...
Monday, September 20, 2010
Iperf - tool for every NetEngineer's Toolbox
Ever wondered, “man, how do I see exactly how much traffic I really can push over this link?” or “I wish I could create some multicast traffic so I can see the bandwidth and make sure everything is working before the vendor gets here.” Well, in steps the simplest program ever, Iperf.
Download the file applicable to your OS. You will need two installs, a source and a destination (Iperf will refer to these as server and client).
Pick one to be your server. Open a command window and browse to the Iperf file. Type: iperf -s
Done, you have an Iperf server running.
Go to machine 2, browse to where you placed the iperf file. To run a simple bandwidth test, just type: iperf -c x.x.x.x where x.x.x.x is the server’s IP address.
View the output, if it is a 100Mb link, you will probably see around 80-85 Mb/s speed and near a 100MB transfered. (If you are running fast, I have seen as quick as 98Mb/s, but that tends to be local to the same switch or nearby)
Now you can start playing with the extra switches. A common one might be:
iperf -c x.x.x.x -r -t 300 -i 2
here, we see -r will run both a regular test from the client to the server, but then reverse it and send traffic from the server to the client. Next we have -t 300 which tells iperf to run for 300 seconds, or 5 minutes. -i 2 informs the program to give you an output ever 2 second interval so you can see what is happening and see any spikes or slowdowns during the test rather than just a averaged output at the end.
Want to test udp traffic, you will need to add a -u on the end of the server command as well as the client command. That lets both sides of the link know to use UDP traffic rather than TCP.
If you are testing a production link, perhaps you don't want to use up the entire link and slow down everyones traffic. use the -b 10m command to set bandwidth to 10Mb/s.
-B x.x.x.x on both ends of the link will bind them to a multicast address, allowing you to look at your router and see the multicast joins and traffic flowing. Don't forget the -T for setting time to live numbers.
This data as pretty specific, and you can show your complainers/customers how well the network handles a solid gig of traffic, when their sluggish program using 50k of bandwidth is not a "slow network issue".
Pair this data with some colorful graphs showing how little they actually use your network and most people accept that the network isn't the problem.
Of course with your excellent customer service skills you have worded this all super politely and let them "arrive at their own conclusion" that there must be something else wrong than the network. Unless of course your gig interface is only able to push 2Mb/s of traffic, then you need to suck it up and figure out why your network is slow!
Download the file applicable to your OS. You will need two installs, a source and a destination (Iperf will refer to these as server and client).
Pick one to be your server. Open a command window and browse to the Iperf file. Type: iperf -s
Done, you have an Iperf server running.
Go to machine 2, browse to where you placed the iperf file. To run a simple bandwidth test, just type: iperf -c x.x.x.x where x.x.x.x is the server’s IP address.
View the output, if it is a 100Mb link, you will probably see around 80-85 Mb/s speed and near a 100MB transfered. (If you are running fast, I have seen as quick as 98Mb/s, but that tends to be local to the same switch or nearby)
Now you can start playing with the extra switches. A common one might be:
iperf -c x.x.x.x -r -t 300 -i 2
here, we see -r will run both a regular test from the client to the server, but then reverse it and send traffic from the server to the client. Next we have -t 300 which tells iperf to run for 300 seconds, or 5 minutes. -i 2 informs the program to give you an output ever 2 second interval so you can see what is happening and see any spikes or slowdowns during the test rather than just a averaged output at the end.
Want to test udp traffic, you will need to add a -u on the end of the server command as well as the client command. That lets both sides of the link know to use UDP traffic rather than TCP.
If you are testing a production link, perhaps you don't want to use up the entire link and slow down everyones traffic. use the -b 10m command to set bandwidth to 10Mb/s.
-B x.x.x.x on both ends of the link will bind them to a multicast address, allowing you to look at your router and see the multicast joins and traffic flowing. Don't forget the -T for setting time to live numbers.
This data as pretty specific, and you can show your complainers/customers how well the network handles a solid gig of traffic, when their sluggish program using 50k of bandwidth is not a "slow network issue".
Pair this data with some colorful graphs showing how little they actually use your network and most people accept that the network isn't the problem.
Of course with your excellent customer service skills you have worded this all super politely and let them "arrive at their own conclusion" that there must be something else wrong than the network. Unless of course your gig interface is only able to push 2Mb/s of traffic, then you need to suck it up and figure out why your network is slow!
Thursday, September 16, 2010
Snort Struggles
Well I apologize for my lack of posting, I have been thrust into the role of "supreme network commander" (my own working title) but all that really means is that I am the lone network engineer until one or more people get hired. Sooo, needless to say I have been overly busy and study/blogging time has been severely hampered. anyway, On to snort:
Well snort is a headache of headaches. Attempting to install snort on a mac and get it to send to aanval is proving to be quite the feet. Mac tutorials for snort are … well lets just say none of them use even remotely the same method, and none seem to work. All instructing me to modify files that are not on the system anywhere or change lines of code that do not appear in the listed files… frustration ensues. On the bright side I am working with the server team to get me a Linux server, and the Linux instructions appear, at least right now, to be much much simpler. Although the fact that we already have a handful of mac minis to locate throughout the network as snort sensors means I will eventually have to get snort running on them… Oh well, a problem I can push off a few days. School starts Monday so other priorities trump IDS for the week.
To wet your appetite, I am about to implement some Cisco IP SLA monitoring, I can blog about how that goes, should be good times. Especially since HP procurve does not have any features that can do this, so I have to add cisco routers behind my procurves to support the needed feature...
Well snort is a headache of headaches. Attempting to install snort on a mac and get it to send to aanval is proving to be quite the feet. Mac tutorials for snort are … well lets just say none of them use even remotely the same method, and none seem to work. All instructing me to modify files that are not on the system anywhere or change lines of code that do not appear in the listed files… frustration ensues. On the bright side I am working with the server team to get me a Linux server, and the Linux instructions appear, at least right now, to be much much simpler. Although the fact that we already have a handful of mac minis to locate throughout the network as snort sensors means I will eventually have to get snort running on them… Oh well, a problem I can push off a few days. School starts Monday so other priorities trump IDS for the week.
To wet your appetite, I am about to implement some Cisco IP SLA monitoring, I can blog about how that goes, should be good times. Especially since HP procurve does not have any features that can do this, so I have to add cisco routers behind my procurves to support the needed feature...
Wednesday, September 8, 2010
OSPF packet types
There are 5 types of OSPF packets. They are encapsulated in an IP packet, but are NOT TCP or UDP. The packets are IP protocol type 89 which is specific to OSPF. This is a reliable transport protocol.
Type 1 is a Hello Packet. This is how routers discover neighbors and build adjacencys between them.
The second type is a DBD or Database Description. This is used to synch the databases between two routers.
Third is an LSR, Link State Request. LSRs are used to request specific link-state records from a different router.
Fourth is an LSU or Link State Update. This is the link state records that were just requested in the LSR.
Fifth is the LSAck or Link State Acknowledgement. This is the reliable portion of the protocol, a LSAck is sent in response (acknowledged) to any of the above packets to let the neighbor know that the packet has been received.
Type 1 is a Hello Packet. This is how routers discover neighbors and build adjacencys between them.
The second type is a DBD or Database Description. This is used to synch the databases between two routers.
Third is an LSR, Link State Request. LSRs are used to request specific link-state records from a different router.
Fourth is an LSU or Link State Update. This is the link state records that were just requested in the LSR.
Fifth is the LSAck or Link State Acknowledgement. This is the reliable portion of the protocol, a LSAck is sent in response (acknowledged) to any of the above packets to let the neighbor know that the packet has been received.
Subscribe to:
Posts (Atom)
