Cisco Expressway Bug CSCuo83458

In a recent Cisco Expressway deployment I ran into a bug that is worth mentioning to the community.

I usually recommend naming the Expressway-E server with the enterprise’s standard naming convention. Then for the _collab-edge SRV record, create a generic name on the public Internet to point to the Expressway-E server. Normally this is not a problem, however the bug CSCuo83458 complicates this.

The official conditions of the bug is “The Expressway-E server is configured with a hostname and/or domain name which does not exactly match what is present in the _collab-edge SRV record.” The symptom is “Clients unable to register via Mobile and Remote access via XMPP or SIP. Jabber clients may show an error message of ‘Unable to connect to server’.”

What is comes down to is that the Expressway-E name, internal DNS records, and the _collab-edge SRV must be exactly the same. If they are not, Jabber will not be able to connect.

There is no fix for this bug as of this writing. The workaround is to name the Expressway-E the same as the SRV record. Until this is fixed, my recommendation to my clients is to name the Expressway-E the generic name on the public Internet. Then once this bug is fixed, the hostname of the Expressway-E can be changed.

Bug ID:

Cisco CLI Alias Tricks

I have used the alias command on both the Cisco IOS and Nexus platforms to shorten long commands. For example, I use the alias ‘sisi’ for the command “show interface status | include”.

NX-OS deprecated many of the legacy commands that IOS still contains. One of these deprecated commands is the “write terminal” command. I have worked on Cisco routers for a while and originally learned to use ‘wri’ (I use three letters to avoid mistakes between ‘show’ and ‘shutdown’) to save my configurations instead of “copy run start”. On the Nexus platform the ‘write terminal’ command is one of those deprecated commands. However, with an alias you can put this legacy command back in the CLI. In fact, this is one of the first commands I put in any Nexus that I work on. Use the alias ‘cli alias name wri copy run start’.

NX-OS allows you to issue show or ping commands while you are in configuration mode. This is extremely help when you need to verify if that the command you just issued fixed the problem. On the IOS platforms you have to use the ‘do’ command in order to execute any exec level commands while in configuration mode. It’s a little annoying that you have to use ‘do’, IMHO. If you create the IOS alias ‘alias configure sho do show’, you will be able to issue show commands within configuration mode. Of course you can substitute the ‘sho’ in that command with whatever your preference of the three variations (sh, sho, show) to issue show commands.

You can do this with more of the exec level commands. Here are a few of the IOS alias that I use:

alias configure sho do show
alias configure ping do ping
alias interface sho do show
alias interface ping do ping

Cable color coding

I like things neat and orderly. Everything should have it’s place. I like standards and sticking to them.

My preferred cable coloring method is blue for general data. Yellow would be for important or critical systems. Cross over cables or an unsecured network area, such as the Internet edge, would use red. Management connections would be green. This is great for helping to identify what the cables function is or assist in tracing a cable through a bundle of wires. When the cables are first installed in a brand new installation, it looks good.

Everything is neat and orderly. How long does that last? A week? A month? Eventually someone who doesn’t care about how the cabling looks or where you connect that cable into the network will come in and disrupt all order and organization. Next thing you know, your cabling turns into something like this.bad_cabling

In a recent project we had to build two brand new Data Centers in a very short time. We didn’t want to have to worry about having enough of the correct color cables at the correct lengths. So we decided to just go with black cables. The racks/cabinets are black, the patch panels are black, the cable management is black, everything except the equipment was black. Using black cables would help to hide any poor cabling that was done. When we were done, it looked good, neat, and clean. However we didn’t have any of our coloring cues. I got to thinking about how we could use black cables and still have those coloring cues.

Electrical tape! It comes in a variety of colors. Just cut off a small piece and wrap around each end of the cable near your label. Yes you wouldn’t have the visual color as you traced a cable, but it would make you double check before you unplug that blue tagged cable you thought was going into the DMZ and should be red. This could even open up more colors than were possible before.

This method has great benefits. You keep the color coding standard, all of the same colored patch cables look neat and clean, the black cables in black rack/cabinets and cable management look uniform and clean, and you don’t have to worry about keeping on hand so many different colors and lengths of cables. It’s like some kind of intergalactic hyper-hearse, correct?

If you have a color coding standard, keep it posted somewhere in the DC or network closet with the colored tape so everyone can easily reference it.

My parting comment is not to move any cabling on someone who is OCD about it. It is very annoying. That means you Terry.


Disclaimer: I can’t post pictures of client’s environments, so I borrowed pictures I found online. Below are those credits.
Neat cabling
Bad cabling
Colored electrical tape

Labeling patch cables, patch panels, and wall jacks

Many network administrators are worried about the colors of the patch cables and assigning them to a certain function. For example, red patch cables could mean crossover or outside the firewall, where blue are servers or desktops, or purple connect to APs, etc. However, the actual labeling of cables, and wall jacks, get severely overlooked. Have you ever unplugged a cable from a switch or patch panel to test something and then when you went to plug it back in you couldn’t remember where it was supposed to go? This article is my recommendation for labeling cables.

Patch Cable

A patch cable typically gets connected to a patch panel on one end and a device on the other. When the cable is labeled, the label needs to clearly indicate what it is connected to on both ends. Let’s say your patch panel is in rack 4.01 (row 4, rack 1) and the top of the patch panel (for multiple rack unit patch panels) is on U34. Then you are going to connect the cable to port 15 on that patch panel. Your description for the patch panel port should be R4.01/U34/P15. Now let’s say you are connecting it to a chassis based switch named Access1 on blade 6, port 29. Your description of the switchport connection should be A1/S6/P29, where A1 is the abbreviation for the switch, S is for slot, and P is for port. Both the description of the patch panel port and the switchport should be on each label at both ends of the cable. When you disconnect the cable on either side, you know exactly where to connect it back from the label and the label tells you where the other end goes. Labels be identical on both ends of every patch cable. You type it once and print it twice.

Patch Panel

Patch panels should be label with what is on the other end. In a Data Center where you have infrastructure cabling going from one rack to another, the patch panels should be labeled with the rack and U of the other patch panel. If it is the same port for port, then you don’t have to label the individual ports. However, if one patch panel is split between two patch panels on the other end, then the ports on all three patch panels should be labeled. When it is clearly defined, then it avoids confusion and mistakes.

When the patch panel is used for drops to user workstations, then each patch panel port should be labeled with the unique ID of the drop on the floor. This should be done by your cabling vendor and part of their standard scope of work. In addition, your cabling vendor should provide you with a digital copy of your floor plan and the unique IDs on the drawing where they are located on the floor plan. This may cost you a little more for them to do this, but it’s worth it’s weight in virtual gold when you are trying to locate where that device is in the building.

Wall Jacks

The drop at the wall jacket, or biscuit, should be labeled too. This label should include the drop ID from the floor plan and the patch panel ID. The patch panel ID should be in the format of IDF identifier, rack, patch panel, and port. For example, let’s use IDF 02, rack 03, patch panel 34, and port 48. The label should look something like I02/R03/PP34/P48. That’s a lot to put on a label going into a small space, so if you have a standard format you can shorten it. A shorter version could be 02/03/34/48, or, non-zero filled, 2/3/34/48. I would encourage you to not drop the IDF or rack identifiers, because while you are told they will never expand or grow the site, it will happen. Develop a standard, stick with it, and be consistent.

For the floor plan drop ID, you should use something that will help you locate the drop on the floor. Something that includes room numbers is always good. Avoid labels that could easily change, like MARK-1 for Mark’s office. When you have both the drop ID and the patch panel port labeled on the jack, it will help reduce time to locate and troubleshoot workstation connectivity issues.
Labeling is only as good as you make it…keep up with it. Make a standard for your labeling, document that standard, and stick to it.

Oh no, not Brocade!

Brocade was the last presenter at Networking Field Day 9 this year. When I saw them on the list of presenters I was not thrilled. I will go into more detail later why I was not interested in Brocade, but they did a great job with their presentations and look like they are moving in the right direction.

SDN with Brocade Vyatta

At NFD9 Brocade talked about their Vyatta Controller. While they have their secret sauce in the applications for the controller, and keep that private, they do contribute back to the ODL (Open Day Light) community and their controller is 100% ODL. You don’t have to use Brocade gear with their controller and they will support the controller communication with the device it is programming. While I work for a partner of a Brocade competitor, I still can’t get access to a VM/NFR/trial of that manufacturer’s SDN solution. Brocade gives away their controller and a license to manage up to 5 nodes for 1 year for free. On top of that they give you 60 days of support and you can also download the virtual Vyatta Router to use with their controller. This is great if you want to play with SDN and a step in the right direction.

Programability of the Network

Jon Hudson didn’t go over any specific product, but talked about the ideas of SDN and programming the network. We have all heard the FUD that eventually network jobs will disappear and networking will be taken over by developers. Jon made this statement in his talk, “Most network engineers know how to code, we went to school, and we chose not to be programmers. That doesn’t mean I don’t love coding on the weekend.” I believe the reciprocal is true also in that most app developers chose to be app developers and not network engineers or administrators. We will eventually have controllers in our networks helping us automate those repeatable events. And the controllers will bring us new features and functionality that we have never really had, like steering traffic in our network. I recommend you go and watch Jon’s presentation.

Volumetric Traffic Management

Brocade’s VTM has some good use cases for an SDN application. The VTM re-routes large traffic flows by using sFlow exports to detect the elephant flows and OpenFlow 1.3 to re-route the traffic. VTM is currently supported with the MLX and ICX platforms with the VDX platform on the roadmap. A good use case for this SDN application would be if your backup traffic transits your primary network and security wants your primary traffic flows to go through an IPS. You could then use VTM to re-route your backup traffic around the IPS while the remainder of your traffic goes through the IPS. Santosh also mentioned that VTM can be used to help mitigate some types of large traffic attacks in that VTM can also program the switches to drop traffic altogether.

After the Camera Goes Off

In my previous job the decision was made to go with Brocade for the IPv4 network infrastructure in a new green field Data Center. We had two MLX switches as the core, two DCX switches with FCoE blades for new servers, and a number of FGS switches for legacy servers and appliances. It was a challenge getting all of those Brocade devices working together, even with Brocade services helping. I came away from that experience with a VERY bad taste for Brocade networking equipment and I have enjoyed ripping any of it out of client networks since then. So when I heard that Brocade was one of the presenters at NFD9 I was not looking forward to hearing from them. I decided that I needed to go in with an open mind and hear what they have to say. What they presented was very intriguing and piqued my interest. After the camera was off we were able to talk with Michael Bushong who is VP of Data Center R/S and was recently hired in December 2014. There were other delegates that had issues with Brocade network gear. Michael had stated that he was aware of issues and he had already started making changes internally. I have seen, from Twitter, Brocade hire a number of smart R/S engineers this year.

Brocade has a good opportunity in becoming a serious player in the IP network in the Data Center. In my opinion they are doing things right from getting their Vyatta controller into engineer’s hands to hiring great people on the R/S team. I look forward to seeing them at the next Networking Field Day.

[Updated: Made correction Lisa Caywood pointed out]

Disclaimer: Brocade was a sponsor of Networking Field Day 9. They indirectly paid for my travel and expenses. They may have also given some sort of swag to the delegates. At no time did they ask for, or were promised, any type of consideration in the writing of this article. The opinions and any inaccuracies are mine alone.

Networking Field Day 9

We are about two weeks away from Networking Field Day 9 this year and I am getting really excited. This will be my first NFD to attend in person and I am very honored to be a part of it.

There are great vendors at NFD this year. I have worked with Cisco, Brocade, and Solarwinds before and look forward to hear what they have to say. For CloudGenix, Cumulus, NEC, NetBeez, Pluribus, and VeloCloud I have only heard about them. I will be doing some homework on what they do before I arrive.

There are thirteen of us delegates this year and I am humbled to be included in the company of such smart people. I hope that hanging out with these people for a few days that some of it will rub off on me.

Thank you Stephen, Tom, and GestaltIT!