Capturing Control Plane Traffic in UCS

So, there is a little known feature in Cisco UCS that allows one to monitor traffic on the control plane without the hassle of actually hooking up an Ethernet analyzer. The control plane is physically the “mgmt0” port on the Fabric Interconnect (FI) and it is used for managing the FIs themselves and for attaching to KVM sessions on the blades. Cisco UCS makes the capture process really easy and in this article I’ll show you how to do it. Remember, this process only captures control plane traffic – not data traffic (UCS has a similar function called Traffic Monitor for that). So, how is this useful? Well, there’s always the chance that something unexpected could happen in UCS Manager as a result of malformed packets entering the Fabric Interconnect’s mgmt0 port. TAC would need to see what the data looks like in order to determine the cause. But the more likely scenario here is that you are using the Cisco UCS XML API and would like to inspect the management traffic being sent to and from the Fabric Interconnect from either UCS Manager or some other external manager controlling UCS. This is an extremely useful tip to aid in the XML API learning process. If you are not familiar with the API of Cisco UCS, it’s an extremely powerful engine provided to customers, system integrators, independent software vendors (ISV). You can find additional information about the API, download it, and learn how to work with it on the following site: Continue reading

Cisco UCS MTU Sizing with VIC

So, in my last article, I discussed Appliance Ports and how to set them up. But there was a hidden gem in there that I felt deserves its own article because it’s just that cool. If you’ve ever setup the MTU on servers because you want to use an iSCSI array, you’ve suffered through how exactly to get the OS to recognize the new MTU size. As I pointed out in my last article, this process this involves a registry hack, ifconfig, esxcfg-vswitch, or setting the MTU manually within the Windows adapter properties. It’s worth the time to investigate because many applications perform better when the conversation doesn’t have to be fragmented into many small chunks. Continue reading

Appliance Ports in Cisco UCS

So, I recently had a customer that wanted to enable “Jumbo Frames” to a UCS server that had the Cisco Virtual Interface Card (VIC) installed in it (also applies to Palo/M81KR, VIC-1240, VIC-1280). You might also know this process as “maximizing the MTU”. In this particular situation, the customer had an iSCSI appliance directly connected to the fabric interconnects (Whiptail in this case, which is not officially supported by Cisco as of this writing, but this process will be the same for any iSCSI appliance – supported or unsupported). It’s not the first time this has come up so I thought I’d write it down so that everyone can benefit (including me when I forget how I did all of this). This article will be helpful if you’re using any NAS storage such as NFS, CIFS/SMB, or iSCSI. Continue reading

UCS Boot-from-SAN Troubleshooting with the Cisco VIC (Part 2)

So, first let me define some terms….the Cisco VIC is also called “Palo” – a codename that sort of stuck (much the chagrin of the marketing team). Palo’s official name is M81KR – now do you see why “Palo” sort of stuck 🙂 ? We have some new VIC cards as well – the VIC-1240 and VIC-1280 and Sean McGee (@mseanmcgee) talks more about the VIC-1280 here. The VIC-1240 is a built-in option on the M3 blades. Now that we settled that, where is Part 1 of this article? Well, my good friend Ryan Hughes (@angryjesters) got the ball rolling on this. He took it upon himself to write an excellent article explaining how to access the obscure-but-useful command called LUNLIST. So if you are looking for Part 1 to this article, I’m not the author of it. I learned some things reading Ryan’s article, which is not all the surprising since I’m rarely with Ryan when I don’t learn something. You should check out his site if you have not seen the article already, but briefly, LUNLIST is a command that shows you what the Cisco VIC HBA can actually “see” on the fabric – much like a typical HBA BIOS would…but way cooler. Continue reading

8 Cool Features You May Not Know About in UCS Manager

So, you’ve probably heard about many of the cool features in our “Capitola” project (aka UCS Manager 2.0) that my colleague and good friend, Sean McGee, blogged about here: . Sean covers many of the new hardware and software features in our 2.0 launch, and it is certainly worth the read. But what you may not know about are some of the small enhancements we make with each release that are handy when you find them and you think “hey, that wasn’t there before…”. I meant to do this article for the “Balboa” release (aka 1.4.x), but just failed to get it done in time. So, I thought now would be a good opportunity to cover both Balboa and Capitola in a single article. Keep in mind that the enhancements we do in each release, whether they be minor or major, are almost completely demand-driven. My team visits customers often and we try to get a better understanding of how they are using UCS and where it falls short in meeting their needs. We provide that feedback to our engineers, and they turn it into products and features. The direct customer interaction also allows us to share our upcoming ideas and concepts with customers before we build them to make sure we are hitting the mark. Continue reading

Schizophrenia and SchizoHPrenia

So, the year was 1996 and it was the first day of the semester in my programming class at Georgia State University (I was one of those who was able to squeeze a 4 year degree into just 10 years of college). I’m sitting there and this kid I have never seen before comes in and sits down beside me with what I thought was an unusual question…”Hey, what makes the most money in the computer field?” At the time, I knew the DB admins were pretty high on the pay scale so I answered with “Oracle guys make a lot”. He thanked me, promptly dropped that class and went to look for an Oracle class. I also remember him asking me how to spell “Oracle” which he still wrote down wrong. I had been in the workplace for a few years by then and I realized that if you’re in a job you hate, it doesn’t matter how much money you make. In fact, the higher your pay at a job you hate, the more imprisoned you feel. It’s a terrible way to live, if you can call it living. Continue reading

Cisco UCS Ports and Protocols




So, a long time back, I had a job as a network administrator for Ungermann-Bass networks (they were re-branding to the more flashy “UB Networks” at the time). We had a firewall in the office that ran on SUN Sparc and no one really knew anything about it other than that it was the DTTM architecture (Don’t Touch This Machine). It was my first exposure to UNIX and to software-based firewalls (not to mention the PBX system that ran on OS/2, but that’s a story for a different time). One thing I really liked about the firewall was that if we ever needed a port opened for a poorly documented application, we would simply run the desired app and the firewall would instantly tell us which ports were attempted. Sadly, not all firewalls are made equal and many times it’s not easy to find the port information you need. While Cisco has done a great job on our UCS documentation (see my previous post on how to provide feedback to the docs team), I thought it might be helpful for everyone if I included all the UCS port information into a single place for reference.
Continue reading

UCS Chassis Discovery Policy

So, today’s article will be a short one, but a useful one nonetheless. Here’s the scenario….You have 3 different sets of workloads on your blades that require 3 different levels of bandwidth. Because of this, you put them in different chassis’ to accommodate. Some of these chassis require 20G, some require 40G, and yet some require 80G. Just because they have varying bandwidth requirements should not mean that you cannot move a workload from say the 20G chassis to the 80G chassis if that happens to be where your excess server capacity lies at the moment. UCS is totally flexible with any bandwidth requirement you have (you might call it a ‘FlexNetwork’, but I won’t J). Unlike competing blade solutions available, UCS can deliver this varying bandwidth functionality while maintaining all the servers under a single UI in a single domain of management. If you are a user of HP Virtual Connect Enterprise Manager, this would be analogues to having all of your blades into a single “Domain Group”, but still have varying bandwidth requirements. But why stop at blades? Why not be able to manage “server” objects generically and allow rack and blade servers to be pooled together? We’ve got ya covered there tooContinue reading