So, today’s article will be a short one, but a useful one nonetheless. Here’s the scenario….You have 3 different sets of workloads on your blades that require 3 different levels of bandwidth. Because of this, you put them in different chassis’ to accommodate. Some of these chassis require 20G, some require 40G, and yet some require 80G. Just because they have varying bandwidth requirements should not mean that you cannot move a workload from say the 20G chassis to the 80G chassis if that happens to be where your excess server capacity lies at the moment. UCS is totally flexible with any bandwidth requirement you have (you might call it a ‘FlexNetwork’, but I won’t J). Unlike competing blade solutions available, UCS can deliver this varying bandwidth functionality while maintaining all the servers under a single UI in a single domain of management. If you are a user of HP Virtual Connect Enterprise Manager, this would be analogues to having all of your blades into a single “Domain Group”, but still have varying bandwidth requirements. But why stop at blades? Why not be able to manage “server” objects generically and allow rack and blade servers to be pooled together? We’ve got ya covered there too…
UCS has a feature called the “Chassis Discovery Policy” that can be used to handle the varying bandwidth requirements. Although the initials are CDP, this is not your father’s CDP (Cisco Discovery Protocol) J. The purpose of the policy is to tell UCS Manager the MINIMUM number of IO Module (IOM) to Fabric Interconnect (FI) links that must be present in order for the chassis to be properly discovered. Go re-read that last sentence – it’s important. This policy can be found on the Global Policies screen when selecting Equipment on the Tree-View of the Equipment tab as seen below. While it may seem straight forward, it has some caveats, so let me answer the most common questions first.
Q1. From what I can tell, there is only one policy for all my chassis. Does this mean that all my chassis have to have the same number of links?
A1. No. That would make my opening paragraph a lie and I don’t (knowingly) lie on my own blog! J Remember, this is a discovery policy only, not a link-management policy. If the chassis is already discovered, this policy has no effect. If you are trying to reduce the number of IOM->FI cables, this setting has no effect.
Q2. Why is there no 3-link setting?
A2. We don’t support 3 links as an initial setting because there are 8 blade slots in the chassis and the connections for those 8 blades does not divide evenly into 3 ports (in a cable failure scenario of an existing and properly configured chassis, we do support 3 links).
Q3.Should I just set the policy to match the number of chassis links I have cabled?
A3. Not necessarily. Read on…
Q4. Do I need change the policy in order to add or remove IOM-FI cables?
A4. No. All you need to do in that situation is acknowledge the chassis.
The policy values above of 1-link, 2-link, and 4-link refer to number of links between a single IO module (IOM) and the respective Fabric Interconnect (FI). In other words, it’s the number of links for a single side of a chassis, not the total number of links from an entire chassis.
Now, because this is my blog and I don’t technically speak for Cisco here, I’m free to give my own opinion, so I will. What I recommend to my customers is to leave this policy at its default value of 1-link – never change it. With this setting, if a chassis with more than one link is discovered, UCS Manager will find it and add it to inventory (along with the hardware it contains). However, the chassis will be in a non-optimal state because not all chassis links are active yet. In turn, UCSM will return some errors such as “FEX not configured”, “fabric-unsupported-conn”, or “unsupported connectivity”. Like I said, the chassis and its blade are functional at this point, but only using a single link from each IO module. To make the additional links functional, you need to right-click the chassis and choose “Acknowledge Chassis”. Because you are still staging the chassis, acknowledge the warning to continue the process. If the chassis were in production already, you should get a maintenance window to do this. Once complete, all links that are cabled become active automatically and begin carrying traffic.
That’s all there is to it. Let me know if you have any questions. Thanks for reading!
P.S. The current User Guide speaks to the fact that an incorrect setting of the Discovery Policy could lead to a chassis not getting discovered at all. That was the case at one time (and may be the case again some day), but the current behavior in 1.3.1 – 1.4.3 is that a chassis is always discovered, regardless of the policy setting. However, the newly discovered chassis will warn the user that the policy doesn’t match the actual topology. If the behavior changes, I’ll try to remember to come back and update this blog. If not, the documentation will likely get updated. If this really bugs you, read my previous article on how you can contact the UCS Docs team and let them know.
Update (5-8-12): Starting with 2.0 versions of UCSM, the chassis will NOT be discovered if the policy is set for a number of links greater than the physical topology contains.