UCS Only Runs at 10 GE and Requires Nexus – NOT!

So, every time I hear someone say “UCS only runs at 10 GE speeds” I look around to see if one of my competitors is talking. And they have been somewhat effective in continuing to spread this non-truth. To be fair, Cisco didn’t help the situation by not enabling this feature when UCS first shipped. However, starting with UCS firmware release 1.3.1, you can indeed run at 1 GbE speeds. As far as requiring Nexus, that’s another bit of FUD that has never been true. Are there advantages to running at 10 GE? Absolutely. Is it required for a supported UCS configuration? Absolutely not.

Most times when we hear the request for 1 GbE, it comes from a customer with a lab environment where 10 GE is not available. However, it also comes up from a few customers who just don’t have enough 10 GE ports in production, but they really want to deploy UCS. The steps required to enable 1 GbE are pretty simple and I’ll cover them here. There are a few things you should know before you just dive in and enable it though. Here are some guidelines:

  1. 1 GbE support is only available on the first 8 ports of a Cisco UCS 6120 Fabric Interconnect (FI) and the first 16 ports of a Cisco UCS 6140 FI.
  2. When you insert a 1 GbE transceiver into an FI, you need to select the speed in UCS Manager.
  3. LACP-based port channels are allowed so long as all port members run at the same speed (and meet the normal port channel requirements).
  4. 1 GbE support is only available in ports defined for Uplinks, not Server Ports (not sure why one might want to try, but I see a lot of whacky things).

So if you wanted to do this, you’d have to know which transceivers Cisco supports. Both copper and fiber are supported and the following table provides more details and part numbers:

Source: http://www.cisco.com/en/US/prod/collateral/ps10265/ps10276/data_sheet_c78-524724.html

Enabling 1 GbE Support in Cisco UCS

There’s really not much to enabling the feature. On the Fabric Interconnect in question, expand Fixed Module->Uplink Ethernet Ports and then select Port X (where X is the port you wish to set to 1 GbE).

On the right side of the screen, Click “Show Interface”. This will produce a screen similar to the following where you simply choose the 1GE radio button and you’re all set as long as you have inserted a 1 GbE transceiver into Port X.

Pretty easy stuff, but extremely important when you need it. One last thing I wanted to point out is that when you don’t set the right port speed, you will see an error of “SFP Validation Failed”. Not the error I would have chosen, but at least you now know what it means if you see it.

Let me know if you have any questions.

-Jeff

13 thoughts on “UCS Only Runs at 10 GE and Requires Nexus – NOT!

  1. I can concur that UCS does not require Nexus. We are an Enterasys shop and managed to get it working. That being said, it is safe to say that UCS was either designed with Nexus in mind or that sales and partners are not familiar with bringing UCS up on non-Cisco networks. We definitely had integration issues, much of which stemmed from gear vendor’s using different terminology.

    If I am wrong on my design statement, then maybe marketing should be adjusted a bit. All I ever see in architecture diagrams are Nexus. At the same time, maybe some documentation about integrating with other network brands should be posted.

  2. Well put I have to say.

    Adam I would be surprised if Cisco would ever really do that with so many things like vBlock and VCE coming to the table. I would agree that it was definitely designed with Nexus in mind or at least some sort of Cisco switching (Catalyst or Nexus). Also if you look at the VCE and Cisco’s gear towards the cloud makes even more sense. Anyways, this is a great post and I also just posted something about Cisco UCS. I am pretty much learning about it and blogging about the experience. Thank Jeff!

    http://bit.ly/fsDJmX

  3. Pingback: Technology Short Take #8 - blog.scottlowe.org - The weblog of an IT pro specializing in virtualization, storage, and servers

  4. Have not been able to do this via CLI. Seems you cannot do a conf t?

    int Eth1/19 for example gets me nowhere. Would like to goto interface and just type speed 1000.

    Craig

    • you need to do the UCS CLI – not the NXOS one. process would be:
      UCS-NAME#Scope eth-uplink
      UCS-NAME /eth-uplink # scope fabric a|b
      UCS-NAME /eth-uplink/fabric #create interface x y (where x is the slot and y is the interface #)
      UCS-NAME /eth-uplink/fabric/interface # set speed 1gbps
      UCS-NAME /eth-uplink/fabric/interface # commit buffer

  5. Hi Jeff,

    I am getting “SFP validation Failed” in Cisco UCS manager for configuring the uplink ports.
    Here is the physical connectivity from FI to nortel Switch details:
    1. For the network uplinks, I have connected FC-SFP-4G in first port of both the FI’s 6120 and on the other end, FC cable is connected to Siemons connector in Nortel switch (model- 8010). Nortel siwtch port supports 1GB speed.
    2. Port channeling is done for both the ports at network switch.
    3. I made 1GB in UCS manager for the particular port.

    After doing all changes also, I am getting same error. Please suggest me ….

  6. Pingback: UCS Setting Fabric Ports to 1G | Macro 32 Ramblings

    • Which Fabric Interconnect? If it’s a 6120/6140, you have to use some of the first 8/16 ports (that number is from memory but I think it’s right).

  7. Hi,

    Do you know if maybe a newer firmware supports 1Gb on server ports?
    I am one of the wackos needing it 🙂
    We have a lab setup and a dual wire setup with a Cisco UCS C240 without a fex in the middle….

    • Sure – depending on which generation FI. The 6100 can run 1GbE on ports 1-8/16 and the 6200 (I believe) can do it on any port. Next you’ll need a 1G SFP like a GLC-T (for copper).

Leave a Reply to Craig Cancel reply

Your email address will not be published. Required fields are marked *