Cisco UCS MTU Sizing with VIC

So, in my last article, I discussed Appliance Ports and how to set them up. But there was a hidden gem in there that I felt deserves its own article because it’s just that cool. If you’ve ever setup the MTU on servers because you want to use an iSCSI array, you’ve suffered through how exactly to get the OS to recognize the new MTU size. As I pointed out in my last article, this process this involves a registry hack, ifconfig, esxcfg-vswitch, or setting the MTU manually within the Windows adapter properties. It’s worth the time to investigate because many applications perform better when the conversation doesn’t have to be fragmented into many small chunks.

If you’re using UCS, and if you are reading this you most likely are, and you are using one of the Cisco VIC cards (including Palo/M81KR, VIC-1240, and VIC-1280), you’re in for a real treat. Due to Cisco’s strong integration with the OS/Hypervisor, the VIC driver will inform the installed OS/Hypervisor of the MTU size designated on vNIC itself in the service profile. It’s simple, elegant and makes a lot of sense. Every other vendor is going to opposite direction – they are making a change in the driver which in turn changes the setting on the hardware. The fault here is that the hardware may not always be the same. If the blade server in use today is upgraded to the newer version, the NIC might change (HP’s BL460 G7 has a different NIC vendor than the HP BL460 Gen8 for example). This means that the registry hack you did for the old server will not work on the new server. It also means a huge headache for server admins to know which tweak to make for which OS, server, and NIC combinations. UCS simplifies all of this. The end goal is to alter the MTU size on the NIC – so why not just start with the NIC?

Just for reference, I thought it might be handy to re-publish the ways to verify the correct MTU size for your OS or Hypervisor. The designated MTU can be verified as follows:

Verifying the MTU Size

Windows:

netsh interface ipv4 show subinterfaces

Linux:

ifconfig

ESX:

esxcfg-vswitch –l (that’s an L)

Testing the MTU Size

Windows:

ping –f –l 8000 <storage appliance ip address>

If you get replies, it’s working. If you get “Packet needs to be fragmented but DF set”, windows is trying to tell you that the packet is too large to pass through and you specifically told ping not to fragment the packet.

Linux:

ping –M do –s 8000 <storage appliance ip address>

ESX:

vmkping –s 8000 <storage appliance ip address>

I use 8000 to keep it common between Windows, Linux, and ESX. The largest true packet size is 8972 which sends a 9000 byte packet when you add in IP and ICMP header info of 28K. Some OS’s accept a parameter of 9000 and others max at 8972, but 8000 works on all of them and demonstrates that the MTU is most likely working as you expect it to.

As always, thanks for stopping by and I hope this article provides some value to you.

3 thoughts on “Cisco UCS MTU Sizing with VIC

  1. while testing UCS MTU Jumbo frames with a customer today, we noticed that the MTU size change was successful even before our reboot of the blade. Do you know why UCS still prompt for the reboot of the blade when changing the MTU size on the vnic template?

  2. Hello Eric,
    I just tested again on UCS v2.1, and this worked as expected on VIC1280. You need to apply the change within UCSM for this to take effect. Can you chime in with the UCS version, and if this is M81, 1280, or 1240?

    I was using the mturoute tool, which tests the PMTU.

    Thanks
    Dan Hanson

  3. Eric – By “expected” Dan means that the new MTU did not take effect until he rebooted the server. We’d love to get the above info as requested above to see what your setup was like.

    Thanks

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>