What is the Minimum MTU for VMware NSX ?
The VXLAN rfc can be found at:
https://www.rfc-editor.org/rfc/rfc7348.txt
Since we are in the Professional field let’s show it with wireshark
From my esxi host we can run the command
pktcap-uw –capture UplinkSnd –uplink vmnic1 -o /tmp/cap2.pcap
This command will capture all my traffic send from VTEP toward the physical switch and save it in file name cap2 with pcap format.
while running this command i will ping from one guest 192.168.1.1 to other guest 192.168.1.2 to generate some traffic.
With WinSCP we can bring the pcap file from the esxi host to my Windows PC and open it with WireShark.
Open the file with show us something like this:
Wireshark 1
We can see udp traffic from VTEP host 192.168.64.130 to VTEP 192.168.64.131 dest to port 8472 (VXLAN) but where is the VXLAN header ?
For wireshark to display VXLAN traffic we will need to change the decode to VXLAN!!!
Right Click to the frame and chose “Decode As…”
wireshark decode as vxlan
Change the Trnasport to VXLAN
Transport k decode as vxlan
wireshark display VXLAN
Now we can see the VXLAN header
MTU Math Time
MTU Math
Outside MTU for IPv4 without Internal Guest OS dot1q Tagging = 20 + 8 + 8 + 14 + 1500 = 1550 bytes
Outside MTU for IPv4 with Internal Guest OS dot1q Tagging = 20 + 8 + 8 + 14 + 4 + 1500 = 1554 bytes
For IPv6 we will need to add more 20 bytes to Outer IPv4 so total max MTU will be 1574 bytes
IPv4 with VXLAN
Conclusion
When we configure VXLAN in DSwitch keeping the default MTU 1600 will keep you in the safe side!!!
NSX MTU 1600
Roie Ben Haim is a Senior Member of Technical Staff who specializes in Networking and Security at VMware and who is currently focused on implementing solutions, which incorporate VMware’s NSX platform as well as integrating with various Cloud platforms on VMware’s infrastructure. Roie works in VMware’s Consulting (PSO) team whose focus is on the delivery of Networking Virtualization and Security solutions. In this role Roie provides technical leadership in all aspects, including the installation, configuration, and implementation of VMware’s products and services. This is also includes being involved from the inception of these project, through requirements assessment, design and deployment phases and then into production which ensures continuity for VMware’s customers. Roie has over a 15 years of experience working on data center technologies, and providing solutions for global enterprises, which primarily focus on Network and Security. A highly motivated and enthusiastic MSc graduate Roie holds a wide range of industry leading certificates, including his most recent Network Virtualization (VCDX-NV). Cisco CCIE x2 (DC/SEC) and Juniper JNCIE-SP. Roie is not only a strong team member, but is also able to demonstrate his skills and experience working in various fields. As a well known and respected blogger, Roie maintains an impressive blog at: http://routetocloud.com
Roie Ben Haim
roie9876.wordpress.com 
[…] 參考資料：NSX Minimum MTU […]