RSS, Chimney & NetDMA

2017-12-08T16:20:56+00:00 March 22nd, 2011|biz|

Hi,

 

I was recently on a call with some Exchange peers & we were fortunate enough to have a guest appearance from Tod Edwards from the Windows Server team at Microsoft. He attended to provide some background & recommendations to making the above technologies work successfully with Windows Server and why they should be enabled where possible. As this information is in the public domain & helps improve everyone’s experience I have reproduced it here.

Background

 

RSS – Receive Side Scaling, this allows a server to spread network interrupt traffic over more than 1 CPU, significant benefits have been seen with this on 1Gb+ network interfaces

TCP Chimney – Allows the offloading of TCP/IP processing to the network card, both Intel and Broadcom have NICs in the market which support this, the benefits with this feature are less pronounced at 1Gb connections but significant at 10Gb connections (think iSCSI storage connections)

Chimney is incompatible with some features in Windows (firewall, etc.), & if they are enabled Chimney will not offload any connection. In W2008 R2, the Windows Firewall is compatible with Chimney, which was a change from previous OS.

TCP Chimney Offload is not the same thing as Checksum Offload, Large Send Offload, etc. Those offloads have been around for since W2003 & are generally problem free.

RSS & Chimney both require the above basic checksum offloads to function, so disabling any of them (in NIC properties) will automatically keep RSS and Chimney from being used.

Compatibility issues

 

As alluded to above, these technologies work really well with 2008r2 out of the box, on older operating systems (2003) you’re probably already familiar with needing to disable these to resolve issues, here are recommendations as to how things should be configured for the various Microsoft Server OSs in the market at this time:

W2003 SP2: RSS, Chimney & NetDMA are known to be problematic. Customers should disable these manually or use KB:948496. If you really need the features, install SNP roll-up (KB:912222) for W2003 and re-enable features.

W2008 RTM: RSS is on by default but Chimney is off. RSS had some issues that customers hit after installing W2008, most were resolved in SP2, & remaining were solved in KB:979614 (which brings RSS/Chimney to W2008 R2 level).

W2008 R2: RSS is on by default & Chimney is in ‘Automatic’ mode, which means the TCP connection being offloaded has to be 10Gb speed, less than 20ms latency between the 2 end points, & transmitted at least 130k. See page 6-7 of ‘W2008 R2 Networking Deployment Guide: Deploying High-Speed Networking Features

Win2008 R2 SP1 does not contain any fixes related to Chimney, and only has 1 fix for RSS that fixes a bug where RSS only uses 1 CPU when you have over 32 processors, KB:977977

 

If you’re seeing issues on modern operating systems which are resolved by disabling the above features ensure you’re systems are using the most recent NIC drivers, filters (teaming, AV, firewall etc..) & BIOS – whilst the market has matured there are still issues resolved with updated vendor drivers. It is important to work through these issues & use the functionality, in the Exchange world customers have achieved higher throughputs & got more value out of their hardware by getting these features turned back on.

 

Rob