Scalability of 64-bit Terminal Server Platforms

Posted by Benny Tritsch on April 24, 2007 – updated on July 17, 2007

[Introduction] [Test Environment] [Tools and Scripts] [Methodology] [Results] [Conclusions]
[Appendix 1: Performance Counters Used] [Appendix 2: Step-by-Step Description]

Conclusion

The initial purpose of the tests was to investigate the behavior of 64-bit server hardware platforms during massive utilization through terminal services user sessions, both under the 32-bit and the 64-bit edition of Windows Server 2003. From the tests, we have drawn the following conclusions.

Memory requirements increase with 64-bit operating systems. When using the 64-bit version of terminal server, overall memory consumption can be up to two times higher than the same workload running on a 32-bit version of Microsoft Windows Server 2003. Data structures are larger mainly because pointers are two times larger, and the System File Cache consumes more memory. On a 4GB system, x64 Windows underperformed x86 Windows.

PTEs are not the sole memory-related limitation of Windows. One of the session bottlenecks in Terminal Services is availability of page table entries (PTEs). PTEs map paged pool (areas of kernel-mode memory that can be paged to disk) into its address space. When the operating system runs out of PTEs, it cannot create any more processes and will actually begin to become unstable when they run low. Because 32-bit Windows reserves only 900MB of memory for PTEs and a terminal server is by its nature process-heavy, PTEs are a scarce commodity. 64-bit Windows removes this bottleneck by allocating 128GB of memory for PTEs.

That said, even with 16 gigabyte of memory installed we saw no significant difference in the number of sessions a terminal server could create. It is possible that the 64-bit server was able to create more usable sessions since the last 20-30% of sessions created on the 32-bit platform may not even load the shell due to the lack of resources. Unfortunately, this is speculation, since Windows lacks any performance counters for testing user response time. Even without that additional data, however, this study shows that limits on PTEs are only one limiting factor on the number of sessions that can be supported.

The following image displays the scalability pattern as we were able to observe it during our tests. The two lines in blue and red show the best case and the worst case for the 32-bit and 64-bit technologies respectively; the real behavior is somewhere between these limits:

• On a 4GB system, x64 Windows underperformed x86 Windows.
• On a 8GB system, the 32-bit and 64-bit Windows editions are able to host roughly the same number of users. Depending on the application set, the 32-bit system may show first indicators of saturation.
• x86 Windows on hardware platforms with old chipsets may show a degration in performance with physical memory beyond 8GB. Removing memory from server hardware with more than 8GB may solve such issues. Server hardware with new chipsets does not show this behavior, even if the additional memory does not increase performance for x86 Windows.
• In many cases x86 Windows is not able to take advantage of physical memory beyond 8 to 12GB.
• We see systems with physical memory between 8GB and 16GB sitting in some sort of "twilight zone". x86 Windows may not be able to use this memory sufficiently and x64 Windows may not be able to outperform x86 Windows with the same amount of memory.
• With physical memory beyond 16GB x64 Windows starts performing significantly better than x86 Windows.

The ROI on 64-bit operating systems may not justify the cost. As noted above, we found no big difference in the number of useful sessions a 32-bit server could support versus a 64-bit server, even with 16GB of RAM installed. (Although the 64-bit server with 16GB of RAM opened more sessions than the 32-bit server — 341 to 270 — it was unable to open Microsoft Word, Adobe Acrobat, and Notepad in about the last hundred sessions.) In this case, there is no benefit to the 64-bit operating system since it performs not significantly better than the 32-bit on a 16GB server.

As a result, we typically recommend the usage of the 64-bit edition of Terminal Servers based on Windows Server 2003 only if the available amount of physical memory is beyond 16GB.

NOTE: I want to thank Christa Anderson, Program Manager for the Terminal Server Product Team at Microsoft, for reviewing this whitepaper and for many fruitful discussions about 64-bit Terminal Server scalability patterns.

Next