 |
Windows Terminal Server, Virtualization, Application Delivery, Windows Software Development, Market Analysis, Training Classes and more... |
 |
Navigation |
 |
|||||||||||||||||||
|
|||||||||||||||||||||
|
|
Awards |
|
||
|
||||
The "Big Iron Test"
Posted by Benny Tritsch on November 25, 2004 – updated on January 31, 2005
[Introduction] [Test Setup] [Testing Methodology] [Analysis 1] [Analysis 2] [Analysis 3]
Introduction and Result Overview
Microsoft Windows Terminal Server allows users to run Windows-based applications on a remote Windows 2000 or Windows 2003-based server. This article focuses on large server platforms (8 CPUs, 8 GB RAM) and contains scalability test results for both Windows 2000 Server and Windows Server 2003 Terminal Servers in application server mode.
In a server-based computing environment, all application execution and data processing occurs on the server. Therefore it is useful and desirable for server manufacturers to test the scalability and capacity of their servers to determine how many client sessions a server can typically support under a variety of different scenarios. This test focuses on large server platforms with 8 processors and 8 megabytes of memory.
Please note that the results and analysis contained in this article should not be interpreted in isolation. The applications used in the tests executed in Terminal Server sessions and were only intended to consume a reproducible amount of server resources. The tests assume a rather static quality, with users only logging onto the system, starting three applications with corresponding documents, and subsequently staying inactive for the rest of their session. This helps to create reproducible results, but your results may vary.
The tests referenced in this whitepaper were performed using a specific hardware platform. Consequently the results may be different on any other hardware platform. This article is primarily intended to document a test methodology and some tendencies derived from test results. You may use the same methodology to test different application sets. But this may not really change the behavior of the operating system!! You should rather think about testing different eight-way hardware architectures, this may change the results in a more impressive way.
Why this Whitepaper?
In many enterprise environments large server hardware (> 4 CPUs, > 4GB RAM) is being used for terminal server projects and quite often I was able to observe that they were not scaling very well. In fact, their performance was lousy compared to smaller servers. Many terminal server specialists know this, but there was this Microsoft Whitepaper on Windows Server 2003 Terminal Server Capacity and Scaling telling a different story. Now, how can you tell your enterprise customers "Get rid of that hardware you just bought for a lot of a money and buy new two-way boxes" without having rock-solid test results proving your opinion? Without real facts you may get into trouble with both your enterprise customers and the hardware vendors, because high-end servers also mean high margins. You don't want to argue with HP, IBM, Dell, Fujitsu Siemens Computer and Microsoft about such a topic without a good reason! So, in 2003 I decided to perform the initial "Big Iron Test" together with Fujitsu Siemens Computers, being a major server manufacturer in Europe. I intentionally did not want to use the latest versions of servers and applications, but a rather average environment as seen at customers' sites many times. This was the reason for using Microsoft Word 2000, Acrobat Reader and Notepad for the initial test.
I used some of the results for my terminal server book published by Microsoft Press at the end of 2003. My technical editor at Microsoft was "not amused" about the draft of my scalability chapter. Even worse, the guy who wrote the Microsoft Terminal Server Capacity and Scaling Whitepaper was really mad at me after reviewing my draft. He had a very different opinion and wanted to stop me from publishing my results. So, I sent my test setup and the results to Mark Russinovich (www.sysinternals.com) and David Solomon (www.solsem.com) for review. They said that they cannot find an error in my methodology and in my results. But even they were not able to explain the number of context switches and the degraded performance I observed. But now nobody could stop me from using some of the results and even one of the screenshots for my book.
When I was at Microsoft Tech-Ed 2004 in Amsterdam, I met with Tad Brockway, the Terminal Services Group Program Manager. He thought that it might be a good idea to publish the full methodology and the initial results of the Big Iron Test. So I wrote this whitepaper, also taking into account some of the of experiences I made after the initial Big Iron Test.
Result Overview
The initial purpose of the tests referenced in this article was to investigate the behavior of large server hardware platforms during massive utilization through terminal services user sessions. The tests showed that using main memory above the 4 GB barriers posed no problems. However, a limiting factor was the massive context switching of the processors, which occurred after initializing more than 100 user sessions even if the users didn’t interact with the applications they had launched. This behavior seems to be specific for server platforms with more than four CPUs because it could not be observed with the same consequences on two-way and four-way machines.
The third test was a significant indication that Windows 2000 Server may not be ready for high-end hardware platforms where Terminal Services are involved. Windows Server 2003 changed this substantially. However, due to the results from tests one and two, I still cannot recommend using hardware platforms with more than four physical CPUs in a Terminal Server infrastructure. Consequently I recommend the usage of two-way or four-way platforms with up to 4 gigabytes of RAM and an appropriate load balancing solution for use in terminal server scenarios.
But now take a look at the detailed test setup and the results documented in the following chapters.
