Practical Performance Models for Complex, Popular Applications

Content Provider	Microsoft Research
Author	Thereska, Eno Doebel, Bjoern Zheng, Alice X. Nobel, Peter
Copyright Year	2010
Abstract	Perhaps surprisingly, no practical performance models exist for popular (and complex) client applications such as Adobes Creative Suite, Microsofts Office and Visual Studio, Mozilla, Halo 3, etc. There is currently no tool that automatically answers program developers, IT administrators and end-users simple what-if questions like what happens to the performance of my favorite application X if I upgrade from Windows Vista to Windows 7?. This paper describes our approach towards constructing practical, versatile performance models to address this problem. The goal is to have these models be useful for application developers to help expand application testing coverage and for IT administrators to assist with understanding the performance consequences of a software, hardware or configuration change. This papers main contributions are in system building and performance modeling. We believe we have built applications that are easier to model because we have proactively instrumented them to export their state and associated metrics. This application-specific monitoring is always on and interesting data is collected from real, "in-the-wild" deployments. The models we are experimenting with are based on statistical techniques. They require no modifications to the OS or applications beyond the above instrumentation, and no explicit a priori model on how an OS or application should behave. We are in the process of learning from models we have constructed for several Microsoft products, including the Office suite, Visual Studio and Media Player. This paper presents preliminary findings from a large user deployment (several hundred thousand user sessions) of these applications that show the coverage and limitations of such models. These findings pushed us to move beyond averages/means and go into some depth into why client application performance has an inherently large variance.
Language	English
Publisher	ACM SIGMETRICS Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page
Publisher Date	2010-06-14
Access Restriction	Open
Rights Holder	Microsoft Corporation
Subject Keyword	Computer systems networking
Content Type	Text
Resource Type	Proceeding