Showing posts with label xobni performance outlook. Show all posts
Showing posts with label xobni performance outlook. Show all posts

Thursday, December 4, 2008

When has Outlook "Started"?

I have recently had the opportunity to work with a small team to define and run some basic performance tests for Xobni. If you've done some serious performance tests before, I probably don't need to tell you what a peculiar beast performance testing can be. The fun part about this project is that we are measuring areas where are customers are having pain points, and building tools to automate the running of those measurements. On the other side of the coin, creating a consistent and controlled environment where performance measurements can be taken without fear that something external is affecting your measurements keeps me up at nights. No, not literally, but environmental control is one of our biggest problems.

One particular measurement which we've been stuggling with for some time is simply known as "Outlook Startup Time". How do you know when Outlook has fully, really, finally, finished starting? Most importantly, when do our users think that Outlook has really "started"? This is an important question for us: if we're going to improve Outlook startup time with Xobni installed, we have to know what that means. Well here are a few ideas for measuring Outlook startup that we've implemented in a tool of ours:
  • When the "Reading Pane" is visible and has text
  • When the Xobni sidebar appears AND the Reading Pane is visible
  • When the Application.Startup event fires in Outlook
  • When Outlook has finished syncing with Exchange
  • When you are able to move to the next mail and have it load within a certain period of time
  • When the CPU usage drops back to a level on-par with usage before you started Outlook
As you can see, we're measuring a lot and trying to see what sticks. What are your thoughts? When is Outlook usable, by your definition?