Performance Dynamics of Translation-Based VM Systems

3.2Performance Dynamics of Translation-Based VM Systems

In the co-designed VM approach based on software translation and code caching, the program binary (containing the architected ISA instructions) is also first loaded from disk into main memory, just as in a conventional design. However, the architected ISA instructions must be translated to the implementation ISA instructions before they can be executed. The translated code is held in the code cache for reuse until it is evicted to make room for other blocks of translated code. Any evicted translation must then be re-translated and re-optimized if it becomes active again. As a program executes, the translated implementation ISA instructions distribute themselves in the cache hierarchy in the same way as architected ISA instructions in a conventional system.

To simplify the analysis for co-designed VM systems, especially regarding the effects of translation, we identify four primary scenarios.

1. 
   Disk startup. This scenario occurs for initial program startup or reloading modules / tasks that were swapped out – the binary needs to be loaded from disk for execution. After memory is loaded, execution proceeds according to scenario 2 below. That is, scenario 1 is the same as scenario 2 below, but with a disk load added at the beginning.
   
2. 
   Memory startup. This scenario models major context switches (or program phase changes) – If a context switch is of long duration or there is a major program phase change to code that has never been executed (or has not been executed for a very long time), then the required translated code may not exist in the code cache. However, the architected ISA code is in main memory, and will need to be (re)translated before it can be executed. This translation time is an additional VM startup overhead which has a negative effect on performance.
   
3. 
   Code cache startup / transient. This scenario models the situation that occurs after a short context switch or short duration program phase change. The translated implementation ISA code is still available in the main memory code cache, but not in the other levels of the cache hierarchy. To resume execution after the context switch (or return to the previous program phase), there are cache misses as instructions are fetched from main memory again. However, there are no instruction translations.
   
4. 
   Steady state. This scenario models the situation where all the instructions in the current working set have been translated and placed properly in the cache hierarchy. The processor is running at “full” speed.
   

In contrast, scenario 2 memory startup is a bad case and the one where VM startup overhead is most exposed. The translation from architected ISA code (in memory) into implementation ISA code (in the code cache) is required and causes the biggest negative performance impact for dynamic binary translation when compared with a conventional superscalar design.

As noted earlier, scenario 1 disk startup is similar to scenario 2, with the added disk access delay. The performance effects of loading from disk are the same in both the conventional and VM systems. Moreover, the disk load time, lasting many milliseconds, will be the dominant part of this scenario. The additional startup time caused by translation will be less apparent and the relative slowdown will be much less in scenario 1 than in scenario 2.

Based on the above reasoning, it is clear that performance analysis of VM system dynamics should focus on scenarios 2 and 4, i.e. steady-state performance and memory startup performance where VM-specific translation benefit and overhead are prominent.

The steady state performance is mainly determined by the effectiveness of the DBT translation algorithms and the collaboration between co-designed hardware processor and translated/native software code. Chapter 4 addresses the translation algorithms and Chapter 6 emphasizes the collaboration and integration aspects for VM systems.

For VM-specific performance modeling and translation strategy research, translation-incurred transient behavior during VM startup phase is the key and will be focused on in this chapter. Particularly, we emphasize the memory startup scenario, that is, when we analyze startup performance, we start with a program binary already loaded from disk, but with the caches empty, and then track startup performance as translation and optimization are performed concurrently with execution.

A set of simulations are conducted for the memory startup scenario to compare performance between the reference superscalar processor and two co-designed VM systems that rely on software for DBT. The first staged translation system uses BBT followed by SBT, and the second uses interpretation followed by SBT. All systems are modeled with the x86vm framework and the specific configurations are the same as in Table 5.4 later in Section 5.5. The simulations start with empty caches and run 500-million x86-instruction traces to track performance. The traces are collected from the ten Windows applications in the Winstone2004 Business suite. The total simulation cycles range from 333-million to 923-million cycles for the reference superscalar.

The simulation results are averaged for the traces and graphed in Figure 3.1. IPC performance is normalized with respect to the steady state reference superscalar IPC performance. The horizontal line across the top of the graph shows the VM steady state IPC performance gain (8% for the Windows benchmarks).

The x-axis shows execution time in cycles on logarithmic scale. The y-axis shows the harmonic mean of their aggregate IPC, i.e. the total instructions executed up to that point divided by the total time. As mentioned, at a given point in time, the aggregate IPC reflect the total number of instructions executed (on a linear scale), making it easy to visualize the relative overall performance up to that point in time.




Figure 3.8 VM startup performance compared with a conventional x86 processor

The breakeven point (as used in Chapter 1 for startup overhead) is the time it takes a co-designed VM to execute the same number of instructions as the reference superscalar processor. This is opposed to the point where the instantaneous IPCs are equal, which happens much earlier. The crossover, or breakeven, point occurs later than 100-million cycles for the baseline VM system using staged BBT followed by SBT. And this co-designed VM system barely reaches half the steady state performance gains (4%) before the traces finish.

For the co-designed VM using interpretation followed by SBT, the startup performance is much worse. The hotspot threshold for switching from interpretation to SBT is 25 (as derived using the method described below in Section 3.3). After finishing the 500-million instruction traces, the aggregate performance is only half that of a conventional superscalar processor.

Clearly, software DBT runtime translation overhead affects VM startup performance when compared with a conventional superscalar processor design, especially for a startup periods less than 100-million cycles (or 50 milliseconds in a 2 GHz processor core). At the one-million-cycle point, the baseline VM system has executed only one fourth the instructions of the reference conventional superscalar implementation.