Type of Talk: Experimental Systems Talk

Sp

eaker Name: Kathryn McKinley

Speaker Affiliation: Computer Science

s at UT Austin

Date: Wednesday November 15 2006

Start Time

: 11:00 a.m.

Location: TAY 2.106

Talk Title: What''s Your

Workload? and Why You Should Care

Talk Abstract:
Since benchmarks
drive computer science research and industry product development which on

es we use and how we evaluate them are key questions for the community. De

spite complex runtime tradeoffs due to dynamic compilation and garbage coll

ection in modern systems many evaluations still use methodologies develope

d for C C++ and Fortran. SPEC the dominant purveyor of benchmarks comp

ounded this problem by institutionalizing these methodologies for their Jav

a benchmark suite. This talk recommends benchmarking selection and evaluati

on methodologies and introduces the DaCapo benchmarks a set of open sourc

e client-side Java benchmarks. We demonstrate that the complex interactio

ns of (1) architecture (2) compiler (3) virtual machine (4) memory manag

ement and (5) application require more extensive evaluation than C C++ a

nd Fortran which stress (4) much less and do not require (3). This work t

akes a step towards improving methodologies for choosing and evaluating ben

chmarks to foster innovation in system design and implementation for Java a

nd other managed languages.