Systems Research Group

Cloud Benchmarking

How can applications be deployed on the cloud to achieve maximum performance? This question is challenging to address with the availability of a wide variety of cloud Virtual Machines (VMs) with different performance capabilities. This research project addresses the above question by proposing a six-step benchmarking methodology in which a user provides a set of weights that indicate how important memory, local communication, computation and storage related operations are to an application. The user can either provide a set of four abstract weights or eight fine grain weights based on the knowledge of the application. The weights along with benchmarking data collected from the cloud are used to generate a set of two rankings – one based only on the performance of the VMs and the other takes both performance and costs into account. Our rankings have been validated on three case study applications, which highlight that maximum performance can be achieved by the three top-ranked VMs, and maximum performance in a cost-effective manner is achieved by at least one of the top three ranked VMs produced by the methodology. Please refer to the following journal paper in the IEEE Transactions on Cloud Computing, and conference paper in IEEE CloudCom.