Abstract
Big data system benchmarking enables practitioners and developers to assess the systems’ functionality and performance so that they can make wise decision to choose the proper big data systems, or improve them. As we are witnessing the emergence and evolvement of various benchmarks for big data systems, either in the form of macro-benchmark or micro-benchmark, it is crucial to thoroughly study, analyze, and understand the key techniques and applications of those benchmarks. In this tutorial, we offer a comprehensive presentation of a wide range of state-of-the-art benchmarks with a focus on big data systems. We classify these benchmarks into five categories: Map-Reduce based system benchmarking, SQL-based analytical system benchmarking, NoSQL-based database benchmarking, Big graph system benchmarking, and Multi-model database benchmarking. We discuss the key techniques of each approach, as well as the current practices. We also provide insights on the research challenges and directions for benchmarking different big data systems.
Please see the details as follow: