Bench-Ranking: A First Step Towards Prescriptive Performance Analyses For Big Data Frameworks
Published in 2021 IEEE International Conference on Big Data (Big Data), 2021
Recommended citation: M. Ragab, F. M. Awaysheh and R. Tommasini, "Bench-Ranking: A First Step Towards Prescriptive Performance Analyses For Big Data Frameworks," 2021 IEEE International Conference on Big Data (Big Data), Orlando, FL, USA, 2021, pp. 241-251, doi: 10.1109/BigData52589.2021.9671277. https://ieeexplore.ieee.org/abstract/document/9671277
This paper aims to fill this timely research gap by proposing ranking criteria (called Bench-ranking) that provide prescriptive analytics via ranking functions. In particular, Bench-ranking starts by describing the current state-of-the-art single-dimensional ranking limitations. Next, we discuss the recent benchmarking requirements for sophisticated approaches over multi-dimensional ranking. Finally, we discuss the ranking criteria goodness by reviewing its conformance and coherence metrics. We validate Bench-ranking by conducting an empirical study using large RDF datasets under a relational BD engine, i.e., Apache Spark-SQL. The proposed ranking techniques provide the practitioners with clear insights to make an informed decision, especially with experimental trade-offs for such complex solution space.