ACSOS 2021
Mon 27 September - Fri 1 October 2021 Washington, DC, United States

Over the recent decade, cloud computing has become a popular method when addressing increasing computational demands. However, increased cloud computing usage has also led to increased resource wastage as machines often standby idly during low workload periods. To better utilize cloud computing resources, an interesting proposal involves effectively using cloud computing resources for distributed data processing in an ad-hoc manner during regular and off-peak hours. An existing framework named Adoop realizes this by extending a previous version (1.0.1) of the widely adopted Apache Hadoop framework. Our proposed framework, named AHA, takes inspiration from Adoop to introduce resource availability considering task speculation within the latest version of Apache Hadoop (3.3.0). The resource availability history of each worker node is stored locally and used during MapReduce (MR) workload scheduling. On average, resource availability-aware job speculation is shown to reduce MR workload runtime by up to 10.9% a simulated ad-hoc cloud environment. In addition, a fuzzy rule-based self-tuning solution is also prototyped to alleviate the need for manual configuration regarding resource availability consideration. Our evaluation results indicate that the self-tuning solution can augment the advantage of AHA over Default Hadoop by up to 20.6% for certain MR workloads. Overall, the approach shows potential in addressing this real-world issue as our proposed framework is upper-bounded by Default Hadoop concerning workload execution time in a simulated ad-hoc environment.

Thu 30 Sep

Displayed time zone: Eastern Time (US & Canada) change

11:45 - 12:40
Resource Management in Data Centers and Cloud Computing IIMain Track at AUDITORIUM 1
Chair(s): Partha Pal Raytheon BBN Technologies
AHA: Adaptive Hadoop in Ad-hoc Cloud Environments
Main Track
Ryan Liu University of Waterloo, Canada, Shizhe Lin University of Waterloo, Canada, Ladan Tahvildari University of Waterloo
Architecture-based Evaluation of Scaling Policies for Cloud Applications
Main Track
Floriment Klinaku University of Stuttgart, Alireza Hakamian University of Stuttgart, Steffen Becker University of Stuttgart
Experience report
Towards Situation-Aware Meta-Optimization of Adaptation Planning Strategies
Main Track
Veronika Lesch , Tanja Noack University of Hohenheim, Germany, Johannes Hefter University of Würzburg, Germany, Samuel Kounev University of Würzburg, Germany, Christian Krupitzer University of Hohenheim, Germany