Over the recent decade, cloud computing has become a popular method when addressing increasing computational demands. However, increased cloud computing usage has also led to increased resource wastage as machines often standby idly during low workload periods. To better utilize cloud computing resources, an interesting proposal involves effectively using cloud computing resources for distributed data processing in an ad-hoc manner during regular and off-peak hours. An existing framework named Adoop realizes this by extending a previous version (1.0.1) of the widely adopted Apache Hadoop framework. Our proposed framework, named AHA, takes inspiration from Adoop to introduce resource availability considering task speculation within the latest version of Apache Hadoop (3.3.0). The resource availability history of each worker node is stored locally and used during MapReduce (MR) workload scheduling. On average, resource availability-aware job speculation is shown to reduce MR workload runtime by up to 10.9% a simulated ad-hoc cloud environment. In addition, a fuzzy rule-based self-tuning solution is also prototyped to alleviate the need for manual configuration regarding resource availability consideration. Our evaluation results indicate that the self-tuning solution can augment the advantage of AHA over Default Hadoop by up to 20.6% for certain MR workloads. Overall, the approach shows potential in addressing this real-world issue as our proposed framework is upper-bounded by Default Hadoop concerning workload execution time in a simulated ad-hoc environment.
Thu 30 SepDisplayed time zone: Eastern Time (US & Canada) change
11:45 - 12:40
|AHA: Adaptive Hadoop in Ad-hoc Cloud Environments|
|Architecture-based Evaluation of Scaling Policies for Cloud Applications|
|Towards Situation-Aware Meta-Optimization of Adaptation Planning Strategies|