Experimental comparison of simulation tools for efficient cloud and mobile cloud computing applications

Abstract

Cloud computing provides a convenient and on-demand access to virtually unlimited computing resources. Mobile cloud computing (MCC) is an emerging technology that integrates cloud computing technology with mobile devices. MCC provides access to cloud services for mobile devices. With the growing popularity of cloud computing, researchers in this area need to conduct real experiments in their studies. Setting up and running these experiments in real cloud environments are costly. However, modeling and simulation tools are suitable solutions that often provide good alternatives for emulating cloud computing environments. Several simulation tools have been developed especially for cloud computing. In this paper, we present the most powerful simulation tools in this research area. These include CloudSim, CloudAnalyst, CloudReports, CloudExp, GreenCloud, and iCanCloud. Also, we perform experiments for some of these tools to show their capabilities.

1 Introduction

Nowadays, smart devices have become more popular and serve users in various ways. Using enhanced capabilities, smart devices have proved to be useful, providing tools that are ready anywhere and anytime, to serve a user’s needs regardless of their location. Cloud computing integrates several scientific models and distributes resources over the network, thus, resolving the issue of massive computations being done on only one site. Additionally, cloud computing is based on a “pay-as-you-go” model.

Cloud computing is defined as an internet-based model, which provides on-demand access to a network of computing resources (e.g., networks, servers, storage, applications, and services), with minimal management effort [1]. The integration of cloud computing and mobile computing yields a new technology called mobile cloud computing (MCC). MCC technology provides access to cloud services for mobile devices, providing a significant number of advantages such as saving mobile battery lifetime [2], improving data storage capacity [3], and improving reliability and security by storing data and applications on many servers [2].

Where researchers in cloud computing need to conduct real experiments in their studies, experimentations in cloud environments often come at a high cost quotient, for both setup and implementation. One suitable alternative is the use of models and simulation tools to emulate cloud environments and conduct the necessary experimentation. CloudSim, CloudAnalyst, CloudReports, CloudExp, GreenCloud, and iCanCloud are some of the most powerful simulation tools for cloud computing environments which enable researchers to model multiple complex scenarios.

In the remainder part of this paper, we focused on the abovementioned simulators. In Section 2, we review related works. In Section 3, we present detailed descriptions of the aforementioned simulators and conduct experiments for their efficacy. A comparison between the major cloud computing simulation frameworks is provided in Section 4, with a conclusion following in Section 5.

2 Related work

Recently, several research works have been done in the area of mobile cloud computing, for instance, authors in [4] provided a survey on mobile cloud computing and presented taxonomy of current approaches in MCC based on the critical issues in this area. Other authors [5] described the originality of mobile cloud computing and how it combines both cloud computing and device computing. They also reviewed the characteristics of MCC including some recently published research, in addition to discussing the challenges of working in this area and some proposed solutions. Benkhelifa et al. [6] attempted to provide an efficient green MCC platform by proposing a system that generates a profile to determine the users’ power usage, based on their interaction and component usage. On the basis of evidence in the abovementioned works, it seems reasonable to suggest that many of these simulation tools were designed for analyzing different types of systems, and to study and observe the operations of each system. As suggested by Anu [7], simulation is a way of evaluating a proposed system for various parameters within a specific period of time [7]. The following are brief examples of simulation tools, they will be discussed in detail in the Section 3.

• SimGrid [8] is a simulation tool for grid computing and distributed systems that studies scheduling algorithms for distributed application.

• Buyya et al. [9] designed a simulation tool called CloudSim for cloud computing experiments. It supports modeling and simulation of cloud computing environments, and allows the simulation of multiple data centers.

• TeachCloud [3] is an extension of CloudSim framework and built to be convenient for teaching purposes. This tool adds a GUI along with several new key features.

• iCanCloud [10] is a simulation platform for conducting large-scale experiments in cloud computing. It can determine cost per user by predicting the trade-offs between cost and performance for each application executed on a particular hardware [11].

• Jararweh et al. [12] designed CloudExp tool to overcome certain limitations of the CloudSim tool. Its efficient GUI is built on top of CloudSim. In addition, this simulator is used to evaluate and analyze cloud components such as processing elements, data centers, storage platforms, and networking. It integrates most of the technologies related to cloud computing such as big data management and mobile cloud computing. CloudExp introduced a specialized modeling and simulation environment for mobile cloud computing [12, 13]

• CloudAnalyst [14] is an extension to CloudSim, and used to simulate a large-scale internet application on the cloud.

• CloudReport [15] is a GUI CloudSim-based tool with an easy-to-use interface.

• DARTCSIM [16] was also built based on CloudSim. It hides simulation details and provides friendly GUI for users to conduct their experiments. In addition, it allows the user to alter simulation parameters easily [16].

It is necessary to mention that some of these simulators are open source, giving opportunity to researchers to develop bespoke code extensions.

3 Cloud simulation tools

In this section, we have conducted a detailed study of commonly used cloud computing simulation tools and discussed their main features.

3.1 CloudSim

CloudSim project has found widespread use in both research-based cloud development and industry-based cloud development. It provides a generalized framework to apply experiments, according to specific design issues that can be seamlessly set to the offered generalized framework [9].

With its earlier versions based on GridSim, CloudSim was developed by the GRIDS laboratory of the University of Melbourne to model and simulate several cloud computing infrastructures. In its latter versions and in-depth re-implementation, CloudSim includes a full implementation of its own kernel to rid itself of reliance on GridSim [17].

Conceptually, CloudSim on one side offers classes representing data centers, physical hosts, virtual machines, services to be executed in the data centers, users of cloud services, internal data center networks, and energy consumption of physical hosts and data centers elements. On the other side, CloudSim supports dynamic insertion of simulation elements and provides message-passing application and data center network topology [9, 18].

In addition, CloudSim supports the modeling and simulating of different cloud computing environments, as well as simulation of many data centers. Furthermore, evidence in literature suggests that CloudSim is capable of instantiating 100,000 machines in less than 5 min [11]. It further enables studying on allocating policies and migrations of virtual machines (VMs) rendering its very useful in the research field for many researchers [19–21]. CloudSim’s architecture consists of specific entities that are represented as Java classes, and as such, developers can simulate their experiments by inheriting these classes or directly by changing the available methods or parameters easily. Figure 1 below illustrates the main components of CloudSim, with each described below.

Fig. 1

The main entities in CloudSim

Figure 1 represents the main entities in CloudSim [22]:

• 1- Data centers (DC): the resource provider—includes one or more hosts.

• 2- Host: the physical machine that allocates one or more VMs).

• 3- Virtual machine (VM): machines on which the cloudlet will be executed.

• 4- Cloud Information Service (CIS): responsible for registering all resources of data centers.

• 5- Broker: when a broker has the DC characteristics, it will submit VMs to the specific host in the specific DC and then allocate the cloudlets (tasks) to specific VMs. Finally, the broker will destroy the free VMs after execution of all cloudlets.

• 6- Cloudlets: In CloudSim, cloudlets are all tasks and applications that are executed on VMs.

These entities interact with each other to perform the simulation: Cloud Information Service (CIS), broker, data center, host, virtual machines (VM1, VM2), and cloudlets (C1, C2 ….).

Each simulation in CloudSim follows basic steps in java code from configuration to simulation. These are as follows:

• 1- Setting the number of users for a current simulation.

• 2- Initializing the simulation by instantiating the common variables (current time, trace flag, number of users).

• 3- Creating CIS instance.

• 4- Creating data center instance and then registering it with CIS.

• 5- Creating physical machines (hosts) with their characteristics.

• 6- Creating data center broker instance.

• 7- Creating VMs with their characteristics.

• 8- Submitting VMs to data center broker.

• 9- Creating cloudlets and specifying their characteristics.

• 10- Submitting cloudlets to data center broker.

• 11- Sending a call to start the simulation once there is an event to be executed.

• 12- Sending a call to stop the simulation once there is no event to be executed.

• 13- Printing the results of the simulation.

CIS registers all data center resources before they are used, where upon each data center contains one or more hosts. In addition, each host allocates one or more virtual machines which run the cloudlets, while the broker is responsible for submitting cloudlets to the specific data centers. Power consumption in CloudSim is described by a linear relationship between the power consumption and CPU utilization. As such, CloudSim provides several power models, such as PowerModelSqrt, PowerModelLinear, PowerModelSquare, and PowerModelCubic, all based on the linear relationship between power consumption and CPU utilization [23].

However, despite its said advantages, CloudSim still has its limitations, for instance, (1) the lack of a GUI—CloudSim does not have the ability to produce results in a graphical format; (2) CloudSim only includes a basic network model with a limited workload traffic generator [12]; (3) CloudSim also lacks support modeling for parallel experiments [12]; (4) lack of some essential cloud computing components such as BPM and SLA [12]; (5) analyzing simulation results in CloudSim is difficult, so a developer might need to analyze data using other tools such as Excel.

3.1.1 CloudSim experiment

The following scenario was assumed to test the capability of CloudSim for modeling and simulation. The objective of this scenario is to calculate the power consumption (watt) for a cloud data center. We assumed that the simulation environment consists of one data center with two hosts (physical machines). Each host has one CPU core (with computing capacity 1000 MIPS), 2 GB of RAM memory, and 1 TB of storage.

In this simulation, we have four VMs running on the host, with the following constraints: one CPU core, with 500 MIPS computing capacity, 512 MB RAM memory, and 10 GB of storage. A default policy allocates each VM to the first available host with the maximum number of free processing elements (Pes) based on FCFS. Each host dynamically distributes/shares its capacity of a core among all VMs running concurrently (time-shared VM scheduler).

The environment is composed of ten cloudlets (task units), where upon each instance of cloudlet requires 400,000 million instructions, or about 6 min to complete execution on a host. Figure 2 shows the power consumption results of the simulation produced by CloudSim.

Fig. 2

Power consumption for cloud data center that executes ten cloudlets (tasks)

Figure 3 shows another power consumption results for simulating different configurations. It has only one difference, that is, the number of cloudlets. It is 100 cloudlet units.

Fig. 3

Power consumption for cloud data center that executes 100 cloudlets (tasks)

Total power consumption of cloud data center is defined as the sum of power consumption for all nodes. The default power model in CloudSim is Linear power model as described in (1) [18].

$$ P(u) = k\ *\ {P}_{\max } + \left(1 - k\right)\ *\ {P}_{\max }*\ u $$

(1)

where k is the idle host power consumption, P _max is the maximum power consumed by the host in full utilization mode, and u is the CPU utilization [18]. However, CloudSim provides some power models that generate power consumption values by implementing linear relationship between power consumption and CPU utilization [23].

3.2 CloudAnalyst

CloudAnalyst is developed at the GRIDS laboratory at the University of Melbourne and designed for some specific goals. What is so special about this simulator is its graphical presentation of output, and the repetition of simulations to suit the user’s requirements. As such, CloudAnalyst is beneficial where the main objective is to simulate distributed applications among several data centers and user groups. It inherits the original features of CloudSim framework, extending it to simulate large-scale Internet applications and study their behaviors in cloud environments. Furthermore, it supports the evaluation of social network tools, based on the geographic distribution of users and data centers. It is our opinion that internet application users care more about the time needed to process their requests, regardless of the server location. When a user base (a group of users) requests cloudlets (tasks), a Service Broker entity in CloudAnalyst routes user base traffic to the best data center, according to a server brokerage policy. CloudAnalyst implements three brokerage policies; service proximity policy, best response time policy, and dynamically reconfiguring router [14].

CloudAnalyst domain model consists of the following:

• 1. Internet: an abstraction of the real word internet. Suitable bandwidth and transfer delays are introduced and are configurable for all six regions of globe.

• 2. Region: the world is divided into six different regions, where upon the user bases (UBs) and DCs belong to one or more of any one of these regions.

• 3. User base: it models the group of users who are considered as a single unit in simulation and generate traffic for the simulation, these can be a single user or a large number of users.

• 4. InternetCloudlet: grouping of request form users. In advance tab, requests can be bundled as a single InternetCloudlet, with information such as the size of request execution command and input output files.

• 5. CloudAppServiceBroker: In CloudAnalyst, the Service Broker controls the traffic between DCs and UBs and retains the authority to decide which data center services a request from the user base.

• 6. Data centre controller: most important entity in the CloudAnalyst domain model. It is the front face used by the CloudAnalyst to access the heart of CloudSim toolkit functions.

• 7. VmLoadBalancer: virtual machine load balancer is used in data center controller to select the virtual machine to assign the next cloudlet for processing. There are three different load balancers used—round robin, equally spread current execution load, and throttled load balancer.

• 8. GUI: this is the set of different screens that allow a user to define the simulation parameters and save, load saved configurations, and execute or cancel simulation.

CloudAnalyst includes new powerful features such as an easy-to-use GUI, the ability to separate a simulation from the programming code, quick simulation setup, and an enhanced graphical results display, featuring useful formats such as tables and charts [14]. Installing CloudAnalyst is very simple and time efficient.

The main window of CloudAnalyst lets the user configure the simulation by determining data centers, hosts, VMs, and user base that represent the number of users (e.g., 1000 users), as shown in Fig. 4. The user base sends requests that are executed according to a routing policy in a specific data center.

Fig. 4

CloudAnalyst configuration panel

When the simulation completes, the output panel displays the response time for each user base. The simulator provides a detailed screen with results that include response time for each user, request servicing time by each data center, and a number of all requests serviced by the data center. All of these results are displayed in tables and charts to simplify the analysis as shown in Fig. 5.

Fig. 5

a CloudAnalyst simulation result window; b, c detailed simulation result window

Evidence in research suggests that many authors have indeed used CloudAnalyst to conduct experiments. For instance, the authors in [24] used CloudAnalyst to contrast various load balancing policies. A service broker strategy also is proposed in [25] which routes the user requests to the cost-effective data center.

3.2.1 CloudAnalyst experiment

In this experiment, we assume that there is an internet application that is located in three data centers around the world. The cloud environment consists of six user bases which are geographically distributed and three data centers with 20, 30, and 40 VMs as shown in Fig. 6 below. The user base traffic is routed to the data center according to the service proximity policy that selects the closest data center in terms of the lowest network delay [14]. The duration of this simulation is 1 h.

Fig. 6

User base and data center configurations

After the simulation is completed, CloudAnalyst shows the regions with response to time results. This is shown in Fig. 7.

Fig. 7

A view of geographical response time results

It is also possible to analyze the results by exporting a .pdf file that includes detailed charts and tables. Tables 1 and 2 are illustrative screenshots of a request servicing time for each data center and cost details, respectively.

Table 1 Data center request servicing time

Table 2 Cloud environment cost results

3.3 CloudReports

CloudReport is another tool which is built upon CloudSim’s framework (see Fig. 8 for CloudReport architecture). However, CloudReport enhances many features of CloudSim such as a friendly graphical user interface, running multiple simulations at the same time, and enhanced simulation results. In addition, CloudReport benefits from the use of Application Programmable Interface (API) for creating specific extensions to simulate specific algorithms.

Fig. 8

CloudReport architecture based on CloudSim [15]

When simulation completes, CloudReport results are shown in tables and charts with information about execution time and resource usage costs. It also provides information about VM allocation, energy consumption, and any other user-defined characteristic [15].

3.3.1 CloudReports experiment

To demonstrate the capabilities of CloudReport, in this experiment we assumed that the simulation environment consists of one data center and two hosts (physical machines). Each host has one CPU core (with computing capacity of 1000 MIPS), 2 GB of RAM memory, and 1 TB of storage. In this simulation, we have four VMs running on the host with the following computing capacities 500, 1000, 500, and 1000 MIPS. These VMs have 512 MB of RAM memory and 10 GB of storage. To initiate this experiment in CloudReports, we configure the data center, change the host configuration, and then add user proprieties, as shown in Fig. 9.

Fig. 9

Add/configure data center, hosts, user properties in CloudReports

When the simulation completes, CloudReports generates the results and represents them as HTML reports. These include charts for representing resource utilization, execution time, power consumption, and the overall resource utilization of each data center. Figure 10 is generated using CloudReports and shows the number of cloudlets which have been successfully executed by the current user on the data center.

Fig. 10

Number of successful cloudlets executed in each VM in the current data center

Figure 11 is generated also by CloudReports and shows the total time in seconds for each cloudlet that is executed successfully in the current data center.

Fig. 11

Execution time for each customer’s cloudlet

3.4 CloudExp simulator

CloudExp is a modeling and simulation environment which introduced a specialized mobile cloud computing experimental framework [12, 13]. It is built on top of “CloudExp” framework with several new features. It provides user-friendly GUI to enhance the users’ experience in building their own infrastructure. This simulator allows researchers to study the communication cost between users and clouds. In CloudExp, cloudlets are created using a simple drag and drop. Additionally, CloudExp conducts various mobility scenarios for mobile devices.

In CloudExp, the users need to configure the workplace properties and then decide the number of users, number of cloudlet in that region, the cloudlet type (fixed or mobile), and other properties of cloudlet; these are shown in Fig. 12 When the simulation completes, CloudExp results are saved as Excel sheets to simplify the analysis of results. CloudExp used to simulate different types of configurations with different objectives. The work in [26] presented a multi-agent-based dynamic resource provisioning and monitoring for cloud computing system infrastructure. CloudExp is extended to support the multi-agent approach for resource management. An evaluation study for map-reduce task scheduling algorithms over cloud computing infrastructure is presented in [27]. The CloudExp tool is extend to implement a number of map-reduce tasks over a cloud-based infrastructure. Evaluating a large-scale health system is presented at [28] which aims at supporting global health awareness. CloudExp is used to evaluate large-scale cloudlet deployment in [29]. An integrated framework for large-scale mobile cloud computing environment is presented in [20].

Fig. 12

GUI of CloudExp framework

3.4.1 CloudExp simulator experiment

The goal of the following experiment is to study the user load changing behavior on CloudExp power consumption and delay. We study the power consumption for each mobile user in CloudExp by looking at the impact of load size on the performance of power and delay. The load produced by each user is measured by packet rate per second. By using this simulator, we conducted ten separate experiments for both heavy and light user loads. The following table summarizes the experiment’s configuration in CloudExp (Table 3).

Table 3 Experiment properties

All users move in a closed area, and each of them sends a packet to the cloudlet using Wi-Fi or cellular technology with 2 m/s speed. In each experiment, the cloudlet is incremented from one to ten cloudlets, with the maximum number of users in each cloudlet set to 500.

This experiment shows performance results of packet transmission power for each user in MCC, for both heavy and normal user loads. From Fig. 13, we can observe that the power consumption and delay values for each user in the heavy load are higher than those in the light load. This is an expected outcome since heavy loads such as 400 packets are transmitted in 1 ms, whereas light loads such as 100 packets are transmitted in less than 1 ms. Furthermore, both transmission power and delay can be reduced by increasing the cloudlets. A detailed account of the foregoing is provided in [12, 13]. Finally, in contrast to other simulators, CloudExp seems superior when similar attributes are used, with better precision in experiments.

Fig. 13

a Average transmission power for each mobile user vs. heavy/light user loads. b Average transmission delay for each mobile user vs. heavy/light user loads

3.5 iCanCloud

iCanCloud [30] simulator is used to model and simulate a cloud computing system with a wide range of users. The user continuum ranges from basic users on one end to developers of large distributed applications on the other. iCanCloud can predict trade-offs between the cost and performance of a given set of applications that execute in specific hardware. This simulator is developed on top of [31], which is an open source communication network simulation package for OMNeT++ (modular, extensible, component-based C++ library and framework which are being used for building network simulators), as such, both are required to develop and execute new modules for iCanCloud.

Since cloud computing provides a variety of features, it is our opinion that a majority of users and consumers have differing interests; however, they share the common objective of optimizing trade-offs between cost and performance. This is a hard task that iCanCloud tries to alleviate.

iCanCloud offers flexible, scalable, fast, and easy-to-use environment for users to obtain results quickly, and enables users make swift decisions about the value of the services. Scalability is paramount as it determines the ability of a simulator to simulate large-scale systems without losing performance. This is particularly critical considering that the time required to run a simulation is directly proportional to the size of the infrastructure.

A flexible simulator allows its users to build environments easily by using different components with different levels of detail. Indeed, the hypervisor model allows users to integrate any cloud brokering policy to manage a set of fully customizable VMs, so an iCanCloud user can fully customize different brokering policies. By using iCanCloud, researchers in [32] simulate Multi-Agent System Cloud (MASCloud) for optimizing cost in cloud computing.

iCanCloud provides a set of components which emulate the behavior of real components in the real architecture, for instance, networks, disks, file systems, and memory. These components constitute the core simulation engine, while additional components can be added to its repository, and the substitution of substitution of components.

Along similar lines to authors in [33], we postulate iCanCloud’s design principles as (i) it should be able to conduct large experiments as compared to other simulators discussed in this paper; (ii) it should provide flexible and fully customizable global hypervisor to integrate a cloud brokering policy; (iii) it should have GUI for user convenience, to configure and launch simulations which can scale up to handle a single VM, to large systems that comprise of thousands of machines; (iv) it should reproduce the instance types which are provided by a given cloud infrastructure.

The following are the features of iCanCloud:

• 1. Existing and non-existing cloud architectures can be modeled and simulated on iCanCloud.

• 2. Uni-core/multi-core systems can be quickly simulated by using customizable VMs.

• 3. An easy method to integrate and test existent and new cloud brokering policies is provided by a flexible cloud hypervisor.

• 4. This simulator offers wide range of configurations to storage systems which include a model for remote storage system and local storage systems, for example NFS, parallel file systems, parallel storage systems, and RAID systems.

• 5. iCanCloud offers graphical user interface which enables easy generation and customization of large-scale distributed models. With the help of this graphical user interface, a user can easily manage a repository of pre-configured VMs, cloud system, pre-configured experiments, launch experiments from graphical user interface, and generate graphical reports.

• 6. To increase the functionality of simulator platforms, new components can be added to the repository of iCanCloud.

• 7. iCanCloud offer a POSIX-based [34] API and Message Passing Interface (MPI) library to model and simulate application.

3.6 GreenCloud

Data centres (DC) distributed over a large geographical area require a considerable amount of energy, which impacts the total operational cost for Cloud DC. The GreenCloud [35] simulator is designed to retrieve the details of the energy consumed by DC components, as well as packet level communication patterns between them. It presents the energy-aware studies of cloud computing DC in a realistic setup. GreenCloud is developed on top of NS2 (a packet level network simulator) to extract aggregates and avail information about the energy consumed by computing and communication components of DC. It is an open source tool, however, with a limited graphical user interface using Nam [36]. In addition, it cannot run on window environments.

A huge advantage of GreenCloud rests in its capabilities to measure DC power consumption, DC load, and DC job load balancing. Around 80 % code of this simulator is implemented in C++, while the rest 20 % is in the form of tool command language scripts.

GreenCloud simulator’s source code [37] have the following major classes: (i) TskObject—it defines tasks representing a single workload which is executed in DC; (ii) TskComAgent and TskComSink classes are protocol agent classes that fragment a workload task into IP packets and perform their delivery; (iii) CloudUser is the parent class of all workload task generators and supports task generation; (iv) ExpCloudUser implements generation of workload task objects with exponentially distributed inter-arrival time; (v) CBRCloudUser class implements constant bit rate generation of workload objects; (vi) DcHost, in DC, this class is the main class coordinating execution and workload distribution; (vii) DcHost computing servers are represented by this class, and receive workload task through TxkComSink, scheduling for it to execute locally. It also constantly maintains a list of active tasks; (viii) SwitchEnergyModel monitors energy consumption of network switches according to load and energy saving mode enabled.

4 Comparison between major mobile cloud computing simulation tools

Table 4 summarizes the major cloud simulation models discussed in this research (CloudSim, CloudAnalyst, CloudReports, CloudExp, GreenCloud, and iCanCloud), for different parameters (language, platform, GUI, availability), communication modes, and the support level that the simulator provides.

Table 4 Cloud computing simulator comparison

5 Conclusions

The ever increasing popularity and complexity of computing systems makes simulation software an important choice for designing, configuring, managing, and analyzing the performance of a system. Cloud computing simulators have found use in evaluating trade-offs between cost and performance in pay-as-you-go (PAYG) environments. Our work therefore sought to present a survey of simulation tools suitable for cloud architectures. A variety of tools, including CloudSim, CloudAnalyst, and CloudReport, CloudExp, iCanCloud, and GreenCloud, are discussed. By comparing the different simulators, it is our opinion that iCanCloud is a superior platform as it provides a full GUI support to the user and runs on JVM. In addition, it is specifically designed for cloud simulation.