Zheng Qing, Chen Haopeng, Wang Yaguang (Intel), Duan Jiangang (Intel), Huang Zhiteng (Intel)
Introduction
According to IDC, the total amount of digital data worldwide is estimated to pass 2.7 zettabytes in the year of 2012, demonstrating a mammoth 48% increase compared to the figure collected in 2011. Although today it is not unusual for an individual to possess a hard disk capable of holding terabytes of data, there is still enormous amount of data lies in data centers of various industry companies.
For example, Facebook had more than 15 petabytes of data in its data warehouse in 2010, with 60 terabytes of uncompressed data being added every day. Similar data scale had also been observed by Microsoft, which reported in 2011 that its Azure storage service was at that time responsible for 70 petabytes of data in production, and was in the process of provisioning a few more hundreds petabytes of raw storage.
Besides the immense data amount, the majority of the digital universe is unstructured data such as documents, photos and videos, as a result of recent explosion of social networking, online applications and mobile phones.
Object Storage Comes to Rescue
The tremendous amount of data in conjunction with the unprecedented growth rate has posed significant challenges on the storage infrastructures for unstructured data. Today, many companies are moving from traditional SAN and NAS based systems to object storage services such as Amazon S3, Google Cloud Storage, or Rackspace Cloud Files.
Object storage services provide RESTful interfaces for one to store and access files in a way that is similar to albeit simpler than regular file systems. In addition, these services are often characterized by what is lacking in conventional technologies: scalability, cost-effectiveness, easy-of-use, as well as high-performance and availability.
Work in Progress
With object storage services becoming increasingly accepted as replacements for traditional file or block systems, it is important to effectively measure the performance of these services. For example, cloud builders may use benchmark tools to compare different software realizations or hardware configurations. Moreover, these tools can also help them conduct system refactoring or algorithm tuning in order to achieve optimal performance. However, to the best of our knowledge, there has been little, if not none, reported study focusing on benchmark technologies for object storage services.
To help address this problem, we are investigating related methodologies with the goal of creating a benchmark tool for characterizing object storage services, thus allowing people to evaluate various implementations or configurations of object storage service.
Challenges
Unfortunately, performance benchmarking against object storage services is not as straightforward as it may appear to be due to several reasons. To start with, there is currently no widely-adopted standard on the interfaces of object storage services, making it challenging for a benchmark tool to work with different service implementations. Second, a benchmark tool should be able to simulate diverse usage patterns, which can also be challenging as it requires a judicious abstraction of real-world workloads. Finally, it is challenging to design a tool that is simple, practical, and extensible simultaneously.
COSBench (Cloud Object Storage Benchmark)
COSBench is benchmark tool that we are designing and implementing for benchmarking cloud object storage services, and is also our current answer to the challenges listed above.
- COSBench v1.0
- COSBench v2.0
Experiment Results
We have used COSBench to evaluate the read performance of a 6-node Swift storage cluster we have set up in Intel lab. Swift is an open source cloud object storage service implementation which exposes similar interface to Amazon S3. Our swift cluster is composed of 1 proxy node and 5 storage nodes. We ran 4 COSBench driver instances on 4 different client nodes with each driver stressing the cluster from 4 to 512 workers using 100% read operations on objects 64 KB in size for 300 seconds.
The hardware configurations are listed as follows. The results are illustrated above in terms of the average response time and the throughput.
System Configuration
Read Performance
As can be seen from the above figure, the read performance increases as the number of workers per client increases until the throughput gets saturated at 64 workers per client and reaches its peak at 128 with 5644 read operations performed per second and each taking 90ms to complete. More workers beyond that only lineally prolong the response time but do not further increase the throughput.