Home

About Network Testing Labs

Contact Network Testing Labs
Independent Reviews of Network Hardware and Software

 

NETWORK TESTING LABS REVIEW

Software Defined Storage -

Acronis Storage, Ceph and OpenStack Swift

 

 



Acronis Storage is up to 6 times faster, more feature-complete, more reliable and easier to
administer than either Ceph or OpenStack Swift. Acronis Storage is also less expensive to use.

by Barry Nance

Executive Summary

Acronis Storage:

  •          Stores and retrieves data up to 6 times faster
  •          Has more useful features
  •          Costs less in both time and money to administer
  •          Is better-designed and more technologically advanced

Impressively, Acronis Storage managed our data better than Ceph or Swift did. Acronis Storage earns the Network Testing Labs World Class Award for best Software Defined Storage tool.


Serious Data. Serious Data Storage

If your server farm is becoming a server jungle and you think “Peta Bytes” instead of “peanut butter” when you see “PB,” you’re ready for Software Defined Storage, or SDS.

SDS is a particularly intelligent way to store and retrieve large amounts of data. You install an SDS product on a group of low-cost, industry-standard computers, link the computers in a cluster with high-speed network connections and then store data in the cluster via file, block or object storage protocols. The cluster automatically manages the data and even backs up the data by replication.

The ideal Software Defined Storage system abstracts the storage and retrieval of large amounts of data on physical devices, virtual devices and data storage services - independent of location or class of storage. You get agility, reliability, high QoS and lower data storage costs. The ideal SDS system gives you data abstraction, instrumentation, programmability, automation, mobility, policy management and orchestration. You can replace or augment traditional storage systems and arrays with SDS.

We evaluated three software-defined storage products in our U.S. Alabama network laboratory to discover which we could recommend to you. They were Acronis Storage 2.0, Ceph 10.2.5 and OpenStack Swift 2.13.0. Our review criteria included performance, usability, features and pricing.

    


Acronis Storage easily surpassed the competition in this review. It performed faster than either Ceph or Swift and it cost less (despite Ceph’s and Swift’s status as “free, open source” software). Acronis Storage was easier to use and administer. Acronis Storage wins the Network Testing Labs World Class Award for best software-defined storage product.




Performance

We benchmarked Acronis Storage, Ceph and OpenStack Swift in a clustered computing environment (See “How We Tested”). We measured data throughput per second and input/output operations per second for six specific activities:

•    Sequential reads
•    Sequential writes
•    Random reads
•    Random writes
•    Putting 128k objects via AWS S3
•    Getting 128k objects via AWS S3

Additionally, in our benchmarks, we used Acronis Storage, Ceph and Swift in three simulated general use cases:

•    Backup/recovery
•    Virtual machine storage
•    Object storage

    

In general, we found that combining network interfaces via bonding (we configured NICs for balance-xor bonding in half of the tests) increased bandwidth by a factor of 1.6 for all three products and thus was a worthwhile step.

On the other hand, asking Acronis Storage, Ceph and Swift to replicate data for the sake of redundancy and reliability slowed storage operations for all three. Only after writing all the replica copies within the cluster do these three products return a status of “Successful Write” to a client. In particular, when we configured these storage products to use erasure coding, we noted that Ceph’s erasure coding is a background process with a very low priority. In tests with high activity levels, Ceph often did not manage to finish encoding the data. We concluded that Ceph’s erasure coding feature is only appropriate for infrequently-accessed “cold data.”







Sequential Reads and Writes – In a series of tests, we used block sizes ranging from 2K to 1Mb to sequentially write and read data. The file sizes varied from 100 GB to 4 TB. Figure 1 shows the averaged results of all the tests.

Impressively, we found Acronis Storage to have as much as 5 to 6 times the throughput of either Ceph or OpenStack Swift. For example – on exactly the same hardware – Acronis Storage transferred 722 MB/sec, Ceph managed only 128 MB/sec and Swift was the slowest with 116 MB/sec in our sequential read tests.


Figure 1. Averaged throughput for Acronis Storage, Ceph and OpenStack Swift in sequential read and sequential write operations.







Random Reads and Writes – In another set of tests, we measured the random read and random write performance of Acronis Storage, Ceph and OpenStack Swift. These random I/O operations used block sizes from 1K to 500K and file sizes from 250 GB to 8 TB.

Figure 2 depicts the averaged results of all the tests. Again, Acronis Storage was more than 5 times faster than the competition.




Figure 2. Averaged throughput for Acronis Storage, Ceph and OpenStack Swift in random read and random write operations.


































S3 Object Storage/Retrieval – In a third set of tests, we used the Amazon S3 protocol to measure the ability of Acronis Storage, Ceph and OpenStack Swift to “Put” and “Get” 128 K objects. The test environment was essentially a private Amazon AWS cloud.

Note that Ceph does not natively incorporate an Amazon S3 interface. Ceph requires that you use the RadosGW add-on software to translate S3 protocol commands into Ceph commands.

Acronis Storage outperformed Ceph and OpenStack Swift by a factor of about 4 in the “Put” tests and by a factor of about 3 in the “Get” tests.

Figure 3 illustrates the averaged test results for the S3 “Put” and “Get” operations.



Figure 3. Averaged throughput for Acronis Storage, Ceph and OpenStack Swift in S3 Put and Get operations.

Acronis Storage was equally agile in terms of handling input/output operations per second (IOPS). Figures 4, 5 and 6 are graphs of the averaged IOPS that we obtained in three other rounds of separate testing (unrelated to each other or to the earlier graphs). 


You can see the obvious advantage that Acronis Storage exhibited. In the sequential read tests, Acronis Storage achieved 94,700 IOPS. Ceph could process only 21,250 IOPS, while Swift’s maximum IOPS was just 19,400.

































Figure 4. Averaged IOPS for Acronis Storage, Ceph and OpenStack Swift in separate sequential read and sequential write tests.


Figure 5. Averaged IOPS for Acronis Storage, Ceph and OpenStack Swift in random read and random write tests.


Figure 6. Averaged IOPS for Acronis Storage, Ceph and OpenStack Swift in S3-protocol Put and S3-protocol Get tests.



When we primarily used the three storage products for backup/recovery, Acronis Storage again showed its speed and agility. Acronis Storage was the faster product in our virtual machine storage tests, and it excelled in the object storage exercises.

Note that Swift does not support virtual machines. You can use Swift to store VM images, but you cannot run VMs from Swift.

Figure 7 is a chart of results for the three separate test groups. Acronis Storage performed much faster than either Ceph or OpenStack Swift.


Figure 7. Acronis Storage, Ceph and OpenStack Swift throughput for three common use cases.




Architecture and Design

Acronis Storage – Acronis Storage has a clean, straightforward and particularly effective design. Data flows to and from Storage Nodes. A Meta Data Server notes each I/O operation, tracks data blocks (and their versions) and is highly available. While clients store or retrieve data on Storage Nodes, the Storage Nodes and Meta Data Server asynchronously manage the data to both optimize fast data access and make extra copies of the data for safekeeping.

Acronis Storage data block management moves frequently-accessed data to faster drives, replicates data for reliability, uses SSD drives for journaling and caching and optimizes data across multiple drives of different sizes. Acronis Storage even reorders data write operations to best use a particular drive’s specific geometry, moves data closer to the client (in terms of network access rates) and monitors latencies to be able to retrieve data more quickly from one replica or another. Acronis Storage employs a PAXOS algorithm to ensure that a problem with one node doesn’t corrupt or stop the flow of data to and from other nodes.




OpenStack Swift – OpenStack Swift transfers data through Proxy Nodes which then distribute data to the Storage Nodes. If you set up multiple OpenStack Swift Proxies, you must purchase and install round robin devices (load balancers) to distribute the workload across the multiple Proxies.

Note that Proxy Nodes are more CPU & I/O intensive than Storage Nodes because they’re busy transferring data. This means that each Swift Proxy Node is a potential bottleneck. Swift I/O can occur no faster than a single Proxy Node’s ability to transport data.

OpenStack Swift processes read requests by passing file data from Storage Nodes through the Proxy and then to the client. Write operations reverse this data flow.





Ceph – To read or write data, Ceph clients contact a Ceph Monitor to obtain the current cluster map. A Ceph storage cluster can operate with a single Monitor, but this introduces a single point of failure. For added reliability and fault tolerance, Ceph supports an odd number of Monitors in a quorum (typically three or five for small to mid-sized clusters). Consensus among various Monitor instances ensures consistent knowledge about the state of the cluster.

A Ceph storage cluster stores data objects in logical dynamic partitions called pools. Pools can be created for particular data types, such as for block devices, object gateways, or simply to separate user groups. The Ceph pool configuration dictates the number of object replicas and the number of placement groups (PGs) in the pool.

Ceph maps objects to placement groups (PGs). PGs are shards or fragments of a logical object pool composed of a group of Ceph OSD daemons that are in a peering relationship.

Ceph Object Storage Daemons (OSDs) use the Controlled Replication Under Scalable Hashing (CRUSH) algorithm for the storage and retrieval of data objects. To read or write data, a Ceph client connects to a logical storage pool in the Ceph cluster. The CRUSH algorithm provides controlled, scalable, and de-clustered placement of data within Ceph and determines how to store and retrieve data by computing data storage locations.
 

Acronis Storage is also smart enough to issue read requests to multiple replicas to eliminate “tail latency.”

Figure 8 is a logical view diagram of Acronis Storage.



Figure 8. Acronis Storage Logical View





Figure 9 shows a logical view diagram of OpenStack Swift.


Figure 9. OpenStack Swift Logical View





Unfortunately, Ceph is prone to heavy back-channel network traffic between nodes, and this congestion can result in performance bottlenecks.

Figure 10 is a logical view diagram of Ceph.


Figure 10. Ceph Logical View


Usability and Features
Acronis Storage is by far the easiest to install, configure and administer. It’s thoughtfully designed to be used for both “cold” (infrequently accessed) and “hot” (frequent, immediate access) data. Installation is easy and simple – just as you’d expect from a commercial software vendor. Monitoring Acronis Storage activity is a breeze. Acronis Storage shows you, at a glance, the physical and logical views of your data. Node health is highlighted, and you get real-time detailed information on CPU, memory, network and disk utilization. Acronis Storage is even a drop-in replacement for Ceph or Swift. You don’t need to convert from one data storage infrastructure to something completely different in order to use Acronis Storage.

Ceph deployments often require a major effort and require careful instrumentation to optimize and fine-tune cluster performance and reliability for the chosen hardware. Note that the fairly recent Ceph Filesystem (Ceph FS) option isn't mature enough to be deployed for production use due to its limited data protection features.

Planning for and running Ceph as a storage platform requires deep Linux and Ceph expertise, and installing Ceph is a multi-module, multi-step deployment process. Ceph takes a non-native, layered approach to delivering block storage, in which “block” is conceptually higher than “object.”


Ceph’s approach to block (RADOS Block Device – RBD) storage incurs extreme amounts of overhead. This results in high latency and an inability to exploit Flash (SSD) media. In addition to downloading and compiling the Ceph source code, you’ll need to compile/decompile/manually edit CRUSH maps. The effort is not for the faint of heart.

Swift has similar limitations. Swift requires significant tuning to optimize performance for a particular configuration of hardware and a specific workload, and this often limits a Swift storage facility to a single use case. Creating a general-purpose Swift storage facility for multiple types of data access may not be possible.

Moreover, Swift has focused much of its efforts on the Amazon S3 API due to the sheer size of the S3 ecosystem and the associated market opportunity. This leaves Swift’s commitment to the OpenStack Swift API in question.

Acronis’ licensing for Acronis Storage is just $0.015 per GB/month, with volume discounts. Considering the costs of Do-It-Yourself Ceph or Swift – compiling, installation, administration and ongoing support – Acronis Storage is actually less expensive in terms of Total Cost of Ownership (TCO).

Conclusion

Acronis Storage easily wins this review. Acronis Storage clearly offers better performance, better features and lower costs than Ceph or Swift. We also found Acronis Storage to have superior, advanced technological advantages over Ceph and Swift.

We note, however, that Software Defined Storage is a technology that’s just starting to take market share from SAN/NAS. At the present, SDS products lack many enterprise grade features, such as tools for geo-replication.
How We Tested

The testbed network consisted of six 10-Gigabit Ethernet subnet domains connected by Cisco ASR 1006-X routers. We configured the NICs to use balance-xor bonding in some tests and no bonding in other tests. Our lab's 150 clients consisted of computing platforms that included Windows 2012, Windows 8, Windows 10, Macintosh 10.x and Red Hat Linux (both server and workstation editions).

The relational databases on the network were Oracle 12 and Microsoft SQL Server 2012. The network also contained two Web servers (Microsoft IIS and Apache), three e-mail servers (Exchange, Notes and iMail) and several Windows 2012 file servers. Our virtual computing environments consisted of VMware, XenServer and Microsoft Hyper-V.

Our test platform consisted of a cluster of eight PowerEdge R720 servers with Dual Xeon E5-26xx processors, 384 GB RAM and 32 TB of disk storage, consisting of four Seagate ST8000NM0055 8TB SATA drives. Four of the servers had two 1.6 TB Dell PowerEdge Express Flash NVMe PCIe SSDs, while the other four had four 1.6 TB Dell PowerEdge Express Flash NVMe PCIe SSDs.

We used a variety of computer programs written in the C language and developed in-house, including a multi-threaded database benchmark, to measure the performance of these storage products.


Vendor Info

Acronis Storage

Ceph

Swift

  Acronis International GmbH



781-782-9000



www.Acronis.com

www.ceph.com

www.openstack.com

Copyright 2017 Network Testing Labs


  
Home

About Network Testing Labs

Contact Network Testing Labs