Metadata operations have become dominant file operations in the storage systems. In the scenarios of read-more and write-less of massive small files, the current distributed file systems suffer from the unsatisfying performance and scalability of metadata service because of random disk I/O during metadata operations. In this paper, a highly efficient metadata management architecture for cloud storage systems is proposed. The cluster design significantly improves the scalability of the system. A concept of the disk I/O channel is introduced, which is an independent data storage pipe by binding an independent CPU-core to each physical disk. In addition, a multi-channel fast key-value storage engine is proposed to provide the extremely efficient performance for the underlying storage service, which takes full advantages of multi-core processors and parallel disks I/O. Besides, a new dynamic load-balancing strategy is proposed to reduce load thrashing and improve the precision of rebalancing among the clusters. Performance measurements under a variety of benchmarks show that the metadata management is capable of handling the massive small files storage and the performance is improved significantly compared to the existing solutions. Copyright © 2016 John Wiley & Sons, Ltd.

+More

multicore processors metadata management architecture multichannel fast keyvalue storage engine distributed file systems cpucore ltd random disk io during metadata parallel disks precision of rebalancing cluster design dynamic loadbalancing strategy load thrashing independent data storage pipe solutions operations writeless of massive small files readmore cloud storage systems

Geyong Min, Pengcheng Lv, Xiaoke Xiang,Hancong Duan, Yulu Zhang, Wenhan Zhan,.A multi-channel architecture for metadata management in cloud storage systems by binding CPU-cores to disks. 28 (13),.

Geyong Min, et al."A multi-channel architecture for metadata management in cloud storage systems by binding CPU-cores to disks" 28

Geyong Min, Pengcheng Lv, Xiaoke Xiang,Hancong Duan, Yulu Zhang, Wenhan Zhan,"A multi-channel architecture for metadata management in cloud storage systems by binding CPU-cores to disks"