leveldb和rocksdb性能对比

  |   0 评论   |   8,429 浏览

    C++版本:https://github.com/facebook/rocksdb/wiki/Performance-Benchmarks

    Java版本:https://github.com/facebook/rocksdb/wiki/RocksJava-Performance-on-Flash-Storage


    详细的机器配置看上面的链接,强烈建议读一下。

    注意:测试使用的是jemalloc


    测试一:将1B(即10亿)个key数据随机批量写入空数据库



    rocksdb:   103 minutes, 80 MB/sec (total data size 481 GB, 1 billion key-values)
    leveldb:   many many days (in 20 hours it inserted only 200 million key-values)


    原因:

    rocksdb其实使用了两步:先做fillrandom,再做compact

    leveldb只是使用了一步:fillrandom


    测试二:将1B个key数据顺序批量插入空数据库



    rocksdb:   36 minutes, 370 MB/sec (total data size 760 GB)
    leveldb:   91 minutes, 146 MB/sec (total data size 760 GB)
    
    
    Java
    
    
    fillseq          :     2.48233 micros/op;  311.2 MB/s; 1000000000 ops done;  1 / 1 task(s) finished.


    原因:

    rocksdb使用了多线程compact


    测试三:随机覆盖1B个key数据




    rocksdb: 15 hours 38 min;  56.295 micros/op, 17K ops/sec,  13.8 MB/sec
    leveldb: many many days;  600 micros/op,     1.6K ops/sec, 1.3 MB/sec
              (in 5 days it overwrote only 662 million out of 1 billion keys, after which I killed the test)


    原因:

    rocksdb使用了20个compact线程,使用了6个level,仅了level 2及以上启用了gzip


    测试四:随机读取1B个key数据



    rocksdb:  70 hours,  8 micros/op, 126K ops/sec (checksum verification)
    leveldb: 102 hours, 12 micros/op,  83K ops/sec (no checksum verification)
    
    
    Java:
    
    
    readrandom       :     7.67180 micros/op;  101.4 MB/s; 1000000000 / 1000000000 found;  32 / 32 task(s) finished.


    配置:rocksdb额外启用了checksum,leveldb关闭了checksum

    原因:rocksdb没有用mmaped IOs,这个在某些linux平台上会慢。rocksdb把block cache分成了64份,以降低锁力度。




    测试五:多线程读,同时单线程写



    rocksdb: 11 hours 30 minutes, 9.640 micros/read, 103734 reads/sec
    leveldb: 20 hours 28 minutes


    Java:

    readwhilewriting :     9.55882 micros/op;   81.4 MB/s; 100000000 / 100000000 found;  32 / 32 task(s) finished.



    配置:rocksdb release 2.7 and leveldb 1.15



    PS: RocksDB的配置,见官网:https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide


    中文参考:http://www.jianshu.com/p/ddf652aa4882



    评论

    发表评论

    validate