dd if=/dev/zero of=benchmark bs=64K count=64K conv=fdatasync 65536+0 records in 65536+0 records out 4294967296 bytes (4.3 GB) copied, 28.5746 s, 150 MB/s dd if=/dev/zero of=benchmark bs=1M count=4096 conv=fdatasync 4096+0 records in 4096+0 records out 4294967296 bytes (4.3 GB) copied, 28.3842 s, 151 MB/s
./fio --time_based --name=benchmark --size=1535M --runtime=60 --ioengine=libaio --randrepeat=1 --iodepth=32 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=8 --rw=randread --blocksize=4k --group_reporting
benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 1535MB)
benchmark: Laying out IO file(s) (1 file(s) / 1535MB)
benchmark: Laying out IO file(s) (1 file(s) / 1535MB)
benchmark: Laying out IO file(s) (1 file(s) / 1535MB)
benchmark: Laying out IO file(s) (1 file(s) / 1535MB)
benchmark: Laying out IO file(s) (1 file(s) / 1535MB)
benchmark: Laying out IO file(s) (1 file(s) / 1535MB)
benchmark: Laying out IO file(s) (1 file(s) / 1535MB)
benchmark: (groupid=0, jobs=8): err= 0: pid=9868: Fri Oct 28 17:09:24 2016
  read : io=76832KB, bw=1278.2KB/s, iops=319, runt= 60111msec
    slat (usec): min=165, max=274942, avg=24991.64, stdev=21310.11
    clat (usec): min=3, max=1708.7K, avg=767720.03, stdev=239039.27
     lat (msec): min=5, max=1768, avg=792.71, stdev=244.29
    clat percentiles (msec):
     |  1.00th=[  281],  5.00th=[  420], 10.00th=[  486], 20.00th=[  578],
     | 30.00th=[  635], 40.00th=[  685], 50.00th=[  742], 60.00th=[  799],
     | 70.00th=[  873], 80.00th=[  963], 90.00th=[ 1090], 95.00th=[ 1205],
     | 99.00th=[ 1434], 99.50th=[ 1500], 99.90th=[ 1614], 99.95th=[ 1631],
     | 99.99th=[ 1663]
    bw (KB  /s): min=    3, max=  493, per=12.45%, avg=159.07, stdev=57.36
    lat (usec) : 4=0.02%, 10=0.02%
    lat (msec) : 10=0.01%, 20=0.02%, 50=0.05%, 100=0.10%, 250=0.59%
    lat (msec) : 500=10.72%, 750=40.38%, 1000=32.05%, 2000=16.04%
  cpu          : usr=0.05%, sys=0.40%, ctx=19272, majf=0, minf=348
  IO depths    : 1=0.1%, 2=0.1%, 4=0.2%, 8=0.3%, 16=0.7%, 32=98.7%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=19208/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
   READ: io=76832KB, aggrb=1278KB/s, minb=1278KB/s, maxb=1278KB/s, mint=60111msec, maxt=60111msec
Disk stats (read/write):
  sda: ios=19206/2, merge=0/1, ticks=478816/172, in_queue=540140, util=99.92%
./fio --time_based --name=benchmark --size=1535M --runtime=60 --ioengine=libaio --randrepeat=1 --iodepth=32 --direct=1 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=8 --rw=randread --blocksize=4k --group_reporting
benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: (groupid=0, jobs=8): err= 0: pid=9878: Fri Oct 28 17:10:25 2016
  read : io=86812KB, bw=1437.2KB/s, iops=359, runt= 60370msec
    slat (usec): min=1, max=925698, avg=15571.12, stdev=86843.13
    clat (msec): min=1, max=2167, avg=694.53, stdev=194.09
     lat (msec): min=1, max=2167, avg=710.10, stdev=204.83
    clat percentiles (msec):
     |  1.00th=[  186],  5.00th=[  474], 10.00th=[  545], 20.00th=[  586],
     | 30.00th=[  635], 40.00th=[  652], 50.00th=[  668], 60.00th=[  701],
     | 70.00th=[  725], 80.00th=[  775], 90.00th=[  881], 95.00th=[  988],
     | 99.00th=[ 1467], 99.50th=[ 1582], 99.90th=[ 1942], 99.95th=[ 1975],
     | 99.99th=[ 2089]
    bw (KB  /s): min=    4, max=  412, per=12.58%, avg=180.74, stdev=53.20
    lat (msec) : 2=0.01%, 10=0.03%, 20=0.09%, 50=0.06%, 100=0.15%
    lat (msec) : 250=1.79%, 500=5.00%, 750=68.85%, 1000=19.37%, 2000=4.64%
    lat (msec) : >=2000=0.01%
  cpu          : usr=0.01%, sys=0.04%, ctx=1704, majf=0, minf=340
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.3%, 16=0.6%, 32=98.9%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=21703/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
   READ: io=86812KB, aggrb=1437KB/s, minb=1437KB/s, maxb=1437KB/s, mint=60370msec, maxt=60370msec
Disk stats (read/write):
  sda: ios=21640/0, merge=0/0, ticks=8630784/0, in_queue=8654628, util=99.92%
./fio --time_based --name=benchmark --size=1535M --runtime=60 --filename=benchmark --ioengine=libaio --randrepeat=1 --iodepth=32 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=8 --rw=randwrite --blocksize=4k --group_reporting
benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: (groupid=0, jobs=8): err= 0: pid=10187: Fri Oct 28 17:12:26 2016
  write: io=4496.7MB, bw=76714KB/s, iops=19178, runt= 60022msec
    slat (usec): min=1, max=1127.6K, avg=384.03, stdev=10705.96
    clat (usec): min=2, max=1478.7K, avg=11933.08, stdev=64162.43
     lat (usec): min=71, max=1478.7K, avg=12317.32, stdev=65215.23
    clat percentiles (usec):
     |  1.00th=[   77],  5.00th=[   82], 10.00th=[   87], 20.00th=[   99],
     | 30.00th=[  110], 40.00th=[  120], 50.00th=[  145], 60.00th=[  167],
     | 70.00th=[  187], 80.00th=[  227], 90.00th=[10048], 95.00th=[35584],
     | 99.00th=[329728], 99.50th=[460800], 99.90th=[831488], 99.95th=[1019904],
     | 99.99th=[1204224]
    bw (KB  /s): min=    0, max=257760, per=14.75%, avg=11314.99, stdev=23499.07
    lat (usec) : 4=0.01%, 10=0.01%, 100=20.19%, 250=64.66%, 500=4.31%
    lat (usec) : 750=0.09%, 1000=0.06%
    lat (msec) : 2=0.12%, 4=0.16%, 10=0.40%, 20=1.64%, 50=4.27%
    lat (msec) : 100=0.98%, 250=1.60%, 500=1.11%, 750=0.27%, 1000=0.08%
    lat (msec) : 2000=0.05%
  cpu          : usr=0.18%, sys=1.74%, ctx=80702, majf=0, minf=85
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1151130/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
  WRITE: io=4496.7MB, aggrb=76713KB/s, minb=76713KB/s, maxb=76713KB/s, mint=60022msec, maxt=60022msec
Disk stats (read/write):
  sda: ios=0/182523, merge=0/50986, ticks=0/433144, in_queue=431548, util=97.56%
./fio --time_based --name=benchmark --size=1535M --runtime=60 --filename=benchmark --ioengine=libaio --randrepeat=1 --iodepth=32 --direct=1 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=8 --rw=randwrite --blocksize=4k --group_reporting
benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 1535MB)
benchmark: (groupid=0, jobs=8): err= 0: pid=9888: Fri Oct 28 17:11:26 2016
  write: io=92792KB, bw=1529.1KB/s, iops=382, runt= 60650msec
    slat (usec): min=2, max=935881, avg=15197.48, stdev=70745.15
    clat (usec): min=346, max=3393.5K, avg=648595.91, stdev=409173.80
     lat (usec): min=627, max=3393.5K, avg=663793.73, stdev=409781.13
    clat percentiles (msec):
     |  1.00th=[   77],  5.00th=[  153], 10.00th=[  212], 20.00th=[  293],
     | 30.00th=[  375], 40.00th=[  474], 50.00th=[  562], 60.00th=[  676],
     | 70.00th=[  816], 80.00th=[  963], 90.00th=[ 1172], 95.00th=[ 1434],
     | 99.00th=[ 1958], 99.50th=[ 2147], 99.90th=[ 2507], 99.95th=[ 2606],
     | 99.99th=[ 3064]
    bw (KB  /s): min=    2, max=  629, per=12.93%, avg=197.75, stdev=116.22
    lat (usec) : 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.18%, 4=0.08%, 10=0.08%, 20=0.03%, 50=0.23%
    lat (msec) : 100=1.29%, 250=12.65%, 500=28.53%, 750=21.97%, 1000=17.10%
    lat (msec) : 2000=17.01%, >=2000=0.81%
  cpu          : usr=0.03%, sys=0.34%, ctx=8572, majf=0, minf=84
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.3%, 16=0.6%, 32=98.9%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=23198/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
  WRITE: io=92792KB, aggrb=1529KB/s, minb=1529KB/s, maxb=1529KB/s, mint=60650msec, maxt=60650msec
Disk stats (read/write):
  sda: ios=0/23354, merge=0/1756, ticks=0/8048180, in_queue=8092952, util=99.91%
Downloaded 104857600 bytes in 1.100 sec Downloaded 104857600 bytes in 1.157 sec Downloaded 104857600 bytes in 1.269 sec Downloaded 104857600 bytes in 1.051 sec Downloaded 104857600 bytes in 1.157 sec Finished! Average download speed is 697.59 Mbit/s
Retrieving speedtest.net configuration... Retrieving speedtest.net server list... Testing from GhostNet ... Selecting 15 servers that are not too close: 1. HillCom Solutions (Alsfeld) [62.87 km]: 13.279 ms 2. Onlime GmbH (Aerzen) [83.89 km]: 40.931 ms 3. Newone (Ilmenau) [120.64 km]: 15.595 ms 4. Thueringer Netkom GmbH (Weimar) [132.12 km]: 15.288 ms 5. Vodafone DE (Frankfurt) [143.27 km]: 419.573 ms 6. SoftLayer Technologies, Inc. (Frankfurt) [143.27 km]: 15.181 ms 7. Tele2 (Frankfurt) [143.27 km]: 8.584 ms 8. fdcservers.net (Frankfurt) [143.27 km]: 8.426 ms 9. LWLcom GmbH (Frankfurt) [143.27 km]: 8.297 ms 10. 23media GmbH (Frankfurt) [143.27 km]: 15.173 ms 11. IP-Projects GmbH & Co. KG (Frankfurt) [143.27 km]: 11.95 ms 12. DEAC (Frankfurt) [143.27 km]: 80.945 ms 13. Jublo Solutions (Frankfurt) [143.27 km]: 17.67 ms 14. Robin Kipp (Frankfurt) [143.27 km]: 12.862 ms 15. LeaseWeb (Frankfurt) [143.27 km]: 13.33 ms Testing upload speeds 1. HillCom Solutions (Alsfeld): ......................... 66.16 Mbit/s 2. Onlime GmbH (Aerzen): ......................... 60.98 Mbit/s 3. Newone (Ilmenau): ......................... 231.96 Mbit/s 4. Thueringer Netkom GmbH (Weimar): ......................... 458.80 Mbit/s 5. Vodafone DE (Frankfurt): ......................... 120.46 Mbit/s 6. SoftLayer Technologies, Inc. (Frankfurt): ......................... 467.19 Mbit/s 7. Tele2 (Frankfurt): ......................... 118.61 Mbit/s 8. fdcservers.net (Frankfurt): ......................... 30.73 Mbit/s 9. LWLcom GmbH (Frankfurt): ......................... 367.30 Mbit/s 10. 23media GmbH (Frankfurt): ......................... 120.96 Mbit/s 11. IP-Projects GmbH & Co. KG (Frankfurt): ......................... 14.73 Mbit/s 12. DEAC (Frankfurt): ......................... 25.44 Mbit/s 13. Jublo Solutions (Frankfurt): ......................... 266.55 Mbit/s 14. Robin Kipp (Frankfurt): ......................... 115.39 Mbit/s 15. LeaseWeb (Frankfurt): ......................... 628.69 Mbit/s Average upload speed is 206.26 Mbit/s