Stopwatch
Benchmark report #3190
OWH – SP-32 Server - E3-1270v6 - 32GB
Report
Share test result

Test results for SP-32 Server - E3-1270v6 - 32GB at OWH


Server specs:


8 × Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz
31 GB RAM / 432 GB disk space
Ubuntu 17.04 Zesty
Beauharnois, Canada

Benchmark results summary:


UnixBench - 7653.5
Disk Read - 18489 MB/s
Disk Write - 3849 MB/s
Bandwidth - 642.41 MB/s

More: https://serverscope.io/trials/rB6z
Share
Share this page
Cpu chip
Server Specs
CPU
8 × Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz
CPU Cores
4 × 900 MHz
RAM
31 GB
Disk
432 GB
OS
Ubuntu 17.04 Zesty
Location
Beauharnois, Canada
Benchmark summary
Benchmark summary
UnixBench
7653.5
Disk Read
18489 MB/s
Disk Write
3849 MB/s
Bandwidth
642.41 Mbit/s
Speedtest
300.34 Mbit/s
Graph analysis
UnixBench Score
UnixBench (all CPUs)
7653.5
UnixBench (one CPU)
2353.8
Terminal
Raw Output
gcc -o pgms/arithoh -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME -Darithoh src/arith.c 
gcc -o pgms/register -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME -Ddatum="register int" src/arith.c 
gcc -o pgms/short -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME -Ddatum=short src/arith.c 
gcc -o pgms/int -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME -Ddatum=int src/arith.c 
gcc -o pgms/long -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME -Ddatum=long src/arith.c 
gcc -o pgms/float -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME -Ddatum=float src/arith.c 
gcc -o pgms/double -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME -Ddatum=double src/arith.c 
gcc -o pgms/hanoi -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME src/hanoi.c 
gcc -o pgms/syscall -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME src/syscall.c 
gcc -o pgms/context1 -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME src/context1.c 
gcc -o pgms/pipe -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME src/pipe.c 
src/pipe.c: In function xe2x80x98mainxe2x80x99:
src/pipe.c:52:2: warning: ignoring return value of xe2x80x98pipexe2x80x99, declared with attribute warn_unused_result [-Wunused-result]
  pipe(pvec);
  ^~~~~~~~~~
gcc -o pgms/spawn -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME src/spawn.c 
gcc -o pgms/execl -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME src/execl.c 
In file included from src/execl.c:34:0:
src/big.c: In function xe2x80x98dummyxe2x80x99:
src/big.c:109:5: warning: ignoring return value of xe2x80x98freopenxe2x80x99, declared with attribute warn_unused_result [-Wunused-result]
     freopen("masterlog.00", "a", stderr);
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/big.c:197:6: warning: ignoring return value of xe2x80x98freopenxe2x80x99, declared with attribute warn_unused_result [-Wunused-result]
      freopen(logname, "w", stderr);
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/big.c:221:3: warning: ignoring return value of xe2x80x98dupxe2x80x99, declared with attribute warn_unused_result [-Wunused-result]
   dup(pvec[0]);
   ^~~~~~~~~~~~
src/big.c:225:6: warning: ignoring return value of xe2x80x98freopenxe2x80x99, declared with attribute warn_unused_result [-Wunused-result]
      freopen(logname, "w", stderr);
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/big.c:318:4: warning: ignoring return value of xe2x80x98writexe2x80x99, declared with attribute warn_unused_result [-Wunused-result]
    write(fcopy, cp->line, p - cp->line + 1);
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
gcc -o pgms/dhry2 -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME -DHZ= ./src/dhry_1.c ./src/dhry_2.c
gcc -o pgms/dhry2reg -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME -DHZ= -DREG=register ./src/dhry_1.c ./src/dhry_2.c
gcc -o pgms/looper -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME src/looper.c 
gcc -o pgms/fstime -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME src/fstime.c 
gcc -o pgms/whetstone-double -Wall -pedantic -O3 -ffast-math -march=native -mtune=native -I ./src -DTIME -DDP -DUNIX -DUNIXBENCH src/whets.c -lm
make all
make[1]: Entering directory "/root/serverscope-2ARbg5/byte-unixbench/UnixBench"
make distr
make[2]: Entering directory "/root/serverscope-2ARbg5/byte-unixbench/UnixBench"
Checking distribution of files
./pgms  exists
./src  exists
./testdir  exists
make[2]: Leaving directory "/root/serverscope-2ARbg5/byte-unixbench/UnixBench"
make programs
make[2]: Entering directory "/root/serverscope-2ARbg5/byte-unixbench/UnixBench"
make[2]: Nothing to be done for "programs".
make[2]: Leaving directory "/root/serverscope-2ARbg5/byte-unixbench/UnixBench"
make[1]: Leaving directory "/root/serverscope-2ARbg5/byte-unixbench/UnixBench"
sh: 3dinfo: command not found

   #    #  #    #  #  #    #          #####   ######  #    #   ####   #    #
   #    #  ##   #  #   #  #           #    #  #       ##   #  #    #  #    #
   #    #  # #  #  #    ##            #####   #####   # #  #  #       ######
   #    #  #  # #  #    ##            #    #  #       #  # #  #       #    #
   #    #  #   ##  #   #  #           #    #  #       #   ##  #    #  #    #
    ####   #    #  #  #    #          #####   ######  #    #   ####   #    #

   Version 5.1.3                      Based on the Byte Magazine Unix Benchmark

   Multi-CPU version                  Version 5 revisions by Ian Smith,
                                      Sunnyvale, CA, USA
   January 13, 2011                   johantheghost at yahoo period com


1 x Dhrystone 2 using register variables  1 2 3 4 5 6 7 8 9 10

1 x Double-Precision Whetstone  1 2 3 4 5 6 7 8 9 10

1 x Execl Throughput  1 2 3

1 x File Copy 1024 bufsize 2000 maxblocks  1 2 3

1 x File Copy 256 bufsize 500 maxblocks  1 2 3

1 x File Copy 4096 bufsize 8000 maxblocks  1 2 3

1 x Pipe Throughput  1 2 3 4 5 6 7 8 9 10

1 x Pipe-based Context Switching  1 2 3 4 5 6 7 8 9 10

1 x Process Creation  1 2 3

1 x System Call Overhead  1 2 3 4 5 6 7 8 9 10

1 x Shell Scripts (1 concurrent)  1 2 3

1 x Shell Scripts (8 concurrent)  1 2 3

8 x Dhrystone 2 using register variables  1 2 3 4 5 6 7 8 9 10

8 x Double-Precision Whetstone  1 2 3 4 5 6 7 8 9 10

8 x Execl Throughput  1 2 3

8 x File Copy 1024 bufsize 2000 maxblocks  1 2 3

8 x File Copy 256 bufsize 500 maxblocks  1 2 3

8 x File Copy 4096 bufsize 8000 maxblocks  1 2 3

8 x Pipe Throughput  1 2 3 4 5 6 7 8 9 10

8 x Pipe-based Context Switching  1 2 3 4 5 6 7 8 9 10

8 x Process Creation  1 2 3

8 x System Call Overhead  1 2 3 4 5 6 7 8 9 10

8 x Shell Scripts (1 concurrent)  1 2 3

8 x Shell Scripts (8 concurrent)  1 2 3

========================================================================
   BYTE UNIX Benchmarks (Version 5.1.3)

   System: node1.execsrvr.xyz: GNU/Linux
   OS: GNU/Linux -- 4.10.0-20-generic -- #22-Ubuntu SMP Thu Apr 20 09:22:42 UTC 2017
   Machine: x86_64 (x86_64)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz (7584.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   CPU 1: Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz (7584.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   CPU 2: Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz (7584.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   CPU 3: Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz (7584.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   CPU 4: Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz (7584.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   CPU 5: Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz (7584.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   CPU 6: Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz (7584.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   CPU 7: Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz (7584.0 bogomips)
          Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET, Intel virtualization
   22:16:03 up 47 min,  1 user,  load average: 7.88, 4.53, 1.90; runlevel 2017-04-25

------------------------------------------------------------------------
Benchmark Run: Tue Apr 25 2017 22:16:03 - 22:45:31
8 CPUs in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       50153480.7 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     3598.2 MWIPS (18.7 s, 7 samples)
Execl Throughput                               9242.7 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks       1697524.1 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          452630.9 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       3966734.7 KBps  (30.0 s, 2 samples)
Pipe Throughput                             3095802.5 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 238014.0 lps   (10.0 s, 7 samples)
Process Creation                               7662.8 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   8389.5 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   5409.5 lpm   (60.0 s, 2 samples)
System Call Overhead                        5572503.5 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   50153480.7   4297.6
Double-Precision Whetstone                       55.0       3598.2    654.2
Execl Throughput                                 43.0       9242.7   2149.5
File Copy 1024 bufsize 2000 maxblocks          3960.0    1697524.1   4286.7
File Copy 256 bufsize 500 maxblocks            1655.0     452630.9   2734.9
File Copy 4096 bufsize 8000 maxblocks          5800.0    3966734.7   6839.2
Pipe Throughput                               12440.0    3095802.5   2488.6
Pipe-based Context Switching                   4000.0     238014.0    595.0
Process Creation                                126.0       7662.8    608.2
Shell Scripts (1 concurrent)                     42.4       8389.5   1978.7
Shell Scripts (8 concurrent)                      6.0       5409.5   9015.9
System Call Overhead                          15000.0    5572503.5   3715.0
                                                                   ========
System Benchmarks Index Score                                        2353.8

------------------------------------------------------------------------
Benchmark Run: Tue Apr 25 2017 22:45:31 - 23:15:20
8 CPUs in system; running 8 parallel copies of tests

Dhrystone 2 using register variables      271857671.6 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                    22344.1 MWIPS (20.5 s, 7 samples)
Execl Throughput                              35772.4 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks       1613293.2 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          423292.7 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       4751809.1 KBps  (30.0 s, 2 samples)
Pipe Throughput                            14558341.1 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                2218428.0 lps   (10.0 s, 7 samples)
Process Creation                             111786.7 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                  49548.9 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   7914.8 lpm   (60.0 s, 2 samples)
System Call Overhead                       10156381.0 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0  271857671.6  23295.4
Double-Precision Whetstone                       55.0      22344.1   4062.6
Execl Throughput                                 43.0      35772.4   8319.2
File Copy 1024 bufsize 2000 maxblocks          3960.0    1613293.2   4074.0
File Copy 256 bufsize 500 maxblocks            1655.0     423292.7   2557.7
File Copy 4096 bufsize 8000 maxblocks          5800.0    4751809.1   8192.8
Pipe Throughput                               12440.0   14558341.1  11702.8
Pipe-based Context Switching                   4000.0    2218428.0   5546.1
Process Creation                                126.0     111786.7   8872.0
Shell Scripts (1 concurrent)                     42.4      49548.9  11686.1
Shell Scripts (8 concurrent)                      6.0       7914.8  13191.4
System Call Overhead                          15000.0   10156381.0   6770.9
                                                                   ========
System Benchmarks Index Score                                        7653.5

Hard drive
dd
dd if=/dev/zero of=benchmark bs=64K count=32K conv=fdatasync
32768+0 records in
32768+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 3.94461 s, 544 MB/s


dd if=/dev/zero of=benchmark bs=1M count=2048 conv=fdatasync
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 3.84832 s, 558 MB/s


Hard drive
FIO random read
Performance
18489.0 MB/s
IOPS
4846899
./fio --time_based --name=benchmark --size=256M --runtime=60 --randrepeat=1 --iodepth=32 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=8 --rw=randread --blocksize=4k --group_reporting
benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=24629: Tue Apr 25 22:13:00 2017
  read : io=1083.4GB, bw=18489MB/s, iops=4733.3K, runt= 60001msec
    clat (usec): min=0, max=10074, avg= 1.24, stdev= 5.44
     lat (usec): min=0, max=10074, avg= 1.27, stdev= 5.44
    clat percentiles (usec):
     |  1.00th=[    0],  5.00th=[    0], 10.00th=[    1], 20.00th=[    1],
     | 30.00th=[    1], 40.00th=[    1], 50.00th=[    1], 60.00th=[    1],
     | 70.00th=[    1], 80.00th=[    1], 90.00th=[    1], 95.00th=[    2],
     | 99.00th=[    2], 99.50th=[    2], 99.90th=[  107], 99.95th=[  131],
     | 99.99th=[  175]
    bw (MB  /s): min=   26, max= 3773, per=12.49%, avg=2308.43, stdev=889.32
    lat (usec) : 2=90.44%, 4=9.35%, 10=0.02%, 20=0.01%, 50=0.01%
    lat (usec) : 100=0.05%, 250=0.13%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%
  cpu          : usr=29.07%, sys=59.84%, ctx=540726, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=283997815/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=1083.4GB, aggrb=18489MB/s, minb=18489MB/s, maxb=18489MB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
    md3: ios=524288/1004, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=262144/389, aggrmerge=0/668, aggrticks=27050/138, aggrin_queue=27398, aggrutil=15.18%
  nvme0n1: ios=266996/396, merge=0/661, ticks=28336/128, in_queue=28820, util=15.18%
  nvme1n1: ios=257292/382, merge=0/675, ticks=25764/148, in_queue=25976, util=14.19%
Hard drive
FIO random direct read
Performance
275.0 MB/s
IOPS
70400
./fio --time_based --name=benchmark --size=256M --runtime=60 --randrepeat=1 --iodepth=32 --direct=1 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=8 --rw=randread --blocksize=4k --group_reporting
benchmark: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes

benchmark: (groupid=0, jobs=8): err= 0: pid=24731: Tue Apr 25 22:14:01 2017
  read : io=16500MB, bw=281601KB/s, iops=70400, runt= 60001msec
    clat (usec): min=19, max=3836, avg=111.52, stdev=34.40
     lat (usec): min=20, max=3837, avg=111.67, stdev=34.40
    clat percentiles (usec):
     |  1.00th=[   79],  5.00th=[   80], 10.00th=[   81], 20.00th=[   82],
     | 30.00th=[   98], 40.00th=[  100], 50.00th=[  103], 60.00th=[  110],
     | 70.00th=[  115], 80.00th=[  139], 90.00th=[  145], 95.00th=[  169],
     | 99.00th=[  217], 99.50th=[  235], 99.90th=[  282], 99.95th=[  310],
     | 99.99th=[ 1080]
    bw (KB  /s): min=27232, max=37904, per=12.50%, avg=35197.28, stdev=2744.97
    lat (usec) : 20=0.01%, 50=0.05%, 100=35.08%, 250=64.57%, 500=0.28%
    lat (usec) : 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%
  cpu          : usr=2.52%, sys=10.07%, ctx=4224191, majf=0, minf=88
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=4224089/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=16500MB, aggrb=281601KB/s, minb=281601KB/s, maxb=281601KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
    md3: ios=4220317/20, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=2112044/31, aggrmerge=0/8, aggrticks=213356/0, aggrin_queue=214432, aggrutil=100.00%
  nvme0n1: ios=2106454/31, merge=0/8, ticks=217448/0, in_queue=219040, util=100.00%
  nvme1n1: ios=2117635/31, merge=0/8, ticks=209264/0, in_queue=209824, util=100.00%
Hard drive
FIO random write
Performance
3849.2 MB/s
IOPS
985595
./fio --time_based --name=benchmark --size=256M --runtime=60 --filename=benchmark --randrepeat=1 --iodepth=32 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=8 --rw=randwrite --blocksize=4k --group_reporting
benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes

benchmark: (groupid=0, jobs=8): err= 0: pid=24852: Tue Apr 25 22:16:02 2017
  write: io=231003MB, bw=3849.2MB/s, iops=985595, runt= 60001msec
    clat (usec): min=0, max=44646, avg= 7.68, stdev=109.97
     lat (usec): min=0, max=44646, avg= 7.73, stdev=109.97
    clat percentiles (usec):
     |  1.00th=[    1],  5.00th=[    2], 10.00th=[    2], 20.00th=[    4],
     | 30.00th=[    6], 40.00th=[    7], 50.00th=[    7], 60.00th=[    7],
     | 70.00th=[    8], 80.00th=[    8], 90.00th=[    8], 95.00th=[    8],
     | 99.00th=[    9], 99.50th=[    9], 99.90th=[   19], 99.95th=[   23],
     | 99.99th=[ 5344]
    bw (KB  /s): min=379720, max=617568, per=12.50%, avg=492907.70, stdev=39360.97
    lat (usec) : 2=1.84%, 4=13.61%, 10=84.11%, 20=0.37%, 50=0.04%
    lat (usec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=5.75%, sys=80.33%, ctx=10135021, majf=0, minf=84
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=59136735/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=231003MB, aggrb=3849.2MB/s, minb=3849.2MB/s, maxb=3849.2MB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
    md3: ios=0/6957, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=0/6810, aggrmerge=0/232, aggrticks=0/154862, aggrin_queue=154882, aggrutil=2.16%
  nvme0n1: ios=0/6819, merge=0/223, ticks=0/155804, in_queue=155844, util=2.16%
  nvme1n1: ios=0/6801, merge=0/241, ticks=0/153920, in_queue=153920, util=2.11%
Hard drive
FIO random direct write
Performance
233.79 MB/s
IOPS
59849
./fio --time_based --name=benchmark --size=256M --runtime=60 --filename=benchmark --randrepeat=1 --iodepth=32 --direct=1 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=8 --rw=randwrite --blocksize=4k --group_reporting
benchmark: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=32
...
fio-2.8
Starting 8 processes
benchmark: Laying out IO file(s) (1 file(s) / 256MB)

benchmark: (groupid=0, jobs=8): err= 0: pid=24791: Tue Apr 25 22:15:01 2017
  write: io=14027MB, bw=239397KB/s, iops=59849, runt= 60001msec
    clat (usec): min=9, max=144382, avg=132.72, stdev=1809.16
     lat (usec): min=9, max=144382, avg=132.91, stdev=1809.17
    clat percentiles (usec):
     |  1.00th=[   11],  5.00th=[   12], 10.00th=[   12], 20.00th=[   13],
     | 30.00th=[   13], 40.00th=[   14], 50.00th=[   14], 60.00th=[   15],
     | 70.00th=[   16], 80.00th=[   17], 90.00th=[   20], 95.00th=[   26],
     | 99.00th=[ 1400], 99.50th=[ 6432], 99.90th=[28032], 99.95th=[41728],
     | 99.99th=[70144]
    bw (KB  /s): min=   97, max=165463, per=12.53%, avg=29995.36, stdev=27419.03
    lat (usec) : 10=0.01%, 20=88.76%, 50=9.98%, 100=0.03%, 250=0.03%
    lat (usec) : 500=0.05%, 750=0.05%, 1000=0.04%
    lat (msec) : 2=0.14%, 4=0.23%, 10=0.36%, 20=0.18%, 50=0.13%
    lat (msec) : 100=0.03%, 250=0.01%
  cpu          : usr=1.02%, sys=6.43%, ctx=7180365, majf=0, minf=83
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=3591009/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=14027MB, aggrb=239396KB/s, minb=239396KB/s, maxb=239396KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
    md3: ios=0/3587567, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=0/3591143, aggrmerge=0/647, aggrticks=0/32000, aggrin_queue=31776, aggrutil=53.80%
  nvme0n1: ios=0/3591143, merge=0/647, ticks=0/31492, in_queue=31228, util=51.97%
  nvme1n1: ios=0/3591143, merge=0/647, ticks=0/32508, in_queue=32324, util=53.80%
Download
Download benchmark results
Download speed
642.41 Mbit/s
Downloaded 104857600 bytes in 2.455887 sec
Downloaded 104857600 bytes in 0.942319 sec
Downloaded 104857600 bytes in 0.941183 sec
Downloaded 104857600 bytes in 0.945551 sec
Downloaded 104857600 bytes in 0.941617 sec
Finished! Average download speed is 642.41 Mbit/s
Upload
Speedtest results
Upload speed
300.34 Mbit/s
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from OVH ...
Selecting 15 servers that are not too close:
  1. cronomagic (Montreal, QC) [31.08 km]: 36.883 ms
  2. TELUS (Montreal, QC) [31.08 km]: 59.571 ms
  3. Fibrenoire Internet (Montreal, QC) [31.08 km]: 76.178 ms
  4. Unilink Networks Inc. (Montreal, QC) [31.08 km]: 175.157 ms
  5. HOTServers LLC (Montreal, QC) [31.08 km]: 31.013 ms
  6. Rogers (Montreal, QC) [31.08 km]: 46.069 ms
  7. Ebox (Montreal, QC) [31.08 km]: 35.957 ms
  8. Netelligent Hosting Services Inc. (Montreal, QC) [31.08 km]: 17.882 ms
  9. Magic-VPN (Montreal, QC) [31.08 km]: 181.088 ms
  10. GlideServers (Montreal, QC) [31.08 km]: 32.041 ms
  11. Hostix Technologies Inc. (Saint-Paul-de-l'lle-aux-Noix, QC) [49.91 km]: 38.1 ms
  12. Internexe (Knowlton, QC) [106.51 km]: 65.569 ms
  13. Sovernet Communications (Burlington, VT) [110.07 km]: 210.996 ms
  14. Slic Network Solutions (Potsdam, NY) [113.54 km]: 66.791 ms
  15. Green Mountain Access (Richmond, VT) [122.67 km]: 50.774 ms
Testing upload speeds
  1. cronomagic (Montreal, QC):                   ......................... 22.90 Mbit/s
  2. TELUS (Montreal, QC):                        ......................... 251.56 Mbit/s
  3. Fibrenoire Internet (Montreal, QC):          ......................... 364.17 Mbit/s
  4. Unilink Networks Inc. (Montreal, QC):        ......................... 328.43 Mbit/s
  5. HOTServers LLC (Montreal, QC):               ......................... 239.00 Mbit/s
  6. Rogers (Montreal, QC):                       ......................... 106.43 Mbit/s
  7. Ebox (Montreal, QC):                         ......................... 378.32 Mbit/s
  8. Netelligent Hosting Services Inc. (Montreal, QC):......................... 412.96 Mbit/s
  9. Magic-VPN (Montreal, QC):                    ......................... 701.83 Mbit/s
  10. GlideServers (Montreal, QC):                ......................... 642.04 Mbit/s
  11. Hostix Technologies Inc. (Saint-Paul-de-l'lle-aux-Noix, QC):......................... 66.56 Mbit/s
  12. Internexe (Knowlton, QC):                   ......................... 78.92 Mbit/s
  13. Sovernet Communications (Burlington, VT):   ......................... 25.44 Mbit/s
  14. Slic Network Solutions (Potsdam, NY):       ......................... 201.48 Mbit/s
  15. Green Mountain Access (Richmond, VT):       ......................... 132.66 Mbit/s
Average upload speed is 263.51 Mbit/s