История изменений
Исправление Harliff, (текущая версия) :
~/fio/rw-test-depth-8 # fio ./rw-test-zfs2.fio
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8
fio-2.16
Starting 2 processes
Jobs: 1 (f=1): [r(1),_(1)] [99.9% done] [174.2MB/0KB/0KB /s] [44.6K/0/0 iops] [eta 00m:02s]
readtest: (groupid=0, jobs=1): err= 0: pid=17037: Sun May 5 06:42:41 2019
read : io=102400MB, bw=41837KB/s, iops=10459, runt=2506328msec
slat (usec): min=1, max=5036, avg= 4.17, stdev= 4.40
clat (usec): min=0, max=207427, avg=759.50, stdev=2928.52
lat (usec): min=7, max=207431, avg=763.78, stdev=2928.76
clat percentiles (usec):
| 1.00th=[ 13], 5.00th=[ 16], 10.00th=[ 21], 20.00th=[ 62],
| 30.00th=[ 124], 40.00th=[ 197], 50.00th=[ 211], 60.00th=[ 221],
| 70.00th=[ 237], 80.00th=[ 354], 90.00th=[ 2608], 95.00th=[ 3760],
| 99.00th=[ 6176], 99.50th=[ 7200], 99.90th=[10432], 99.95th=[100864],
| 99.99th=[103936]
lat (usec) : 2=0.01%, 4=0.01%, 10=0.06%, 20=8.81%, 50=7.58%
lat (usec) : 100=11.67%, 250=46.67%, 500=5.80%, 750=0.68%, 1000=0.80%
lat (msec) : 2=5.04%, 4=8.67%, 10=4.09%, 20=0.05%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.06%
cpu : usr=2.72%, sys=6.82%, ctx=18518361, majf=0, minf=16
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=26214400/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=8
writetest: (groupid=0, jobs=1): err= 0: pid=17038: Sun May 5 06:42:41 2019
write: io=102400MB, bw=47904KB/s, iops=11976, runt=2188892msec
slat (usec): min=2, max=4064, avg= 4.17, stdev= 4.26
clat (usec): min=1, max=2039.4K, avg=662.80, stdev=18673.45
lat (usec): min=14, max=2039.5K, avg=667.06, stdev=18673.46
clat percentiles (usec):
| 1.00th=[ 17], 5.00th=[ 21], 10.00th=[ 24], 20.00th=[ 31],
| 30.00th=[ 70], 40.00th=[ 90], 50.00th=[ 149], 60.00th=[ 241],
| 70.00th=[ 370], 80.00th=[ 490], 90.00th=[ 676], 95.00th=[ 836],
| 99.00th=[ 1432], 99.50th=[ 2160], 99.90th=[100864], 99.95th=[103936],
| 99.99th=[1056768]
lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=2.85%, 50=24.13%
lat (usec) : 100=15.73%, 250=18.00%, 500=19.94%, 750=12.12%, 1000=4.63%
lat (msec) : 2=2.04%, 4=0.39%, 10=0.07%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.07%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%, >=2000=0.01%
cpu : usr=2.25%, sys=6.79%, ctx=10904659, majf=0, minf=9
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=26214400/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=8
Run status group 0 (all jobs):
READ: io=102400MB, aggrb=41837KB/s, minb=41837KB/s, maxb=41837KB/s, mint=2506328msec, maxt=2506328msec
WRITE: io=102400MB, aggrb=47904KB/s, minb=47904KB/s, maxb=47904KB/s, mint=2188892msec, maxt=2188892msec
fio ./rw-test-zfs2.fio 120.68s user 322.81s system 17% cpu 41:45.90 total
Исходная версия Harliff, :
Тест zfs на /dev/sdb (где тестировал до этого lvm)...
~/fio/rw-test-depth-8 # fio ./rw-test-zfs2.fio readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8 writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8 fio-2.16 Starting 2 processes Jobs: 1 (f=1): [r(1),_(1)] [99.9% done] [174.2MB/0KB/0KB /s] [44.6K/0/0 iops] [eta 00m:02s] readtest: (groupid=0, jobs=1): err= 0: pid=17037: Sun May 5 06:42:41 2019 read : io=102400MB, bw=41837KB/s, iops=10459, runt=2506328msec slat (usec): min=1, max=5036, avg= 4.17, stdev= 4.40 clat (usec): min=0, max=207427, avg=759.50, stdev=2928.52 lat (usec): min=7, max=207431, avg=763.78, stdev=2928.76 clat percentiles (usec): | 1.00th=[ 13], 5.00th=[ 16], 10.00th=[ 21], 20.00th=[ 62], | 30.00th=[ 124], 40.00th=[ 197], 50.00th=[ 211], 60.00th=[ 221], | 70.00th=[ 237], 80.00th=[ 354], 90.00th=[ 2608], 95.00th=[ 3760], | 99.00th=[ 6176], 99.50th=[ 7200], 99.90th=[10432], 99.95th=[100864], | 99.99th=[103936] lat (usec) : 2=0.01%, 4=0.01%, 10=0.06%, 20=8.81%, 50=7.58% lat (usec) : 100=11.67%, 250=46.67%, 500=5.80%, 750=0.68%, 1000=0.80% lat (msec) : 2=5.04%, 4=8.67%, 10=4.09%, 20=0.05%, 50=0.01% lat (msec) : 100=0.01%, 250=0.06% cpu : usr=2.72%, sys=6.82%, ctx=18518361, majf=0, minf=16 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=26214400/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0 latency : target=0, window=0, percentile=100.00%, depth=8 writetest: (groupid=0, jobs=1): err= 0: pid=17038: Sun May 5 06:42:41 2019 write: io=102400MB, bw=47904KB/s, iops=11976, runt=2188892msec slat (usec): min=2, max=4064, avg= 4.17, stdev= 4.26 clat (usec): min=1, max=2039.4K, avg=662.80, stdev=18673.45 lat (usec): min=14, max=2039.5K, avg=667.06, stdev=18673.46 clat percentiles (usec): | 1.00th=[ 17], 5.00th=[ 21], 10.00th=[ 24], 20.00th=[ 31], | 30.00th=[ 70], 40.00th=[ 90], 50.00th=[ 149], 60.00th=[ 241], | 70.00th=[ 370], 80.00th=[ 490], 90.00th=[ 676], 95.00th=[ 836], | 99.00th=[ 1432], 99.50th=[ 2160], 99.90th=[100864], 99.95th=[103936], | 99.99th=[1056768] lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=2.85%, 50=24.13% lat (usec) : 100=15.73%, 250=18.00%, 500=19.94%, 750=12.12%, 1000=4.63% lat (msec) : 2=2.04%, 4=0.39%, 10=0.07%, 20=0.01%, 50=0.01% lat (msec) : 100=0.01%, 250=0.07%, 500=0.01%, 750=0.01%, 1000=0.01% lat (msec) : 2000=0.01%, >=2000=0.01% cpu : usr=2.25%, sys=6.79%, ctx=10904659, majf=0, minf=9 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=0/w=26214400/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0 latency : target=0, window=0, percentile=100.00%, depth=8
Run status group 0 (all jobs): READ: io=102400MB, aggrb=41837KB/s, minb=41837KB/s, maxb=41837KB/s, mint=2506328msec, maxt=2506328msec WRITE: io=102400MB, aggrb=47904KB/s, minb=47904KB/s, maxb=47904KB/s, mint=2188892msec, maxt=2188892msec fio ./rw-test-zfs2.fio 120.68s user 322.81s system 17% cpu 41:45.90 total