FreeBSD 11.3-STABLE/amd64 r354667 with five SSD da[2-6] in ZFS RAIDZ. Processes like 'zfs list -a -r -t all' periodically stall in 'D' state for long time (tens of seconds), pool has many snapshots. System runs several bhyve instances over ZVOLs. # gstat -aI3s -f 'da[2-6].*' dT: 3.000s w: 3.000s filter: da[2-6].* L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 64 85 0 0 0.0 0 0 0.0 113.2| da2 64 85 0 0 0.0 0 0 0.0 112.8| da3 64 85 0 0 0.0 0 0 0.0 112.8| da4 64 64 0 0 0.0 0 0 0.0 97.0| da5 64 85 0 0 0.0 0 0 0.0 104.7| da6 64 85 0 0 0.0 0 0 0.0 112.8| da3.eli 64 85 0 0 0.0 0 0 0.0 112.8| da4.eli 64 85 0 0 0.0 0 0 0.0 113.2| da2.eli 64 64 0 0 0.0 0 0 0.0 97.0| da5.eli 64 85 0 0 0.0 0 0 0.0 104.7| da6.eli # zpool status -v sata pool: sata state: ONLINE scan: resilvered 58,0G in 0 days 00:08:02 with 0 errors on Wed Mar 13 21:34:28 2019 config: NAME STATE READ WRITE CKSUM sata ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da2.eli ONLINE 0 0 0 da3.eli ONLINE 0 0 0 da4.eli ONLINE 0 0 0 da5.eli ONLINE 0 0 0 da6.eli ONLINE 0 0 0 logs raid/r0s3 ONLINE 0 0 0 errors: No known data errors AVAGO MegaRAID SAS FreeBSD mrsas driver version: 07.709.04.00-fbsd mrsas0: port 0x4000-0x40ff mem 0xa0c00000-0xa0c0ffff,0xa0b00000-0xa0bfffff irq 32 at device 0.0 numa-domain 0 on pci5 mrsas0: FW now in Ready state mrsas0: Using MSI-X with 32 number of vectors mrsas0: FW supports <96> MSIX vector,Online CPU 32 Current MSIX <32> mrsas0: max sge: 0x46, max chain frame size: 0x400, max fw cmd: 0x39f mrsas0: Issuing IOC INIT command to FW. mrsas0: IOC INIT response received from FW. mrsas0: System PD created target ID: 0x0 mrsas0: System PD created target ID: 0x1 mrsas0: System PD created target ID: 0x2 mrsas0: System PD created target ID: 0x3 mrsas0: System PD created target ID: 0x4 mrsas0: System PD created target ID: 0x5 mrsas0: System PD created target ID: 0x6 mrsas0: FW supports: UnevenSpanSupport=1 mrsas0: max_fw_cmds: 927 max_scsi_cmds: 911 mrsas0: MSI-x interrupts setup success mrsas0: mrsas_ocr_thread da5 at mrsas0 bus 1 scbus17 target 5 lun 0 da2 at mrsas0 bus 1 scbus17 target 2 lun 0 da3 at mrsas0 bus 1 scbus17 target 3 lun 0 da4 at mrsas0 bus 1 scbus17 target 4 lun 0 da4: Fixed Direct Access SPC-4 SCSI device da4: Serial Number 18531FF58A9B da4: 150.000MB/s transfers da4: 3662830MB (7501476528 512 byte sectors) da2: Fixed Direct Access SPC-4 SCSI device da2: Serial Number 18532039E29C da2: 150.000MB/s transfers da2: 3662830MB (7501476528 512 byte sectors) da3: Fixed Direct Access SPC-4 SCSI device da3: Serial Number 18531FF58CF6 da3: 150.000MB/s transfers da3: 3662830MB (7501476528 512 byte sectors) da5: Fixed Direct Access SPC-4 SCSI device da5: Serial Number 18531FF586C2 da5: 150.000MB/s transfers da5: 3662830MB (7501476528 512 byte sectors) da6 at mrsas0 bus 1 scbus17 target 6 lun 0 da6: Fixed Direct Access SPC-4 SCSI device da6: Serial Number HKT01DYR da6: 150.000MB/s transfers da6: 3662830MB (7501476528 512 byte sectors)