Provided by: bpfcc-tools_0.8.0-4_all bug


       ext4slower - Trace slow ext4 file operations, with per-event details.


       ext4slower [-h] [-j] [-p PID] [min_ms]


       This tool traces common ext4 file operations: reads, writes, opens, and syncs. It measures
       the time spent in these operations, and prints details for each that exceeded a threshold.

       WARNING: See the OVERHEAD section.

       By default, a minimum millisecond threshold of 10 is used. If a threshold of  0  is  used,
       all events are printed (warning: verbose).

       Since  this  works  by  tracing the ext4_file_operations interface functions, it will need
       updating to match any changes to these functions.

       Since this uses BPF, only the root user can use this tool.


       CONFIG_BPF and bcc.


       -p PID Trace this PID only.

       min_ms Minimum I/O latency (duration) to trace, in milliseconds. Default is 10 ms.


       Trace synchronous file reads and writes slower than 10 ms:
              # ext4slower

       Trace slower than 1 ms:
              # ext4slower 1

       Trace slower than 1 ms, and output just the fields in parsable format (csv):
              # ext4slower -j 1

       Trace all file reads and writes (warning: the output will be verbose):
              # ext4slower 0

       Trace slower than 1 ms, for PID 181 only:
              # ext4slower -p 181 1


              Time of I/O completion since the first I/O seen, in seconds.

       COMM   Process name.

       PID    Process ID.

       T      Type of operation. R == read, W == write, O == open, S == fsync.

       OFF_KB File offset for the I/O, in Kbytes.

       BYTES  Size of I/O, in bytes.

              Latency (duration) of I/O,  measured  from  when  it  was  issued  by  VFS  to  the
              filesystem,  to when it completed. This time is inclusive of block device I/O, file
              system CPU cycles, file system locks, run queue latency, etc. It's a more  accurate
              measure of the latency suffered by applications performing file system I/O, than to
              measure this down at the block device interface.

              A cached kernel file name (comes from dentry->d_iname).

              Completion timestamp, microseconds (-j only).

              File offset, bytes (-j only).

              Latency (duration) of the I/O, in microseconds (-j only).


       This adds low-overhead instrumentation to  these  ext4  operations,  including  reads  and
       writes  from  the file system cache. Such reads and writes can be very frequent (depending
       on the workload; eg, 1M/sec), at which point the overhead of this tool (even if it  prints
       no  "slower"  events) can begin to become significant. Measure and quantify before use. If
       this continues to be a problem,  consider  switching  to  a  tool  that  prints  in-kernel
       summaries only.

       Note  that  the  overhead  of  this  tool  should be less than fileslower(8), as this tool
       targets ext4 functions only, and not all file read/write paths (which can  include  socket


       This is from bcc.


       Also  look  in  the bcc distribution for a companion _examples.txt file containing example
       usage, output, and commentary for this tool.




       Unstable - in development.


       Brendan Gregg


       biosnoop(8), funccount(8), fileslower(8)