focal (8) wakeuptime-bpfcc.8.gz

NAME

       wakeuptime - Summarize sleep to wakeup time by waker kernel stack. Uses Linux eBPF/bcc.

SYNOPSIS

       wakeuptime [-h] [-u] [-p PID] [-v] [-f] [--stack-storage-size STACK_STORAGE_SIZE] [-m MIN_BLOCK_TIME] [-M
       MAX_BLOCK_TIME] [duration]

DESCRIPTION

       This program shows the kernel stack traces for threads that woke up other blocked threads, along with the
       process  names  of  the  waker  and target, along with a sum of the time that the target was blocked: the
       "blocked time".  It works by tracing when threads block and when they were then woken up,  and  measuring
       the  time  delta.  This  time measurement will be very similar to off-CPU time, however, off-CPU time may
       include a little extra time spent waiting on a run queue to  be  scheduled.  The  stack  traces,  process
       names, and time spent blocked is summarized in the kernel using an eBPF map for efficiency.

       The  output summary will help you identify reasons why threads were blocking by showing who woke them up,
       along with the time they were blocked. This spans all types of blocking activity: disk I/O, network  I/O,
       locks, page faults, involuntary context switches, etc.

       This  can  be  used  in  conjunction  with offcputime, which shows the stack trace of the blocked thread.
       wakeuptime shows the stack trace of the waker thread.

       See http://www.brendangregg.com/FlameGraphs/offcpuflamegraphs.html

       This tool only works on Linux 4.6+. It uses the new `BPF_STACK_TRACE` table  APIs  to  generate  the  in-
       kernel stack traces.

REQUIREMENTS

       CONFIG_BPF and bcc.

OPTIONS

       -h     Print usage message.

       -f     Print output in folded stack format.

       -u     Only trace user threads (not kernel threads).

       -v     Show raw addresses (for non-folded format).

       -p PID Trace this process ID only (filtered in-kernel).

       --stack-storage-size STACK_STORAGE_SIZE
              Change the number of unique stack traces that can be stored and displayed.

       duration
              Duration to trace, in seconds.

       -m MIN_BLOCK_TIME
              The amount of time in microseconds over which we store traces (default 1)

       -M MAX_BLOCK_TIME
              The amount of time in microseconds under which we store traces (default U64_MAX)

EXAMPLES

       Trace all thread blocking events, and summarize (in-kernel) by kernel stack trace and total blocked time:
              # wakeuptime

       Trace user-mode target threads only:
              # wakeuptime -u

       Trace for 5 seconds only:
              # wakeuptime 5

       Trace for 5 seconds, and emit output in folded stack format (suitable for flame graphs):
              # wakeuptime -f 5

       Trace PID 185 only:
              # wakeuptime -p 185

OVERHEAD

       This  summarizes  unique  stack  traces  in-kernel  for efficiency, allowing it to trace a higher rate of
       events than methods that post-process in user space. The stack trace and time data is only copied to user
       space  once,  when the output is printed. While these techniques greatly lower overhead, scheduler events
       are still a high frequency event, as they can exceed 1 million events per second, and so  caution  should
       still be used. Test before production use.

       If  the  overhead  is still a problem, take a look at the min block option.  If your aim is to chase down
       longer blocking events, then this could be increased to filter shorter blocking events, further  lowering
       overhead.

SOURCE

       This is from bcc.

              https://github.com/iovisor/bcc

       Also  look  in  the bcc distribution for a companion _examples.txt file containing example usage, output,
       and commentary for this tool.

OS

       Linux

STABILITY

       Unstable - in development.

AUTHOR

       Brendan Gregg

SEE ALSO

       offcputime(8), stackcount(8)