NAME

       gpu-mon - continuously update a table with the state of all compatible GPUs

SYNOPSIS

       gpu-mon [--help | --about]
       gpu-mon [--gui] [--no_fan] [--plot] [--ltz] [--sleep N] [--debug] [--pdebug]

DESCRIPTION

       gpu-mon  displays  the  current  state  of all compatible GPUs. The default behavior is to
       continuously update a text based table in the current  window  until  Ctrl-C  is  pressed.
       With  the  --gui  option,  a table of relevant parameters will be updated in a Gtk window.
       You can specify the delay between updates with the --sleep N option where N is an  integer
       > zero that specifies the number of seconds to sleep between updates.  The --no_fan option
       can be used to disable the reading and display of fan information.  The  --log  option  is
       used to write all monitor data to a psv log file.  When writing to a log file, the utility
       will indicate this in red at the top of the window with a message that  includes  the  log
       file  name. The --plot will display a plot of critical GPU parameters which updates at the
       specified --sleep N interval. If you need both the plot and monitor displays,  then  using
       the  --plot  option  is  preferred over running both tools as a single read of the GPUs is
       used to update both displays.  The --ltz option results in the use of local  time  instead
       of UTC.

OPTIONS

        --about
              Will display details about gpu-mon.

        --gui The  table of relevant parameters will be updated in a Gtk window instead of a text
              table in the terminal window.

        --log Write all mon data to a logfile.  The real-time display will indicate that  logging
              is enabled and will show the filename used.

        --ltz Use local time zone instead of UTC for displays and logging.

        --no_fan
              Will exclude fan information from the display.  Useful with water cooled GPUs.

        --pdebug
              Will enable debug output for the gpu-plot plotting utility.

        --plot
              Open and write to, gpu-plot, the gpu-util plotting utility.

        --sleep N
              Specifies N, the number of seconds to sleep between updates.

        --debug
              Will output additional useful debug/troubleshooting details to a log file.

       -h, --help
              Display help text and exit.

EXAMPLES

       gpu-mon --sleep 5 --log

       Will  display  a  continuously updating table of GPU operating parameters updating with an
       interval of 5 sec. All parameters will be written to a logfile which will be indicated  in
       the  table.   The  displayed parameters include GPU model, GPU and memory load percentage,
       power, power cap, energy consumption, temperature, voltage (not  available  for  NV),  fan
       speed,  Sclk  frequency/p-state,  Mclk frequency/pstate, and performance mode. Updating of
       the table will continue until ctrl-c is pressed.

       gpu-mon --gui

       Will open a new Gtk window and display basic parameters updated with the default interval.

       gpu-mon --plot

       Will open 2 new Gtk windows. One will display the basic parameters  and  the  second  will
       display  a  continuously  updating  plot  of  these parameters.  It is suggested that this
       method be used if both displays are desired, instead of executing both  gpu-mon  and  gpu-
       plot as the later will result in twice the reads of GPU data.

CONFIGURATION

       In order to get maximum capability of these utilities, you should be running with a kernel
       that provides support of the GPUs you have installed.  If using AMD GPUs,  installing  the
       latest  amdgpu  driver  package  or  the  latest  ROCm  release,  may  provide  additional
       capabilities. If you have Nvidia GPUs installed, nvidia-smi  must  also  be  installed  in
       order  for  the utility reading of the cards to be possible.  Writing to GPUs is currently
       only possible for AMD  GPUs,  and  only  with  compatible  cards  and  with  the  the  AMD
       ppfeaturemask    set    to    0xfffd7fff.    This    can   be   accomplished   by   adding
       amdgpu.ppfeaturemask=0xfffd7fff    to    the    GRUB_CMDLINE_LINUX_DEFAULT    value     in
       /etc/default/grub and executing sudo update-grub.

FILES

       /usr/share/misc/pci.ids
              The  system  list of all known PCI ID's (vendors, devices, classes and subclasses).
              It can be updated with the update-pciids command.

       /sys/class/drm/card*/device/pp_od_clk_voltage
              Special driver file for each AMD GPU required by some rickslab-gpu-utils.

       /etc/default/grub
              The grub defaults file where amdgpu.ppfeaturemask needs to be set.

BUGS

       Known to not work well with Fiji ProDuo cards and will issue  warning  messages  for  Fiji
       Nano cards.  Please report any additional bugs/issues at https://github.com/Ricks-Lab/gpu-
       utils

SEE ALSO

       gpu-plot(1), amdgpu(4), nvidia-smi(1), update-grub(8), update-pciids(8), lspci(8)

AVAILABILITY

       The gpu-mon command is part of  the  rickslab-gpu-utils  package  and  is  available  from
       https://github.com/Ricks-Lab/gpu-utils