Provided by: crash_8.0.0-1ubuntu1_amd64 bug


       crash - Analyze Linux crash dump data or a live system


       crash [OPTION]... NAMELIST MEMORY-IMAGE[@ADDRESS]    (dumpfile form)
       crash [OPTION]... [NAMELIST]                         (live system form)


       Crash  is  a  tool  for  interactively analyzing the state of the Linux system while it is
       running, or after a kernel crash has occurred and a core dump  has  been  created  by  the
       netdump, diskdump, LKCD, kdump, xendump kvmdump or VMware facilities.  It is loosely based
       on the SVR4 UNIX crash command, but has been significantly enhanced by completely  merging
       it  with  the  gdb(1)  debugger.  The marriage of the two effectively combines the kernel-
       specific nature of the traditional UNIX crash utility with the source code level debugging
       capabilities of gdb(1).

       In the dumpfile form, both a NAMELIST and a MEMORY-IMAGE argument must be entered.  In the
       live system form, the NAMELIST argument must be entered if the kernel's  vmlinux  file  is
       not  located  in a known location, such as the /usr/lib/debug/lib/modules/<kernel-version>

       The crash utility has also been extended to support the analysis of dumpfiles generated by
       a  crash  of  the Xen hypervisor.  In that case, the NAMELIST argument must be that of the
       xen-syms binary.  Live system analysis is not supported for the Xen hypervisor.

       The crash utility command set consists of common kernel core analysis tools such as kernel
       stack  back  traces  of all processes, source code disassembly, formatted kernel structure
       and variable displays, virtual memory  data,  dumps  of  linked-lists,  etc.,  along  with
       several  commands  that  delve  deeper  into  specific kernel subsystems.  Appropriate gdb
       commands may also be entered, which in turn are passed on to the gdb module for execution.
       If  desired,  commands  may be placed in either a $HOME/.crashrc file and/or in a .crashrc
       file in the current directory.  During initialization, the commands in $HOME/.crashrc  are
       executed first, followed by those in the ./.crashrc file.

       The  crash  utility  is designed to be independent of Linux version dependencies. When new
       kernel source code impacts the correct functionality of crash and  its  command  set,  the
       utility  will be updated to recognize new kernel code changes, while maintaining backwards
       compatibility with earlier releases.


              This is a pathname to an uncompressed kernel image  (a  vmlinux  file),  or  a  Xen
              hypervisor  image  (a  xen-syms file) which has been compiled with the "-g" option.
              If using the dumpfile form, a vmlinux file may be  compressed  in  either  gzip  or
              bzip2 formats.

              A  kernel  core  dump  file  created  by the netdump, diskdump, LKCD kdump, xendump
              kvmdump or VMware facilities.

              If a MEMORY-IMAGE argument is not entered, the session will be invoked on the  live
              system, which typically requires root privileges because of the device file used to
              access system RAM.  By default, /dev/crash will be used if it exists.  If  it  does
              not  exist,  then /dev/mem will be used; but if the kernel has been configured with
              CONFIG_STRICT_DEVMEM,  then  /proc/kcore  will  be  used.   It  is  permissible  to
              explicitly enter /dev/crash, /dev/mem or /proc/kcore.

              An @ADDRESS value must be appended to the MEMORY-IMAGE if the dumpfile is a raw RAM
              dumpfile that has no header information describing  the  file  contents.   Multiple
              MEMORY-IMAGE@ADDRESS  ordered pairs may be entered, with each dumpfile containing a
              contiguous block of RAM, where the ADDRESS value is the physical start  address  of
              the  block expressed in hexadecimal.  The physical address value(s) will be used to
              create a temporary ELF header in /var/tmp, which will only exist during  the  crash
              session.   If  a  raw  RAM  dumpile  represents  a live memory source, such as that
              specified by the QEMU mem-path  argument  of  a  memory-backend-file  object,  then
              "live:" must be prepended to the MEMORY-IMAGE name.

              As  VMware  facility,  the  crash  utility is able to process VMware VM memory dump
              generated by VM suspend or guest core dump. In that  case,  .vmss  or  .guest  file
              should be used as a MEMORY-IMAGE and .vmem file must be located in the same folder.

              If  the NAMELIST file is not the same kernel that is running (live system form), or
              the kernel that was running when the  system  crashed  (dumpfile  form),  then  the
     file of the original kernel should be entered on the command line.

       -h [option]
       --help [option]
              Without  an  option  argument,  display  a crash usage help message.  If the option
              argument is a crash command name, the help page for that command is displayed.   If
              it  is  the  string "input", a page describing the various crash command line input
              options is displayed.  If it is the string "output", a page describing command line
              output  options  is displayed.  If it is the string "all", then all of the possible
              help messages are displayed.  After the help message is displayed, crash exits.

       -s     Silently proceed directly to the "crash>" prompt without  displaying  any  version,
              GPL,  or  crash initialization data during startup, and by default, runtime command
              output is not passed to any scrolling command.

       -i file
              Execute the command(s) contained in file prior to displaying  the  "crash>"  prompt
              for interactive user input.

       -d num Set  the internal debug level.  The higher the number, the more debugging data will
              be printed when crash initializes and runs.

       -S     Use /boot/ as the mapfile.

       -e vi | emacs
              Set the readline(3) command line editing mode to  "vi"  or  "emacs".   The  default
              editing mode is "vi".

       -f     Force  the  usage  of a compressed vmlinux file if its original name does not start
              with "vmlinux".

       -k     Indicate that the NAMELIST file is an LKCD "Kerntypes" debuginfo file.

       -g [namelist]
              Determine if a vmlinux or xen-syms namelist file contains debugging data.

       -t     Display the system-crash timestamp and exit.

       -L     Attempt  to  lock  all  of  its  virtual  address  space  into  memory  by  calling
              mlockall(MCL_CURRENT|MCL_FUTURE)  during initialization.  If the system call fails,
              an error message will be displayed, but the session continues.

       -c tty-device
              Open the tty-device as the console used for debug messages.

       -p page-size
              If a processor's page size cannot be determined by the dumpfile, and the  processor
              default cannot be used, use page-size.

       -o filename
              Only  used  with the MEMORY-IMAGE@ADDRESS format for raw RAM dumpfiles, specifies a
              filename of a new ELF vmcore that will be created and used  as  the  dumpfile.   It
              will  be  saved  to allow future use as a standalone vmcore, replacing the original
              raw RAM dumpfile.

       -m option=value
       --machdep option=value
              Pass an option and value  pair  to  machine-dependent  code.   These  architecture-
              specific option/pairs should only be required in very rare circumstances:

                vm=orig       (pre-2.6.11 virtual memory address ranges)
                vm=2.6.11     (2.6.11 and later virtual memory address ranges)
                vm=xen        (Xen kernel virtual memory address ranges)
                vm=xen-rhel4  (RHEL4 Xen kernel virtual address ranges)
                vm=5level     (5-level page tables)
                vm=2.6.14     (4-level page tables)
                vm=4l         (4-level page tables)

       -x     Automatically  load  extension modules from a particular directory.  If a directory
              is  specified  in  the  CRASH_EXTENSIONS  shell  environment  variable,  then  that
              directory    will   be   used.    Otherwise   /usr/lib64/crash/extensions   (64-bit
              architectures) or /usr/lib/crash/extensions (32-bit architectures) will be used; if
              they do not exist, then the ./extensions directory will be used.

              Track only the active task on each cpu.

              Display  the crash binary's build date, the user ID of the builder, the hostname of
              the machine where the build was done, the target architecture, the version  number,
              and the compiler version.

       --memory_module modname
              Use the modname as an alternative kernel module to the crash.ko module that creates
              the /dev/crash device.

       --memory_device device
              Use device as an alternative device to  the  /dev/crash,  /dev/mem  or  /proc/kcore

       --log dumpfile
              Dump  the  contents  of  the  kernel log buffer.  A kernel namelist argument is not
              necessary, but the dumpfile  must  contain  the  VMCOREINFO  data  taken  from  the
              original /proc/vmcore ELF header.

              Do  not  use  kallsyms-generated  symbol information contained within kernel module
              object files.

              Do not access or display any kernel module related information.

              Do not attempt to read configuration data that was built  into  kernels  configured
              with CONFIG_IKCONFIG.

              Do not verify the validity of all structure member offsets and structure sizes that
              it uses.

              Do not initialize the kernel's slab cache infrastructure,  and  commands  that  use
              kmem_cache-related data will not work.

              Do not use the registers from the ELF NT_PRSTATUS notes saved in a compressed kdump
              header for backtraces.

              Delay the initialization of the kernel's slab  cache  infrastructure  until  it  is
              required by a run-time command.

              Pass  this  flag  to  the  embedded  gdb  module, which will override its two-stage
              strategy that it uses for reading symbol tables from the NAMELIST.

       --smp  Specify that the system being analyzed is an SMP kernel.

              Display the version of the crash utility, the version of the embedded  gdb  module,
              GPL information, and copyright notices.

       --cpus number
              Specify the number of cpus in the SMP system being analyzed.

       --osrelease dumpfile
              Display the OSRELEASE vmcoreinfo string from a kdump dumpfile header.

              Force the session to be that of a Xen hypervisor.

       --p2m_mfn pfn
              When  a  Xen  Hypervisor  or  its  dom0  kernel  crashes, the dumpfile is typically
              analyzed with either the Xen hypervisor or the dom0 kernel.  It is also possible to
              analyze  any of the guest domU kernels if the pfn_to_mfn_list_list pfn value of the
              guest kernel is passed on  the  command  line  along  with  its  NAMELIST  and  the

       --xen_phys_start physical-address
              Supply  the  base physical address of the Xen hypervisor's text and static data for
              older xendump dumpfiles that did not pass that information in the dumpfile header.

              If the makedumpfile(8) facility has filtered a compressed kdump dumpfile to exclude
              various  types  of  non-essential  pages,  or  has marked a compressed or ELF kdump
              dumpfile as incomplete due to an ENOSPC or other error  during  its  creation,  any
              attempt  to  read missing pages will fail.  With this flag, reads from any of those
              pages will return zero-filled memory.

              Do not attempt to find the task that was running when the kernel crashed.  Set  the
              initial context to that of the "swapper" task on cpu 0.

       --more Use   /bin/more   as  the  command  output  scroller,  overriding  the  default  of
              /usr/bin/less and any settings in either ./.crashrc or $HOME/.crashrc.

       --less Use /usr/bin/less as the command output scroller, overriding any settings in either
              ./.crashrc or $HOME/.crashrc.

       --hex  Set the default command output radix to 16, overriding the default radix of 10, and
              any radix settings in either ./.crashrc or $HOME/.crashrc.

       --dec  Set the default command output radix to 10, overriding any radix settings in either
              ./.crashrc or $HOME/.crashrc. This is the default radix setting.

              Use the output paging command defined in the CRASHPAGER shell environment variable,
              overriding any settings in either ./.crashrc or $HOME/.crashrc.

              Do not pass run-time command output to any scrolling command.

              Do not strip cloned kernel text symbol names.

              Do not execute the commands in either $HOME/.crashrc or ./.crashrc.

       --mod directory
              When loading the debuginfo data of kernel modules with the mod -S  command,  search
              for their object files in directory instead of in the standard location.

       --kaslr offset|auto
              If  an x86_64 kernel was configured with CONFIG_RANDOMIZE_BASE, the offset value is
              equal to the difference between the symbol values compiled into  the  vmlinux  file
              and  their  relocated KASLR values.  If set to auto, the KASLR offset value will be
              automatically calculated.

       --reloc size
              When analyzing live x86 kernels that were configured with  a  CONFIG_PHYSICAL_START
              value  that  is  larger  than  its  CONFIG_PHYSICAL_ALIGN  value,  then  it will be
              necessary to enter a relocation size  equal  to  the  difference  between  the  two

       --hash count
              Set  the  number  of  internal  hash  queue  heads  used  for  list  gathering  and
              verification.  The default count is 32768.

              Bring up a session that is restricted to the log, dis, rd, sym, eval, set and  exit
              commands.   This option may provide a way to extract some minimal/quick information
              from a corrupted or truncated dumpfile, or in situations where one of  the  several
              kernel subsystem initialization routines would abort the crash session.

       --kvmhost [32|64]
              When  examining  an x86 KVM guest dumpfile, this option specifies that the KVM host
              that created the dumpfile was an  x86  (32-bit)  or  an  x86_64  (64-bit)  machine,
              overriding the automatically determined value.

       --kvmio <size>
              override the automatically-calculated KVM guest I/O hole size.

       --offline [show|hide]
              Show  or  hide command output that is related to offline cpus.  The default setting
              is show.


       Each crash command generally falls into one of the following categories:

       Symbolic display
              Displays of kernel text/data, which take full advantage of  the  power  of  gdb  to
              format and display data structures symbolically.

       System state
              The  majority  of crash commands consist of a set of "kernel-aware" commands, which
              delve into various kernel subsystems on a system-wide or per-task basis.

       Utility functions
              A set of useful helper commands serving various purposes, some simple, others quite

       Session control
              Commands that control the crash session itself.

       The  following alphabetical list consists of a very simple overview of each crash command.
       However, since individual commands often have several options resulting  in  significantly
       different  output,  it is suggested that the full description of each command be viewed by
       executing crash -h <command>, or during a crash session by simply entering help command.

       *      "pointer to" is shorthand for either the struct or union commands.  It displays the
              contents of a kernel structure or union.

       alias  creates a single-word alias for a command.

       ascii  displays an ascii chart or translates a numeric value into its ascii components.

       bt     displays  a  task's  kernel-stack  backtrace.   If  it  is  given the -a option, it
              displays the stack traces of the active tasks on all CPUs.  It is often  used  with
              the foreach command to display the backtraces of all tasks with one command.

       btop   translates a byte value (physical offset) to its page number.

       dev    displays  data  concerning  the  character  and  block device assignments, I/O port
              usage, I/O memory usage, and PCI device data.

       dis    disassembles memory,  either  entire  kernel  functions,  from  a  location  for  a
              specified number of instructions, or from the start of a function up to a specified
              memory location.

       eval   evaluates an expression or numeric type and displays  the  result  in  hexadecimal,
              decimal, octal and binary.

       exit   causes crash to exit.

       extend dynamically loads or unloads crash shared object extension modules.

       files  displays information about open files in a context.

              repeats a specified command for the specified (or all) tasks in the system.

       fuser  displays the tasks using the specified file or socket.

       gdb    passes  its  argument  to  the embedded gdb module.  It is useful for executing gdb
              commands that have the same name as crash commands.

       help   alone displays the command menu; if followed by a command name, a full  description
              of  a  command,  its  options,  and examples are displayed.  Its output is far more
              complete and useful than this man page.

       ipcs   displays data about the System V IPC facilities.

       irq    displays data  concerning  interrupt  request  numbers  and  bottom-half  interrupt

       kmem   displays information about the use of kernel memory.

       list   displays the contents of a linked list.

       log    displays the kernel log_buf contents in chronological order.

       mach   displays data specific to the machine type.

       mod    displays  information  about  the  currently  installed  kernel modules, or adds or
              deletes symbolic or debugging information about specified kernel modules.

       mount  displays information about the currently-mounted filesystems.

       net    display various network related data.

       p      passes its arguments to the gdb "print" command for evaluation and display.

       ps     displays process status for specified, or all, processes in the system.

       pte    translates the hexadecimal contents of a PTE into its  physical  page  address  and
              page bit settings.

       ptob   translates a page frame number to its byte value.

       ptov   translates a hexadecimal physical address into a kernel virtual address.

       q      is an alias for the "exit" command.

       rd     displays  the  contents  of  memory, with the output formatted in several different

       repeat repeats a command indefinitely, optionally  delaying  a  given  number  of  seconds
              between each command execution.

       runq   displays the tasks on the run queue.

       search searches a range of user or kernel memory space for given value.

       set    either sets a new context, or gets the current context for display.

       sig    displays signal-handling data of one or more tasks.

       struct displays  either  a structure definition or the contents of a kernel structure at a
              specified address.

       swap   displays information about each configured swap device.

       sym    translates a symbol to its virtual address, or a static kernel virtual  address  to
              its symbol -- or to a symbol-plus-offset value, if appropriate.

       sys    displays system-specific data.

       task   displays the contents of a task_struct.

       tree   displays the contents of a red-black tree or a radix tree.

       timer  displays the timer queue entries, both old- and new-style, in chronological order.

       union  is similar to the struct command, except that it works on kernel unions.

       vm     displays basic virtual memory information of a context.

       vtop   translates a user or kernel virtual address to its physical address.

       waitq  walks  the  wait queue list displaying the tasks which are blocked on the specified
              wait queue.

       whatis displays the definition of structures, unions, typedefs or text/data symbols.

       wr     modifies the contents of memory on a live system.  It can only be used if  /dev/mem
              is  the  device  file being used to access system RAM, and should obviously be used
              with great care.

       When crash is invoked with a Xen hypervisor binary as the NAMELIST,  the  command  set  is
       slightly  modified.   The  *,  alias, ascii, bt, dis, eval, exit, extend, gdb, help, list,
       log, p, pte, rd, repeat, search, set, struct, sym, sys, union, whatis, wr and  q  commands
       are the same as above.  The following commands are specific to the Xen hypervisor:

       domain displays the contents of the domain structure for selected, or all, domains.

       doms   displays domain status for selected, or all, domains.

              displays Xen dump information for selected, or all, cpus.

       pcpus  displays physical cpu information for selected, or all, cpus.

       vcpus  displays vcpu status for selected, or all, vcpus.


              Initialization  commands.   The  file  can  be located in the user's HOME directory
              and/or the current directory.  Commands found in the  .crashrc  file  in  the  HOME
              directory are executed before those in the current directory's .crashrc file.


       EDITOR Command  input  is  read  using  readline(3).  If EDITOR is set to emacs or vi then
              suitable keybindings are used.  If EDITOR is not set, then vi is used.  This can be
              overridden  by  set  vi  or  set  emacs  commands located in a .crashrc file, or by
              entering -e emacs on the crash command line.

              If CRASHPAGER is set, its value is used as the name of the program to which command
              output will be sent.  If not, then command output is sent to /usr/bin/less -E -X by

              Specifies an alternative directory tree to search for kernel module object files.

              Specifies  a  directory  containing  extension  modules   that   will   be   loaded
              automatically if the -x command line option is used.


       If  crash does not work, look for a newer version: kernel evolution frequently makes crash
       updates necessary.

       The command set scroll off will cause output to be sent directly to  the  terminal  rather
       than through a paging program.  This is useful, for example, if you are running crash in a
       window of emacs.


       Dave Anderson <> wrote crash.

       Jay Fenlason <> and Dave Anderson <> wrote this  man


       The  help command within crash provides more complete and accurate documentation than this
       man page. - the home page of the crash utility.

       netdump(8), gdb(1), makedumpfile(8)