Provided by: crash_7.1.4-1ubuntu4_i386 bug


       crash - Analyze Linux crash dump data or a live system


       crash [OPTION]... NAMELIST MEMORY-IMAGE[@ADDRESS]    (dumpfile form)
       crash [OPTION]... [NAMELIST]                         (live system form)


       Crash  is  a  tool  for  interactively analyzing the state of the Linux
       system while it is running, or after a kernel crash has occurred and  a
       core  dump  has  been  created  by  the netdump, diskdump, LKCD, kdump,
       xendump or kvmdump facilities.  It is loosely based on  the  SVR4  UNIX
       crash  command,  but  has  been  significantly  enhanced  by completely
       merging  it  with  the  gdb(1)  debugger.  The  marriage  of  the   two
       effectively combines the kernel-specific nature of the traditional UNIX
       crash utility with the source  code  level  debugging  capabilities  of

       In  the dumpfile form, both a NAMELIST and a MEMORY-IMAGE argument must
       be entered.  In the live system form, the  NAMELIST  argument  must  be
       entered  if  the  kernel's  vmlinux  file  is  not  located  in a known
       location,  such  as   the   /usr/lib/debug/lib/modules/<kernel-version>

       The  crash  utility  has  also been extended to support the analysis of
       dumpfiles generated by a crash of the Xen hypervisor.   In  that  case,
       the NAMELIST argument must be that of the xen-syms binary.  Live system
       analysis is not supported for the Xen hypervisor.

       The crash utility command set consists of common kernel  core  analysis
       tools  such  as  kernel stack back traces of all processes, source code
       disassembly, formatted kernel structure and variable displays,  virtual
       memory  data,  dumps of linked-lists, etc., along with several commands
       that delve deeper into specific  kernel  subsystems.   Appropriate  gdb
       commands  may  also  be entered, which in turn are passed on to the gdb
       module for execution.  If desired, commands may be placed in  either  a
       $HOME/.crashrc file and/or in a .crashrc file in the current directory.
       During initialization, the  commands  in  $HOME/.crashrc  are  executed
       first, followed by those in the ./.crashrc file.

       The  crash  utility  is  designed  to  be  independent of Linux version
       dependencies.  When  new  kernel  source  code  impacts   the   correct
       functionality of crash and its command set, the utility will be updated
       to recognize new  kernel  code  changes,  while  maintaining  backwards
       compatibility with earlier releases.


              This  is  a  pathname to an uncompressed kernel image (a vmlinux
              file), or a Xen hypervisor image (a  xen-syms  file)  which  has
              been compiled with the "-g" option.  If using the dumpfile form,
              a vmlinux file  may  be  compressed  in  either  gzip  or  bzip2

              A  kernel  core dump file created by the netdump, diskdump, LKCD
              kdump, xendump or kvmdump facilities.

              If a MEMORY-IMAGE argument is not entered, the session  will  be
              invoked  on  the  live  system,  which  typically  requires root
              privileges because of the device file used to access system RAM.
              By  default,  /dev/crash  will be used if it exists.  If it does
              not exist, then /dev/mem will be used; but  if  the  kernel  has
              been configured with CONFIG_STRICT_DEVMEM, then /proc/kcore will
              be used.  It is  permissible  to  explicitly  enter  /dev/crash,
              /dev/mem or /proc/kcore.

              An  @ADDRESS  value  must be appended to the MEMORY-IMAGE if the
              dumpfile is a raw RAM dumpfile that has  no  header  information
              describing  the  file  contents.   Multiple MEMORY-IMAGE@ADDRESS
              ordered pairs may be entered, with each  dumpfile  containing  a
              contiguous block of RAM, where the ADDRESS value is the physical
              start address  of  the  block  expressed  in  hexadecimal.   The
              physical address value(s) will be used to create a temporary ELF
              header in /var/tmp, which  will  only  exist  during  the  crash

              If  the  NAMELIST  file  is  not the same kernel that is running
              (live system form), or the kernel  that  was  running  when  the
              system  crashed (dumpfile form), then the file of the
              original kernel should be entered on the command line.

       -h [option]
       --help [option]
              Without an option argument, display a crash usage help  message.
              If  the  option  argument is a crash command name, the help page
              for that command is displayed.  If it is the string  "input",  a
              page  describing the various crash command line input options is
              displayed.  If it is the  string  "output",  a  page  describing
              command  line  output options is displayed.  If it is the string
              "all", then all of the possible  help  messages  are  displayed.
              After the help message is displayed, crash exits.

       -s     Silently   proceed  directly  to  the  "crash>"  prompt  without
              displaying any version, GPL, or crash initialization data during
              startup, and by default, runtime command output is not passed to
              any scrolling command.

       -i file
              Execute the command(s) contained in file prior to displaying the
              "crash>" prompt for interactive user input.

       -d num Set  the  internal debug level.  The higher the number, the more
              debugging data will be printed when crash initializes and runs.

       -S     Use /boot/ as the mapfile.

       -e vi | emacs
              Set the  readline(3)  command  line  editing  mode  to  "vi"  or
              "emacs".  The default editing mode is "vi".

       -f     Force  the  usage  of  a compressed vmlinux file if its original
              name does not start with "vmlinux".

       -k     Indicate that the NAMELIST file is an LKCD "Kerntypes" debuginfo

       -g [namelist]
              Determine  if  a  vmlinux  or  xen-syms  namelist  file contains
              debugging data.

       -t     Display the system-crash timestamp and exit.

       -L     Attempt to lock all of its virtual address space into memory  by
              calling  mlockall(MCL_CURRENT|MCL_FUTURE) during initialization.
              If the system call fails, an error message  will  be  displayed,
              but the session continues.

       -c tty-device
              Open the tty-device as the console used for debug messages.

       -p page-size
              If a processor's page size cannot be determined by the dumpfile,
              and the processor default cannot be used, use page-size.

       -o filename
              Only used with  the  MEMORY-IMAGE@ADDRESS  format  for  raw  RAM
              dumpfiles, specifies a filename of a new ELF vmcore that will be
              created and used as the dumpfile.  It will  be  saved  to  allow
              future  use  as  a standalone vmcore, replacing the original raw
              RAM dumpfile.

       -m option=value
       --machdep option=value
              Pass an option and value pair to machine-dependent code.   These
              architecture-specific  option/pairs  should  only be required in
              very rare circumstances:

                vm=orig       (pre-2.6.11 virtual memory address ranges)
                vm=2.6.11     (2.6.11 and later virtual memory address ranges)
                vm=xen        (Xen kernel virtual memory address ranges)
                vm=xen-rhel4  (RHEL4 Xen kernel virtual address ranges)
                vm=2.6.14     (4-level page tables)
                vm=4l         (4-level page tables)

       -x     Automatically  load  extension   modules   from   a   particular
              directory.   If a directory is specified in the CRASH_EXTENSIONS
              shell environment variable, then that directory  will  be  used.
              Otherwise  /usr/lib64/crash/extensions (64-bit architectures) or
              /usr/lib/crash/extensions (32-bit architectures) will  be  used;
              if  they  do  not exist, then the ./extensions directory will be

              Track only the active task on each cpu.

              Display the crash binary's  build  date,  the  user  ID  of  the
              builder,  the  hostname of the machine where the build was done,
              the target architecture, the version number,  and  the  compiler

       --memory_module modname
              Use  the modname as an alternative kernel module to the crash.ko
              module that creates the /dev/crash device.

       --memory_device device
              Use device as an alternative device to the /dev/crash,  /dev/mem
              or /proc/kcore devices.

       --log dumpfile
              Dump  the  contents of the kernel log buffer.  A kernel namelist
              argument is not necessary, but the  dumpfile  must  contain  the
              VMCOREINFO data taken from the original /proc/vmcore ELF header.

              Do  not  use  kallsyms-generated  symbol  information  contained
              within kernel module object files.

              Do not access or display any kernel module related information.

              Do not attempt to read configuration data that  was  built  into
              kernels configured with CONFIG_IKCONFIG.

              Do  not  verify the validity of all structure member offsets and
              structure sizes that it uses.

              Do not initialize the kernel's slab  cache  infrastructure,  and
              commands that use kmem_cache-related data will not work.

              Do not use the registers from the ELF NT_PRSTATUS notes saved in
              a compressed kdump header for backtraces.

              Delay  the   initialization   of   the   kernel's   slab   cache
              infrastructure until it is required by a run-time command.

              Pass  this  flag to the embedded gdb module, which will override
              its two-stage strategy that it uses for  reading  symbol  tables
              from the NAMELIST.

       --smp  Specify that the system being analyzed is an SMP kernel.

              Display  the  version  of  the crash utility, the version of the
              embedded gdb module, GPL information, and copyright notices.

       --cpus number
              Specify the number of cpus in the SMP system being analyzed.

       --osrelease dumpfile
              Display the OSRELEASE vmcoreinfo string from  a  kdump  dumpfile

              Force the session to be that of a Xen hypervisor.

       --p2m_mfn pfn
              When  a  Xen Hypervisor or its dom0 kernel crashes, the dumpfile
              is typically analyzed with either the Xen hypervisor or the dom0
              kernel.   It  is  also possible to analyze any of the guest domU
              kernels if the  pfn_to_mfn_list_list  pfn  value  of  the  guest
              kernel is passed on the command line along with its NAMELIST and
              the dumpfile.

       --xen_phys_start physical-address
              Supply the base physical address of the  Xen  hypervisor's  text
              and  static  data  for older xendump dumpfiles that did not pass
              that information in the dumpfile header.

              If the makedumpfile(8) facility has filtered a compressed  kdump
              dumpfile to exclude various types of non-essential pages, or has
              marked a compressed or ELF kdump dumpfile as incomplete  due  to
              an  ENOSPC  or  other  error during its creation, any attempt to
              read missing pages will fail.  With this flag, reads from any of
              those pages will return zero-filled memory.

              Do not attempt to find the task that was running when the kernel
              crashed.  Set the initial context to that of the "swapper"  task
              on cpu 0.

       --more Use  /bin/more  as  the  command output scroller, overriding the
              default of /usr/bin/less and any settings in  either  ./.crashrc
              or $HOME/.crashrc.

       --less Use /usr/bin/less as the command output scroller, overriding any
              settings in either ./.crashrc or $HOME/.crashrc.

       --hex  Set the default command  output  radix  to  16,  overriding  the
              default radix of 10, and any radix settings in either ./.crashrc
              or $HOME/.crashrc.

       --dec  Set the default command output radix to 10, overriding any radix
              settings  in  either  ./.crashrc  or $HOME/.crashrc. This is the
              default radix setting.

              Use the output paging command defined in  the  CRASHPAGER  shell
              environment   variable,   overriding   any  settings  in  either
              ./.crashrc or $HOME/.crashrc.

              Do not pass run-time command output to any scrolling command.

              Do not strip cloned kernel text symbol names.

              Do  not  execute  the  commands  in  either  $HOME/.crashrc   or

       --mod directory
              When  loading  the debuginfo data of kernel modules with the mod
              -S command, search for their object files in  directory  instead
              of in the standard location.

       --kaslr offset|auto
              If  an  x86_64 kernel was configured with CONFIG_RANDOMIZE_BASE,
              the offset value is equal to the difference between  the  symbol
              values  compiled into the vmlinux file and their relocated KASLR
              values.  If  set  to  auto,  the  KASLR  offset  value  will  be
              automatically calculated.

       --reloc size
              When  analyzing  live  x86  kernels  that were configured with a
              CONFIG_PHYSICAL_START   value   that   is   larger   than    its
              CONFIG_PHYSICAL_ALIGN  value, then it will be necessary to enter
              a relocation size  equal  to  the  difference  between  the  two

       --hash count
              Set  the  number  of  internal  hash  queue  heads used for list
              gathering and verification.  The default count is 32768.

              Bring up a session that is restricted to the log, dis, rd,  sym,
              eval,  set  and exit commands.  This option may provide a way to
              extract some  minimal/quick  information  from  a  corrupted  or
              truncated  dumpfile,  or  in situations where one of the several
              kernel subsystem initialization routines would abort  the  crash

       --kvmhost [32|64]
              When  examining an x86 KVM guest dumpfile, this option specifies
              that the KVM host that created the dumpfile was an x86  (32-bit)
              or  an  x86_64  (64-bit)  machine,  overriding the automatically
              determined value.

       --kvmio <size>
              override the automatically-calculated KVM guest I/O hole size.

       --offline [show|hide]
              Show or hide command output that is  related  to  offline  cpus.
              The default setting is show.


       Each   crash   command  generally  falls  into  one  of  the  following

       Symbolic display
              Displays of kernel text/data, which take full advantage  of  the
              power of gdb to format and display data structures symbolically.

       System state
              The  majority  of  crash  commands  consist of a set of "kernel-
              aware" commands, which delve into various kernel subsystems on a
              system-wide or per-task basis.

       Utility functions
              A  set  of useful helper commands serving various purposes, some
              simple, others quite powerful.

       Session control
              Commands that control the crash session itself.

       The following alphabetical list consists of a very simple  overview  of
       each  crash  command.   However,  since  individual commands often have
       several options resulting in  significantly  different  output,  it  is
       suggested  that  the  full  description  of  each  command be viewed by
       executing crash -h <command>, or  during  a  crash  session  by  simply
       entering help command.

       *      "pointer  to"  is  shorthand  for  either  the  struct  or union
              commands.  It displays the contents of  a  kernel  structure  or

       alias  creates a single-word alias for a command.

       ascii  displays  an  ascii chart or translates a numeric value into its
              ascii components.

       bt     displays a task's kernel-stack backtrace.  If it is given the -a
              option,  it displays the stack traces of the active tasks on all
              CPUs.  It is often used with the foreach command to display  the
              backtraces of all tasks with one command.

       btop   translates a byte value (physical offset) to its page number.

       dev    displays   data   concerning  the  character  and  block  device
              assignments, I/O port usage, I/O memory usage,  and  PCI  device

       dis    disassembles  memory,  either  entire  kernel  functions, from a
              location for a specified number of  instructions,  or  from  the
              start of a function up to a specified memory location.

       eval   evaluates  an expression or numeric type and displays the result
              in hexadecimal, decimal, octal and binary.

       exit   causes crash to exit.

       extend dynamically loads  or  unloads  crash  shared  object  extension

       files  displays information about open files in a context.

              repeats  a specified command for the specified (or all) tasks in
              the system.

       fuser  displays the tasks using the specified file or socket.

       gdb    passes its argument to the embedded gdb module.   It  is  useful
              for  executing  gdb  commands  that  have the same name as crash

       help   alone displays the command menu; if followed by a command  name,
              a  full  description of a command, its options, and examples are
              displayed.  Its output is far more complete and useful than this
              man page.

       ipcs   displays data about the System V IPC facilities.

       irq    displays  data  concerning interrupt request numbers and bottom-
              half interrupt handling.

       kmem   displays information about the use of kernel memory.

       list   displays the contents of a linked list.

       log    displays the kernel log_buf contents in chronological order.

       mach   displays data specific to the machine type.

       mod    displays  information  about  the  currently  installed   kernel
              modules,  or  adds  or deletes symbolic or debugging information
              about specified kernel modules.

       mount  displays information about the currently-mounted filesystems.

       net    display various network related data.

       p      passes its arguments to the gdb "print" command  for  evaluation
              and display.

       ps     displays  process status for specified, or all, processes in the

       pte    translates the hexadecimal contents of a PTE into  its  physical
              page address and page bit settings.

       ptob   translates a page frame number to its byte value.

       ptov   translates  a hexadecimal physical address into a kernel virtual

       q      is an alias for the "exit" command.

       rd     displays the contents of memory, with the  output  formatted  in
              several different manners.

       repeat repeats  a  command  indefinitely,  optionally  delaying a given
              number of seconds between each command execution.

       runq   displays the tasks on the run queue.

       search searches a range of user or kernel memory space for given value.

       set    either sets a new context,  or  gets  the  current  context  for

       sig    displays signal-handling data of one or more tasks.

       struct displays  either  a  structure  definition  or the contents of a
              kernel structure at a specified address.

       swap   displays information about each configured swap device.

       sym    translates a symbol to its virtual address, or a  static  kernel
              virtual  address  to  its  symbol  -- or to a symbol-plus-offset
              value, if appropriate.

       sys    displays system-specific data.

       task   displays the contents of a task_struct.

       tree   displays the contents of a red-black tree or a radix tree.

       timer  displays the timer queue entries, both old-  and  new-style,  in
              chronological order.

       union  is similar to the struct command, except that it works on kernel

       vm     displays basic virtual memory information of a context.

       vtop   translates a user or kernel  virtual  address  to  its  physical

       waitq  walks the wait queue list displaying the tasks which are blocked
              on the specified wait queue.

       whatis displays the  definition  of  structures,  unions,  typedefs  or
              text/data symbols.

       wr     modifies  the  contents of memory on a live system.  It can only
              be used if /dev/mem is the device  file  being  used  to  access
              system RAM, and should obviously be used with great care.

       When crash is invoked with a Xen hypervisor binary as the NAMELIST, the
       command set is slightly modified.  The *, alias, ascii, bt, dis,  eval,
       exit,  extend,  gdb,  help, list, log, p, pte, rd, repeat, search, set,
       struct, sym, sys, union, whatis, wr and q  commands  are  the  same  as
       above.  The following commands are specific to the Xen hypervisor:

       domain displays  the  contents of the domain structure for selected, or
              all, domains.

       doms   displays domain status for selected, or all, domains.

              displays Xen dump information for selected, or all, cpus.

       pcpus  displays physical cpu information for selected, or all, cpus.

       vcpus  displays vcpu status for selected, or all, vcpus.


              Initialization commands.  The file can be located in the  user's
              HOME  directory and/or the current directory.  Commands found in
              the .crashrc file in the  HOME  directory  are  executed  before
              those in the current directory's .crashrc file.


       EDITOR Command  input  is  read using readline(3).  If EDITOR is set to
              emacs or vi then suitable keybindings are used.   If  EDITOR  is
              not  set,  then vi is used.  This can be overridden by set vi or
              set emacs commands located in a .crashrc file, or by entering -e
              emacs on the crash command line.

              If  CRASHPAGER  is  set,  its  value  is used as the name of the
              program to which command output will  be  sent.   If  not,  then
              command output is sent to /usr/bin/less -E -X by default.

              Specifies  an  alternative  directory  tree to search for kernel
              module object files.

              Specifies a directory containing extension modules that will  be
              loaded automatically if the -x command line option is used.


       If  crash  does  not  work,  look for a newer version: kernel evolution
       frequently makes crash updates necessary.

       The command set scroll off will cause output to be sent directly to the
       terminal  rather  than  through  a paging program.  This is useful, for
       example, if you are running crash in a window of emacs.


       Dave Anderson <> wrote crash.

       Jay    Fenlason     <>     and     Dave     Anderson
       <> wrote this man page.


       The  help  command  within  crash  provides  more complete and accurate
       documentation than this man page. - the home page of the crash utility.

       netdump(8), gdb(1), makedumpfile(8)