icrash(1M) icrash(1M) NAME icrash - IRIX system crash analysis utility SYNOPSIS icrash [-e cmd] [-f cmdfile] [-n bounds] [-r] [-v] [-w outfile] [-I icrashdef_file] [-S] [-Q] [-M path] [-H path ] namelist corefile DESCRIPTION icrash is a hands-on utility that generates detailed kernel information in an easy-to-read format. icrash also provides the ability to generate reports about system crash dumps created by savecore(1M). Depending on the type of system crash dump, icrash can create a unique report that contains information about what happened when the system crashed. icrash can be run on live systems or with any namelist and corefile specified on the command line. namelist contains symbol table information needed for symbolic access to the system memory image being examined. The default namelist is /unix, which is used when analyzing a live system. If the memory image being analyzed is from a system core dump (vmcore.N.comp), then namelist must be a copy of the unix file that was executing at the time of the crash (unix.N). corefile is a file containing the system memory image. The default corefile is /dev/mem, which provides access to system memory when analyzing a live system. corefile can also be a pathname to a file (vmcore.N.comp) produced by the savecore(1M) utility. The icrash command has the options listed below. By default, all information is sent to the standard output, unless the -w option is used: -e cmd Specifies on the command line a set of commands that icrash runs automatically. If more than one command is specified, or if there are any command line options, then cmd must be enclosed within quotation marks. -f cmdfile Specifies a cmdfile that contains a set of commands that icrash runs automatically. -n bounds bounds specifies the numeric value associated with a particular namelist/corefile pair. -r Generates a standard report for the namelist and corefile specified. The reporting style differs slightly depending on the type of system dump being analyzed (panic or NMI). -v Prints the current version number. -w outfile Writes any generated output to outfile. -I icrashdef file In very rare instances, icrash will fail to start up due to corruption of the icrashdef kernel data structure (the icrashdef structure contains key system information necessary for icrash to bootstrap). Assuming that the rest of the vmcore image has not been corrupted, it is possible to use a copy of an icrashdef structure from another, similarly configured system, a good vmcore image from the same system, or the live system itself. First, start up icrash against one of the above mentioned options. Then issue the 'icrashdef' command, followed by the name of the file where the good icrashdef structure will be written.. Then, include this file when you start icrash, using the -I command line option flag. Note that, in most cases, having a bad icrashdef structure usually means you have a corrupt vmcore image or there is a mismatch between the namelist file (unix.N) and vmcore image you are using. Even if icrash is able to start up, it is possible that very little useful information can be obtained from dump. -S Suppresses all output except that which comes from icrash commands. Normally used in conjunction with the -e flag. -Q Starts icrash in "quick" mode. Prevents the sial macro commands from being loaded at startup (the sial commands can be loaded at a later time using the load command). -M path Specifies the PATH to search for loading macros either on startup or when the 'load' command is issued at the icrash prompt. On startup all files in these directories are loaded automatically. The default macro path is ~/.icrash:/usr/share/sial/icrash. The environment variable IC_MPATH can be used for that purpose also. (see MACROS section below). Note command-line argument takes precedence over environment variable. -H path Specifies the PATH to search for #include directives in macro files. The default value for this path is ~/.icrash/include:/usr/share/sial/icrash/include:/usr/include. The environment variable IC_HPATH can be used for that perpose also. (see MACROS section below). Note command-line argument takes precedence over environment variable. REPORT USAGE In order to generate a crash report, you must invoke the icrash command using the -r flag. For example: icrash -r namelist corefile This creates an ICRASH CORE FILE REPORT that prints to standard output. The resulting report can then be analyzed to determine what type of failure caused the system to crash and dump core. Note that you cannot use the -r flag to generate a crash report on a live system. INTERACTIVE USAGE Input during an icrash session is of the form: command [argument ... ] where command is one of the icrash commands described in the COMMANDS section of this reference page and argument is qualifying data that determines the specific behavior of the icrash command, which kernel structure to print, the number of items to display, etc. The following options are available to all icrash commands wherever they are semantically valid (see the COMMANDS section below). -a Display all entries -- even those that are currently unallocated, on a free list, or have a reference count of zero. -f Display additional (full) information for a structure. Note that information from all of the fields will not necessarily be displayed. Only those fields that are deemed to be most useful to the debugging process. The icrash print command can be used to display the contents of entire kernel structures. -n Follow the links of related kernel structures to their logical conclusion. For example, when used with the stream command, information for the stdata structure for the stream is displayed, followed by information on each of the queue pairs on the stream. -w outfile Redirect the output of a commands to the named outfile. The output from all icrash commands can be piped to various IRIX commands in the following manner: command [ argument ... ] | shell_command Depending on the context of the command, numeric arguments are assumed to be in a specific radix. Counts are assumed to be decimal. Addresses are always hexadecimal (with or without a leading 0x). Table address arguments larger than the size of a table are interpreted as hexadecimal addresses; those smaller are assumed to be decimal slots in the table. COMMANDS Below is a list of the interactive commands that are included with icrash. Please note that this list can change from one release to the next. To get an on-line list of all the commands available within icrash, use the help command. addtype namelist [symbol_list] Read in all the type definitions from namelist. The object file with the desired type definitions in the namelist must be compiled with debugging flags turned on (-g). If a data type of a particular name is already know by icrash, then icrash will not load that type. Note that caution should be used when intermixing 32-bit object namelists with this command when debugging 64-bit vmcores and viceversa. anon [-a] [-f] [-n] [-w outfile] anon_list Display the anon structure located at each virtual address included in anon_list. If the -a option is specified, the pfdats in the pcache is displayed in ascending order of their page numbers. Use the -f option if the command hangs when used with the -a option. The -f option displays all the pfdat's in the pcache on a "first-find" basis. anontree [-a] [-f] [-n] [-w outfile] anon_list Display the anon tree structure for each virtual address included in anon_list. avlnode [-f] [-n] [-w outfile] avlnode_list Display the avlnode structure located at each virtual address included in avlnode_list. If the -n option is specified, the root of the avltree is identified (using the parent pointer contained in the avlnode structure) and all the avlnodes in the tree are displayed. base numeric_value ... Display a number in binary, octal, decimal, and hexadecimal. A number in a radix other then decimal should be preceded by a prefix that indicates its radix as follows: 0x, hexadecimal; 0, octal; and 0b, binary. config [-f] [-w outfile] [cpu_list] Display the board configuration for a system using the information contained in the hwgraph. ctrace [-a] [-f] [-w outfile] [cpu_list] Displays a stack trace for each CPU included in cpu_list. If cpu_list is empty and defkthread is set, ctrace displays a stack trace for the cpu that defkthread was running on. If defkthread is not set, ctrace displays a trace for the cpu dumpproc was running on. If there isn't any dumpproc, ctrace displays a stack trace for the cpu that initiated the dump (based on the SP saved in dumpregs). Or, if the -a option is included in the command line, ctrace displays a stack trace for each cpu in the system. curkthread [-f] [-w outfile] Display the process, sthread, or xthread entry for the kthread currently running on all CPUs. defkthread [-w outfile] [kthread] Set the default kthread if one is indicated on the command line. Otherwise, display the current value of defkthread. When icrash is run against a system core dump, defkthread gets set automatically to the kthread that was active when the system panic occurred. When icrash is run on a live system, defkthread does not get set. The defkthread value is used by icrash in a number of ways. The trace command will display a stack trace for the default kthread if one is set. Also, the translation of certain kernel virtual addresses (primarily those contained in the kernelstack) depend upon defkthread being set. die [-w outfile] [die_addr] Print out a DWARF information entry at a given die_addr. dirmem [-b|-o|-d|-h] [-s] [-m] [-p] [-e] [-r low[-hi]] addr Display contents of the directory memory. The memory address addr can be given as a kernel symbolic address (example could be 'dirmem fork', or a hexadecimal address such as 'dirmem 0xa800000f47160000'. The leading '0x' can be omitted, the address will be understood to be hexadecimal. dirmem -h|-help Display more expanded help info. dis [-f] [-w outfile] dis_addr [count] Display the disassembled code located at dis_addr for count instructions. The default count is 1. dump [-d] [-o] [-x] [-B] [-D] [-H] [-W] [-w outfile] addr [count] Display count values starting at virtual address addr in one of the following formats: decimal (-d), octal (-o), or hexadecimal (-x). The default format is hexadecimal, and the default count is 1. eframe [-w outfile] eframe_addr Display the exception frame (containing a register dump, EPC, cause register, and status register) located at eframe_addr. etrace [-f] [-w outfile] eframe_addr Display a stack trace using the PC and SP found in the exception frame pointed to by eframe_addr. The stack address is determined using the stack pointer from the exception frame. Note that defkthread must be set equal to the appropriate kthread when stack_addr is from the kernelstack or when stack_addr is the CPU interrupt stack and there is a kthread running on that CPU. eval [C code] Execute a piece of C code inside the sial interpreter and preprocessor. Any valid C code or preprocessor commands can be given. Apart from standard expressions the following can also be executed. Global variables can be set from the command line: example: >> eval int myglobal=3; Preprocessor defines can also be set from the command line: example: eval #define MYDEBUG 1 Unified types and values can be printed (unified types are all types defined by the images and by already loaded sial scripts) like so: example: >> eval print(struct mystruct); or eval print(*(struct mystruct)0xabcd1234); With no arguments, eval will read input untill end of file is encountered or ^D. Because of the above feature the -w is not supported for eval. use the outfile command to achieve the same result. file [-a] [-n] [-p proc_spec] [vfile_list] This command lists opened files on the system. Use -n to display intermediate values (vnode, snode etc..). Use -p proc_spec to list files for a specific process. proc_spec is either a 'struct proc*' or a '#' followed by a pid. findsym [-f] [-w outfile] address_list Locate the kernel symbol closest to each virtual address contained in address_list. from cmdfile Read in commands from cmdfile and run them as if typed from the icrash command line. Note that the -w option and pipe (|) option are not valid options for the from command. These options can, however, can be included in the commands contained in cmdfile. fstype [-w outfile] [vfssw_list] Display the vfssw structure for each entry in vfssw_list. Entries in vfssw_list can take the form of a virtual address or vfssw table slot number. If no vfssw entries are specified, display the entire vfssw table. func [-w outfile] [func_addr] Print out information about the function located at func_addr. help [all | command_list] Display a description of the named commands, including command syntax and an overview of how to use the command. The all option displays help information for every command. history [-w outfile] Dump out the last 20 history commands. You can also use ! to access old commands (including !!, !-N, etc.) hubreg [-b|-o|-d|-x] [-s] [-n n] register Display contents of a hub register. The register name can be given in upper case or in lower case or even as a full address. If the address corresponds to a real hub register, its name will be found and displayed. The register given by its name (and not by its address), does not have to be its full name, only the first so many letters making its name unique are needed. If a unique match cannot be found, the command will list all the register names matching the characters given, and you can then pick up the correct one from the list. hubreg -l|-list List all defined register names. hubreg -h|-help Display more expanded help hubreg -listacc Display info on access mode definitions. hwpath [-f] [-w outfile] element_lsit Display all pathnames from the /hw file system that terminate with the names included in element_list, or that contain label names included in element_list. If no names are specified, display all unique pathnames. Unique pathnames are ones that terminate in a file or symbolic link rather than in a directory. When the hwpath command is issued with a -f command line option, the vertex handle for each element in the path is also displayed. Note that the hwpath command provides a view of the the internal hwgraph data structures which are are presented to the user via a filesystem-like interface. inode [-f] [-w outfile] inode_list Display the inode structure located at each virtual address included in inode_list. inpcb [-f] [-w outfile] inpcb_list Display the inpcb structure located located at each virtual address included in inpcb_list. kthread [-a] [-f] [-w outfile] [kthread_list] Display relevant information for each entry in kthread_list. If no entries are specified, display information for all active kthreads (processes, xthreads, and sthreads). Entries in kthread_list should be in the form of a virtual address. klist [-s] [-u] [-x] With no addition arguments, this command gives a list of all kthreads on the system. If one or more of the -s, -u, -x options are supplied, then the corresponding list(s) of threads is given i.e. sthreads, uthreads and/or xthreads. ktrace [-a] [-f] [-w outfile] [kthread_list] Displays a stack trace for each kthread included in kthread_list. Entries in kthread_list should be in the form of a virtual address. lsnode [-f] [-w outfile] lsnode_list Display the lsnode structure located at each virtual address included in lsnode_list. mbuf [-a] [-f] [-n] [-w outfile] mbuf_list Display the mbuf structure located at each virtual address included in mbuf_list. In irix6.5 the mbuf command when used with the -a option will display details of all the mbufs allocated on the system along with output similar to the "netstat -m" output on a live machine. memory [-f] [-w outfile] Display information about the memory installed on each node in the system. If the memory command is issued with the -f option, display information about the memory contained in each memory slot on each node. mlinfo [-a] [-f] [-n] [-w outfile] mlinfo_list. Display information from the ml_info struct for each virtual address included in mlinfo_list. mntinfo [-f] [-n] [-w outfile] mntinfo_list Display the mntinfo structure for each virtual address included in mntinfo_list. The mntinfo contains information about NFS mounted file systems. mrlock [-f] [-n] [-w outfile] mrlock_list Display the mrlock_s structure located at each virtual address included in mrlock_list. If the -f command line option is specified, display all queues of WAITERS and HOLDERS. If the -n command line option is specified (in conjunction with the -f option), then walk each queue displaying information on associated kthreads. nodepda [-f] [-w outfile] [nodepda_list] Display the nodepda_s structure associated with each item in nodepda_list. Entries in nodepda_list can take the form of a node ID or a virtual address pointing to a nodepda_s struct in memory. If nodepda_list is empty, information for all nodes will be displayed. outfile [outfile] Set outfile (the file where all command output is sent) if outfile is indicated. Otherwise, display the current value of outfile. pager [-w outfile] [on | off] When pager is on, all command output is piped through pg. When the pager is off, all output is printed continuously without page breaks. When the pager command is issued without a command line option, the current state of the pager is displayed (on or off). Note that when the pager is on, command output cannot be redirected to a pipe. pda [-f] [-w outfile] cpu_list Display the pda_s structure associated with each item in cpu_list. Entries in cpu_list can take the form of a CPU ID or a virtual address pointing to a pda_s struct in memory. If cpu_list is empty, the pda_s structure for all CPUs will be displayed. pde [-w outfile] pde_list Display the pde (page descriptor entry) structure located at each virtual address included in pde_list. pfdat [-a] [-f] [-w outfile] pfdat_list Display the pfdat structure located at each virtual address included in pfdat_list. If the -a flag is issued, display all of the pfdat structures on the system (one per physical page). pid [-a] [-f] [-w outfile] pid_list Display the pid_entry structure for each entry in pid_list. Entries in pid_list can take the form of a pid table slot number, process PID (following a #), or a virtual address. If no entries are specified, display all active (in use) pid entries. pregion [-a] [-f] [-n] [-p proc] [-w outfile] [-u uthread] [pregion_list] Display the pregion structure located at each virtual address included in pregion_list. If the -p option is used, display all pregions allocated to proc. When used with the -n option display all the pde information for the proc or uthread specified with the -p or -u options respectively. proc can be specific as a process PID (following a #) or virtual address. print [-d] [-o] [-x] [-w outfile] expression Evaluate expression and print the result. An expression can consist of numeric values, operators, kernel variables, typedefs, struct/union members, or a combination of above. The following are examples of valid expressions: (((2*3+4/2)*2+(2/6))/2) *((struct socket*)0xa80000001019f2c8)->so_rcv->sb_mb (((pte_t*)0xa8000000005ef088)->pte_pfn)<<14|0xa800000000000000 printd [-w outfile] expression The printd command is the same as the print command except that all integers are printed as decimal values. printo [-w outfile] expression The printo command is the same as the print command except that all integers are printed as octol values. printx [-w outfile] expression The printx command is the same as the print command except that all integers are printed as hexadecimal values. proc [-a] [-f] [-l] [-t] [-h] [proc list] This command displays various information about processes. Option -t gives a list of the current timers for each threads and a aproximated elaps time (in seconds) in the corresponding state. Option -l Show the complete command line w/ arguments. Option -h show the hiearchy of processes. Option -a gives the list of all processes (default) Option -f gives a full listing of each process including opened files ptov [-w outfile] address_list Display all possible virtual address mappings (K0, K1, and K2) for each entry in address_list. Entries in address_list can be a hexadecimal physical address or a PFN (following a #). queue [-f] [-n] [-w outfile] queue_list Display the queue structure located at each virtual address included in queue_list. If the next option (-n) is specified, a linked list of queues, starting with each specified queue then following the q_next field, is displayed. quit Exit icrash. Note that q prompts for confirmation unless a ! is appended to the command line. region [-f] [-n] [-p proc] [-w outfile] region_list Display the region structure located at each virtual address included in region_list. If the -p option is used, display all regions allocated to proc. proc can be specific as a process PID (following a #) or virtual address. report [-w outfile] Print out the base report automatically generated by the savecore script when the system comes back up after a system crash. rnode [-f] [-n] [-w outfile] rnode_list Display the rnode structure for each virtual address included in rnode_list. The rnode contains information about a virtual node that acts as an interface between two NFS systems. runq [-c] | [-g] | [cpulist] This command displays the current run queues on all cpus , the global RT queue and cpuset queues. Use cpulist to limit the report to only some cpus. Use the -g option to limit the report to the global queues. Use the -c option to limit the report to the cpuset queues. sbe [-w outfile] Print out the single bit error information for a Challenge or Onyx system (including R8K and R10K systems), assuming the right single bit flags in the kernel are turned on. search [-B] [-D] [-H] [-W] [-w outfile] [-m mask] pattern [address] [length] Locate contiguous bytes of memory that match the values contained in pattern, beginning at address for length bytes. pattern consists of a string of, from one to 256 hexadecimal digits (with no embedded spaces). For full word searches (the default), the first word of mask is anded (&) with each word of memory and the result compared against the first word in pattern (which is also anded with the first word of mask). If there is a match, subsequent words in memory are compared with their respective words in pattern (if there are any) to see if a match exists for the entire pattern. If the -D option is issued, the search is conducted on double word boundaries. If the -H option is issued, the search is conducted on halfword boundaries. If the -B option is issued, the search will be performed without regard to double word, word or halfword boundaries. If a mask is not specified, mask defaults to all ones for the size of pattern. For all but string searches, the number of hex digits specified for pattern cannot be less than will fit into the memory boundary size specified. For example, patterns for halfword boundary searches, must contain (at least) eight hex digits (two per byte). Also, any extra digits beyond the specified boundary will be trimmed off from the right side of the input line. In addition to finding matches for hex values in memory, it is possible to search for strings of characters as well. Just begin and end the character search pattern with double quotes ("). The ASCII value for each character in the string will form the corresponding byte values in the search pattern. address can be specified as either a virtual address (K0, K1, K2, etc.), a physical address, or as a PFN (directly following a pound # sign). If no address is specified (or if the one specified does not map to a valid physical memory address), address defaults to the K0 address mapped to the start of physical memory. An optional length parameter specifies the number of bytes to search. If length is not specified, it will be set equal to the size of physical memory minus the starting physical address. Note that length can be specified ONLY when an address has been specified. sema [-f] [-w outfile] [sema_list] Display the sema_s structure located at each virtual address included in sema_list. With the -f option a list of waiters, waiting on the sema, are displayed. sh [command] Escape to the shell with no arguments or execute command. Note that it uses your SHELL environment variable to determine which shell to use. sizeof structure[.field] Dump out the size of structure entered on the command line. The value returned will be in bytes. The user can also use a structure.field notation in order to find out what the size of a field in structure. If the sizeof command is unable to find structure or field, then the structure information is not located in the symbol table. Also note that the sizeof command only returns the size of struct and union types. slpproc [-w outfile] Print out the set of sleeping processes on the system. socket [-f] [-n] [-w outfile] [socket_list] Display the socket structure for each virtual address included in socket_list. If no entries are specified, display all sockets that are currently allocated. If the next option (-n) is specified, a linked list of protocol control block structures associated with each socket will also be displayed. stack [-w outfile] stack_pointer Print out the stack type, stack address, and stack size for a the stack stack_pointer is from. If stack_pointer is from a kthread stack, then defkthread must be set equal to the address of the appropriate kthread. stat [-w outfile] Display system statistics and the putbuf array, which contains the latest messages printed via the kernel printf/cmn_err routines. sthread [-a] [-f] [-w outfile] [sthread_list] Display relevant information for each entry in sthread_list. If no entries are specified, display information for all active sthreads. Entries in sthread_list can take the form of a virtual address. strace [-a] [-f] [-l] [-w outfile] [pc] [sp] [stack_addr] [level] Displays all complete and unique stack traces (containing level or more stack frames) from the stack starting at stack_addr. If level isn't specified, then each stack trace must have at least five frames to be considered valid. Alternately, use a specific pc and sp to generate a stack trace from the stack starting at stack_addr. Or, when the -l command line option is specified, displays a list of all valid kernel code addresses contained in the stack starting at stack_addr, along with their location in the stack, source file and line number. Or, if the -a option is specified, display ALL traces of level or more frames, including invalid traces and duplicate (sub) traces. Note that defkthread must be set equal to the kthread being analyzed when stack_addr is the kernel stack address or the CPU interrupt stack and there is a process running on that CPU. This is necessary so that the trace can automatically "jump" from the first stack to the kernel stack of the process being analyzed. stream [-a] [-f] [-n] [-w outfile] [stream_list] Display the stdata structure for each virtual address included in stream_list. If no entries are specified, display all streams that are currently allocated. If the next option (-n) is specified, a linked list of queues that are associated with the stream is also displayed. strings [-w outfile] start_address | symbol [count] Display count strings of ASCII characters starting at start_address (or address for symbol). strstat [-f] [-w outfile] Display information about streams related resources (streams, queues, message blocks, data blocks, and zones). struct [-f] [-l] [-n] [-w outfile] struct addr Print structure information for struct using the block of memory pointed to by addr. swap [-f] [-w outfile] [swap_list] Dump out the list of swap devices, including the vnodes that are represented. The number of pages, number of free pages, number of max pages, priority, and device are listed. The -f flag dumps out the name of the swapinfo entry as well. symbol [-f] [-w outfile] symbol_list Displays information about each kernel symbol included in symbol_list. tcp [-f] [-w outfile] tcpcb_list Display the tcpcb structure for each virtual address included in tcpcb_list. tlbdump [-w outfile] [cpu_list] Display TLB information for each CPU indicated in cpu_list. If no CPUs are indicated, TLB information for all CPUs is displayed. trace [-a] [-f] [-w outfile] [kthread_list] Displays a stack trace for each kthread included in kthread_list. If kthread_list is empty and defkthread is set, then a stack trace for the default kthread is displayed. If defkthread is not set, then a trace will be displayed for the process stored in dumpproc. If there isn't any dumpproc, then trace displays a trace for the cpu that generated the core dump. type [-w outfile] [symbol] | type_addr Display the contents of memory starting at addr using the type definition type to format the output. Structures, unions, etc. are displayed in a manner similar to the way they were originally defined. The values contained in the various struct/union members are displayed in accordance with their type declarations. For example, kernel addresses are displayed as hexadecimal values, signed and unsigned integers are displayed as decimal values, etc. Struct/union members that are themselves structs or unions are expanded in the much the same way. Alternately, if a kernel symbol is entered, instead of type_addr, an attempt is made to determine the type of the symbol. If the type can be determined, it will be displayed as described above. unpcb [-f] [-w outfile] [unpcb_list] Display the unpcb structure for each virtual address included in unpcb_list. uthread [-l] [-f] [-n] [-k] [-S] [-w outfile] uthread_list Display relevant information for each entry in uthread_list. If no entries are specified, display information for all active uthreads. When the -S command line option is specified, displays a list of all siblings for the specified uthread. Entries in uthread_list can take the form of a process PID (following a '#'), or virtual address. Note that with the PID option, all uthreads that share a specified PID will be displayed. When the -k command line option is specified, display all uthread_s structs as kthread structs. When -l is used, only the uthreads contributing to the load average are shown. vertex [-a] [-f] [-w outfile] [vertex_list] Display the graph_vertex_s structure located at each virtual address included in vertex_list. vfs [-f] [-w outfile] vfs_list Display the vfs structure for each virtual address included in vfs_list. vnode [-a] [-f] [-n] [-w outfile] vnode_list Display the vnode structure for each virtual address included in vnode_list. vproc [-a] [-f] [-n] [-w outfile] vproc_list Display the vproc structure for each virtual address included in vproc_list. vsocket [-a] [-f] [-n] [-w outfile] vsocket_list Display the vsocket structure for each virtual address included in vsocket_list. vtop [-w outfile] address_list Display all possible virtual address mappings (K0, K1, and K2) for each virtual address in address_list. walk [-f] [-l] [-s] [-w outfile] [struct field|offset addr] [addr offset size] Walk a linked list of kernel structures or memory blocks. The walk command has three modes of operation. By default, output from the walk command consists of a linked list of formatted structure entries. Each entry contains a single line of output, similar to the output of other icrash commands. The list of structures which can be displayed in this manner is limited. To see a listing, issue the walk (or struct) command with the -l command line option. Note that when viewing the list of structures, only those structures marked YES in the LINKS column contain links that can be followed. When the walk command is issued with the -s option, each structure is displayed, in its entirity, in a C-like format. The only limitation on which structures can be walked in this manner is that structure related information must be contained in the kernel's symbol table. Even if information about a structure is not available, it is possible to do a hex memory dump of each structure in the list. With the first two options outlined above, the structure name, field name (or byte offset), and next pointer are required. With the third option, a start address, byte offset, and struct size are required. Examples: walk xfs_inode i_next a800000000345678 walk xfs_inode 8 a800000000345678 walk -s mrlock_s mr_queue a80000000345678 walk -s mrlock_s 8 a80000000345678 walk a800000000345678 8 40 The last example illustrates the use of the walk command to display debugging information about a structure even if its type information is not found. It does a memory dump of each structure in the list. whatis [-a] [-f] [-l] [-w outfile] expression Display type specific information for expression (see the print command for what constitutes a valid expression). If the -a option is specified, display a list of all structures, unions, typedefs and variables. When, in conjunction with the -a option, the -f option is specified, expand all struct and union definitions to show information for each field. xthread [-f] [-n] [-k] [-S] [-w outfile] xthread_list Display relevant information for each entry in xthread_list. If no entries are specified, display information for all active xthreads. Entries in xthread_list can take the form of a virtual address. When the -k command line option is specified, display all xthread_s structs as kthread structs. zone [-a] [-f] [-w outfile] [zone_list] Display information about zone memory allocation resources. If the -a option is specified, display a list of all zones allocated in the system by zone type. ? [-w outfile] Displays a list of available commands. MACROS Macros of C code can be used to create new commands and expand icrash's functionality. They can be useful for commands that are specific to a site or engineering group. > Preprocessor #define's The interpreter makes a number of preprocessor defines available to the macros. This can be important for proper structure member alignment or to execute different pieces of code within the macro according to architecture or processor type. All kernel compile time -D options (see the DW_AT_producer records from 'dwarfdump -i /unix') have their corresponding defines available in the macro context. Additionally, these mips compiler #defines are pushed into the macro context: Name Value(32 bit kernel) Value(64 bit kernel) LANGUAGE_C 1 1 mips 1 1 unix 1 1 sgi 1 1 host_mips 1 1 _SGI_SOURCE 1 1 _LONGLONG 1 1 _SVR4_SOURCE 1 1 _LANGUAGE_C 1 1 _MODERN_C 1 1 __sgi 1 1 __host_mips 1 1 _SYSTYPE_SVR4 1 1 __unix 1 1 _SYSTYPE_SVR4 1 1 _MIPS_ISA 4 4 _ABIN32 2 2 _MIPS_SIM _ABIN32 _ABIN32 _MIPS_FPSET 32 32 _MIPS_SZINT 32 32 _MIPS_SZLONG 32 64 _MIPS_SZPTR 32 64 IRIX_RELEASE 0xmmnnrr 0xmmnnrr where mm is the major version of IRIX nn is the minor version of IRIX rr is the revision number of IRIX Note: numbers are in hexadecimal to ease arithmetic comparision IRIX_MAJOR (IRIX_RELEASE & 0xff0000) >>16 IRIX_MINOR (IRIX_RELEASE & 0x00ff00) >>8 IRIX_REV IRIX_RELEASE & 0x0000ff) FEATURE_STREAM 1 if 'f' stream. 0 otherwise. ACTIVE 1 if working on a live kernel. 0 otherwise. > Macro related commands >> load <filename>|<directory> Will load/compile either a single file or in the case of a directory, all the files in that directory. Icrash will automatically unload previously compiled files as new ones are compiled successfully. >> unload <filename>|<directory> Will unload either a single file or in the case of a directory, all the files in that directory. >> vi -l|-f <filename>|<funcname> Will start editing a macro from your icrash session. On exit from the editor, the macro will be recompiled and reloaded automatically. The -l option can be used when a compile or run time error as been encountered. It will position the editor at the file and line coordinates of that error. The -f must be used to start editing files that have not been loaded yet. filename must be an absolute path name. When simply specifying the function name funcname, the editor will be positioned on the first line of the statement body of that function. > Creating new commands for icrash Each time a macro is loaded, icrash will look for specific sets of functions. When it founds these sets, it will register a new command for it. Actually any global macro function (not static) is callable from the icrash prompt, but only functions with a complete set will have a corresponding entry with the help command. The set of functions is : int <funcname>() { do_the_real_work_here; } string <funcname>_usage() { return "list of permited parameters"; } string <funcname>_opt() { return "corresponding getopt option list"; } string <funcname>_help() { return "Help text for this command"; } After the load the user will be able to run the command <funcname> by simply typing 'funcname [args]' at the icrash prompt. Notice that the main entry point 'funcname' does not have any parameters. This is because options and their associated values and any additional arguments are made available by creating global variables that are accessible by all macro functions throughout the execution of the command. Each option, if specified, will trigger the existence of a flag variable. These variables are called Xflag, where X is the letter for this options. For example a function with a getopt string of "a" will possibly have a aflag variable available. The builtin exists("variable name") can be used to test for this option. If an option has an associated value (the getopt ':' was specified), that value is made available as a string type variable called Xarg, where X is the option letter. So for a command with a getopt string of "b:" funcname might check for the bflag existance and use barg for it's associated value. The rest of the arguments supplied by the user are made available in an array of 'string' called argv. argv[0] being set to the name of the command. The global argc specifies the total number of entries in argv[]. For more information on the interpreter itself (builtin functions, api, dso support etc.) please refer to the lkcdutils/libsial/README file at http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/?cvsroot=lkcd > Example command /* Example macro. Both, C and C++ type comments are supported. */ static int all=0; string foo_opt(){ return "ab:c"; } string foo_usage() { return "[-a] [-b barg] [-c] addr [addr [addr...]]"; } static void foo_showusage() { printf("Usage: foo %s0, foo_usage()); } string foo_help() { return "Foo is an example function " + "which shows some of the key features of the "+ "sial interpreter. " + // anything between <<>> will show in bold. "Option <<-a>> or <<-b barg>> are supported "+ "Additional arguments will be placed in <<argv[]>>"; } static int doproc(proc_t *p) { printf("p=0x%p uid=%d pid=%d ppid=%d name=[%s]0, p, p->p_cred->cr_uid, p->p_pid, p->p_ppid, // extract a string from the image with getstr() getstr(p->p_ui.u_comm)); } // sial support local type declarations typedef struct mystruct { int a; short b; char c[200]; } my_t; int foo() { int all, i; string bval; #if 0 char a[3]; // array extenciations are not permitted. // use dynamic arrays if you need them. #endif /* Comment about accessing symbols in the image: All symbols in the image are treated as (void*). That is, their value becomes their address in the image. That's because there is no type information attached to the symbols in /unix. Based on that, if the kernel declares a : struct mystruct my; A macro will use it like so: my_t *lmy=(my_t*)my; If the kernel declares a : struct mystruct *my; A macro will use it like so: my_t *lmy=*((my_t**)my); */ if(exists("aflag")) all=1; else all=0; if(exists("bflag")) bval=barg; // argv[0] is set to the name of the command i.e. "foo" for(i=1; i<argc;i++) { proc_t *p; p=(proc_t*)atoi(argv[i]); doproc(p); } // need at least one arg for this command. if(i==1) foo_showusage(); } NOTES Each version of icrash is specific to the OS release that it came from and does not necessarily work on any other OS release. Do not copy icrash to any other IRIX system unless the OS versions are identical (including patch levels). Running icrash on a live system can sometimes generate random results, as the information being viewed is volatile at the time it is displayed. When running against compressed system crash dumps, an index file will be created to allow icrash quick access to compressed pages inside the dump. In addition, if an index file exists, icrash will start much more quickly against a crash dump. icrash will always try to write the index file to the same directory where the system crash dumps are stored. If the directory is not writable, you will see the message: icrash: save of index "filename" failed: permission denied where filename is the name of the index file that icrash tried to create. To avoid this, users can create symbolic links to the namelist and corefile in a writable directory, and run icrash on the symbolic links. The index file will then be created in the writable directory where the symbolic links are located. BUGS & ISSUES Some icrash commands will not work completely on O2's with the R10000 cpu in them. This is because kernel space in the address range 0x800000- 0x9fffff are not readable to user processes for this hardware configuration. The commands that are most affected are the "pfdat" and "zone" commands. Page 20