White Paper: Red Hat Crash Utility


< Prev Contents Next >

Command Extensions

Frequently users wish to add an additional option to an existing crash command, or add a brand new command, in order to address a kernel issue they are debugging or developing. For those reasons, the crash utility was designed with extensibility in mind. There are two ways to add new functionality:

  1. adding new code and compiling it into the crash executable,
  2. creating a shared object library that can be dynamically loaded by using the extend command.

This section consists of a quick guide that describes how to get started using both methods.


Adding new code and compiling it into the crash executable

The current set of crash commands can be seen by entering the help command with no arguments:

crash> help

*              files          mod            runq           union
alias          foreach        mount          search         vm
ascii          fuser          net            set            vtop
bt             gdb            p              sig            waitq
btop           help           ps             struct         whatis
dev            irq            pte            swap           wr
dis            kmem           ptob           sym            q
eval           list           ptov           sys
exit           log            rd             task
extend         mach           repeat         timer

crash version: 3.7-1   gdb version: Red Hat Linux (5.3post-0.20021129.36rh)
For help on any command above, enter "help ".
For help on input options, enter "help input".
For help on output options, enter "help output".

crash> 

For each command in the menu above, there is an entry in a data structure in the file global_data.c, which is located in the top-level directory of the crash source code tree:

/*
 *  To add a new command, declare it in defs.h and enter it in this table.
 */

struct command_table_entry base_command_table[] = {
	{"*", 	    cmd_pointer, help_pointer, 0},
	{"alias",   cmd_alias,   help_alias,   0},
        {"ascii",   cmd_ascii,   help_ascii,   0},
        {"bt",      cmd_bt,      help_bt,      REFRESH_TASK_TABLE},
	{"btop",    cmd_btop,    help_btop,    0},
	{"dev",     cmd_dev,     help_dev,     0},
	{"dis",     cmd_dis,     help_dis,     0},
	{"eval",    cmd_eval,    help_eval,    0},
	{"exit",    cmd_quit,    help_exit,    0},
	{"extend",  cmd_extend,  help_extend,  0},
	{"files",   cmd_files,   help_files,   REFRESH_TASK_TABLE},
	{"foreach", cmd_foreach, help_foreach, REFRESH_TASK_TABLE},
	{"fuser",   cmd_fuser,   help_fuser,   REFRESH_TASK_TABLE},
	{"gdb",     cmd_gdb,     help_gdb,     REFRESH_TASK_TABLE},
        {"help",    cmd_help,    help_help,    0},
	{"irq",     cmd_irq,     help_irq,     0},
	{"kmem",    cmd_kmem,    help_kmem,    0},
	{"list",    cmd_list,    help__list,   REFRESH_TASK_TABLE},
	{"log",     cmd_log,     help_log,     0},
	{"mach",    cmd_mach,    help_mach,    0},
	{"mod",     cmd_mod,     help_mod,     0},
	{"mount",   cmd_mount,   help_mount,   0},
	{"net",	    cmd_net,	 help_net,     REFRESH_TASK_TABLE},
	{"p",       cmd_p,       help_p,       0},
	{"ps",      cmd_ps,      help_ps,      REFRESH_TASK_TABLE},
	{"pte",     cmd_pte,     help_pte,     0},
	{"ptob",    cmd_ptob,    help_ptob,    0},
	{"ptov",    cmd_ptov,    help_ptov,    0},
        {"q",       cmd_quit,    help_quit,    0},
        {"rd",      cmd_rd,      help_rd,      0},
	{"repeat",  cmd_repeat,  help_repeat,  0},
	{"runq",    cmd_runq,    help_runq,    REFRESH_TASK_TABLE},
        {"search",  cmd_search,  help_search,  0},
        {"set",     cmd_set,     help_set,     REFRESH_TASK_TABLE},
        {"sig",     cmd_sig,     help_sig,     REFRESH_TASK_TABLE},
        {"struct",  cmd_struct,  help_struct,  0},
	{"swap",    cmd_swap,    help_swap,    0},
        {"sym",     cmd_sym,     help_sym,     0},
        {"sys",     cmd_sys,     help_sys,     REFRESH_TASK_TABLE},
        {"task",    cmd_task,    help_task,    REFRESH_TASK_TABLE},
	{"test",    cmd_test,    NULL,         HIDDEN_COMMAND},
        {"timer",   cmd_timer,   help_timer,   0},
	{"union",   cmd_union,   help_union,   0},
	{"vm",      cmd_vm,      help_vm,      REFRESH_TASK_TABLE},
	{"vtop",    cmd_vtop,    help_vtop,    REFRESH_TASK_TABLE},
	{"waitq",   cmd_waitq,   help_waitq,   REFRESH_TASK_TABLE},
	{"whatis",  cmd_whatis,  help_whatis,  0},
	{"wr",      cmd_wr,      help_wr,      0},
	{(char *)NULL}
};

Each entry consists of the following simple data structure:

struct command_table_entry {               /* one for each command in menu */
        char *name;
        cmd_func_t func;
        char **help_data;
        ulong flags;
};

The structure members consist of:

name The character string that appears in the help menu output. It must be unique among other crash commands, and preferably should not clash with any gdb command.
func A pointer to the function that performs the command. To add an option to an existing command, find the file that contains this function, and modify it as desired.
help_data A pointer to an array of character strings that make up the help data for that command. Although optional, it's certainly useful to create help data for any new commands or options.
flags If REFRESH_TASK_TABLE, the set of running tasks on a live system will be updated just prior to executing the command.
If HIDDEN_COMMAND, the command will not be shown in the help menu. The only command with this flag is the test command, whose function cmd_test() exists solely as an builtin aid for quickly developing new or temporary commands.

For a newly-written command to appear in the help menu, it simply requires a reference to it in the structure above. To test a new command without adding it to the menu, use the hidden "test" command, found in test.c:

void
cmd_test(void)
{
        int c;

        while ((c = getopt(argcnt, args, "")) != EOF) {
                switch(c)
                {
                default:
                        argerrs++;
                        break;
                }
        }

        if (argerrs)
                cmd_usage(pc->curcmd, SYNOPSIS);

        while (args[optind]) {
                ;
                optind++;
        }
} 

The test command contains the basic template used by crash commands to accept dash-arguments, which are fielded by the getopt() routine, while all other command line arguments are fielded in the while loop. To add an option to the test command (or any other existing command), simply fit it into the appropriate argument-gathering mechanism. Or, for that matter, if no arguments are required, put the functionality at the end of the command's function. Here is a trivial example of a change to the cmd_test() function, with the modifications highlighted:

void
cmd_test(void)
{
        int c;

        while ((c = getopt(argcnt, args, "xa:")) != EOF) {
                switch(c)
                {
                case 'x':
                        fprintf(fp, "arg: -x\n");
                        break;
                case 'a':
                        fprintf(fp, "arg: -a %s\n", optarg);
                        break;
                default:
                        argerrs++;
                        break;
                }
        }

        if (argerrs)
                cmd_usage(pc->curcmd, SYNOPSIS);

        while (args[optind]) {
                fprintf(fp, "arg: %s\n", args[optind]);
                ;
                optind++;
        }

        fprintf(fp, "do test work here...\n");
}

Then re-compile crash by entering make:

# make
TARGET: X86
 CRASH: 3.7-1
   GDB: 5.3post-0.20021129.36rh

cc -c -g -DX86 -D_FILE_OFFSET_BITS=64 build_data.c
cc -c -g -DX86 -D_FILE_OFFSET_BITS=64 test.c
ar -rs crashlib.a main.o tools.o global_data.o memory.o filesys.o help.o task.o
build_data.o kernel.o test.o gdb_interface.o net.o dev.o alpha.o x86.o ppc.o ia6
4.o s390.o s390x.o extensions.o remote.o va_server.o va_server_v1.o symbols.o cm
dline.o lkcd_common.o lkcd_v1.o lkcd_v2_v3.o lkcd_v5.o lkcd_v7.o s390_dump.o s39
0x_dump.o netdump.o lkcd_x86_trace.o unwind.o
rm -f gdb
gcc -g -O2         \
-o `cat mergeobj` main.o cli-dump.o cli-decode.o cli-script.o cli-cmds.o cli-set
show.o cli-utils.o mi-out.o mi-console.o mi-cmds.o mi-cmd-var.o mi-cmd-break.o m
i-cmd-stack.o mi-cmd-disas.o mi-main.o mi-parse.o mi-getopt.o libgdb.a \
           ../bfd/libbfd.a ../readline/libreadline.a ../opcodes/libopcodes.a  ..
/libiberty/libiberty.a -lncurses     -lm   ../libiberty/libiberty.a  -ldl -rdyna
mic `cat mergelibs`
#

Then run the test command with the tree possible argument types:

crash> test -x -a dasharg this that the other
arg: -x
arg: -a dasharg
arg: this
arg: that
arg: the
arg: other
do test work here...
crash>

The easiest way to implement a new command's functionality is to use an existing command as a template. There is a broad range of utility routines that handle argument strings, read memory, access data structures and their members, display data, and so on, that obviate the need to reinvent the wheel to accomplish a task. Look at what other similar commands do, and copy their mechanisms.


Creating a shared object library and loading it with extend

While adding a new command and/or command option in the manner above is useful, it would require the same integration mechanism with each subsequent release of the crash utility. Since that could become tedious, another extension mechanism exists in which share objects containing one or more crash commands can be written, and then dynamically attached to a running crash session, using the extend command. Once loaded, the command(s) in the shared object library automatically appear in the help menu, as if they were compiled into the crash executable. As an quick aid in creating a shared object, the help page for the extend contains an example C program tagged onto the end, which adds a new echo command (which simply echoes back all arguments). The C program piece can be cut and pasted into a file, say extlib.c for example, and then compiled like so:

# gcc -nostartfiles -shared -rdynamic -o extlib.so extlib.c
#

The resultant extlib.so file may be dynamically linked into crash during runtime using the extend command:

crash> extend extlib.so
./extlib.so: shared object loaded
crash>

Or, to automatically load the shared library during crash session initialization, put the following string into a .crashrc file located in the current directory, or in the user's $HOME directory:

extend extlib.so

Here is the help menu once the library is loaded; note the integration of the new echo command:

crash> help

*              extend         mach           repeat         timer
alias          files          mod            runq           union
ascii          foreach        mount          search         vm
bt             fuser          net            set            vtop
btop           gdb            p              sig            waitq
dev            help           ps             struct         whatis
dis            irq            pte            swap           wr
echo           kmem           ptob           sym            q
eval           list           ptov           sys
exit           log            rd             task

crash version: 3.7-1    gdb version: Red Hat Linux (5.3post-0.20021129.36rh)
For help on any command above, enter "help ".
For help on input options, enter "help input".
For help on output options, enter "help output".

With this extension mechanism, the most that would be required to use the shared library with subsequent versions of crash would be a simple re-compile of the extlib.c file.


< Prev Contents Next >