Provided by: id-utils_4.6+git20120811-4ubuntu1_amd64
NAME
xtokid - Print all tokens found in a source file
SYNOPSIS
xtokid [OPTION]... [FILE]...
DESCRIPTION
Print all tokens found in a source file. -i, --include=LANGS include languages in LANGS (default: "C C++ asm") -x, --exclude=LANGS exclude languages in LANGS -l, --lang-option=L:OPT pass OPT as a default for language L (see below) -m, --lang-map=MAPFILE use MAPFILE to map file names onto source language -d, --default-lang=LANG make LANG the default source language -p, --prune=NAMES exclude the named files and/or directories --files0-from=F tokenize only the files specified by NUL-terminated names in file F --help display this help and exit --version output version information and exit The following arguments apply to the language-specific scanners: C language: -k,--keep=CHARS Allow CHARS in single-token strings, keep the result -i,--ignore=CHARS Allow CHARS in single-token strings, toss the result -u,--strip-underscore Strip a leading underscore from single-token strings C++ language: -k,--keep=CHARS Allow CHARS in single-token strings, keep the result -i,--ignore=CHARS Allow CHARS in single-token strings, toss the result -u,--strip-underscore Strip a leading underscore from single-token strings Java language: -k,--keep=CHARS Allow CHARS in single-token strings, keep the result -i,--ignore=CHARS Allow CHARS in single-token strings, toss the result -u,--strip-underscore Strip a leading underscore from single-token strings Assembly language: -c,--comment=CHARS Any of CHARS starts a comment until end-of-line -k,--keep=CHARS Allow CHARS in tokens, and keep the result -i,--ignore=CHARS Allow CHARS in tokens, and toss the result -u,--strip-underscore Strip a leading underscore from tokens -n,--no-cpp Don't handle C pre-processor directives Text language: -i,--include=CHAR-CLASS Treat characters of CHAR-CLASS as token constituents -x,--exclude=CHAR-CLASS Treat characters of CHAR-CLASS as token delimiters Perl language: -i,--include=CHAR-CLASS Treat characters of CHAR-CLASS as token constituents -x,--exclude=CHAR-CLASS Treat characters of CHAR-CLASS as token delimiters -d,--dtags Include documentation tags Lisp language:
REPORTING BUGS
Report bugs to bug-idutils@gnu.org
SEE ALSO
The full documentation for the id-utils package is maintained as a Texinfo manual. If the info and id-utils packages are properly installed, the command info 'ID database' should give you access to the complete manual.