Provided by: fcode-utils_1.0.2-8_amd64 bug

NAME

       toke - OpenBIOS tokenizer

SYNOPSIS

       toke [options] forth-file

DESCRIPTION

       toke is an open-source FCode Tokenizer from the OpenBIOS project.

OPTIONS

       -h, -? Print a brief help message and then exit.

       -i, --ignore-errors
              Generate a Binary Output even if errors were reported.

       -l, --load-list
              Collect the names of floaded files into an FLoad-List file. The names collected are
              in the same form as they were presented in the fload statements.
              The name of the FLoad-List File is derived from the name of the binary output file,
              by  replacing  its  extension  with .fl , or, if the binary output file name had no
              extension, merely appending the extension .fl.
              The binary output file name used for this purpose is either the  one  specified  on
              the command line, or the one created by default.

       -o, --output-name OutputFileName,
              Direct  the  binary output (FCode result of Tokenization) to the named file instead
              of to the default-named file. This option is not valid when  multiple  input  files
              are named.

       -P, --dependencies
              Collect   the  fully-resolved  pathnames  of  floaded  and  ENCODEd  files  into  a
              dependency-list file.  The names collected are in the form that is presented to the
              host  operating system: shell environment Variables and related expressions will be
              fully expanded, and the directory within the include-list in  which  the  file  was
              found will be attached.
              The  name  of  the  dependency-list file will be the same as that of the FLoad-list
              file, except that its extension will be .P instead of .fl.

       -v, --verbose
              Print additional messages (including advisories) during tokenization.

       -I, --Include directory
              This tokenizer supports the  notion  of  an  include-list.  The  user  creates  the
              include-list  by specifying a number of -I directory pairs on the command-line. All
              file-reads, whether for an fload command or an encode-file directive, will  involve
              a  search  for  the  named file through the directories of the include-list, in the
              order they were supplied on the command-line.
              If no include-list is created, file-reads  are  relative  to  the  current  working
              Directory.  If  an  include-list  is  created,  file-reads  are  restricted  to the
              directories within it.  For the current working directory to  be  included  in  the
              file-search,  it  must  be  specified  explicitly.  -I.  will accomplish that quite
              effectively.

       -T, --Trace symbol
              This tokenizer supports the notion of a trace-list. The User creates the trace-list
              by specifying a number of -T symbol pairs on the command-line.
              When  a  name  is defined, whether as an FCode, an alias, a Macro or anything else,
              either in normal tokenization mode or "Tokenizer  Escape"‐mode,  if  it  matches  a
              symbol  that  has been added to the trace list, a trace note message will be issued
              indicating that a definition of that name has been created.  Subsequent trace  note
              Messages will be issued when the definition of that name is invoked.
              This  trace-symbols  feature  can be helpful during maintenance of legacy code, for
              instance, when multiple symbols carry the same name.

       -d, --define Symbol[=Value]
              Define a command-line symbol and optionally, assign a value to it. If you wish  the
              value  to  contain spaces or quotes, you can accomplish that using the shell escape
              conventions. This sequence may be  repeated.  Once  a  Symbol  is  defined  on  the
              command-line,  it  stays  in  effect  for  the  duration  of  the  entire  batch of
              tokenizations (i.e., if there are multiple input files named on the command  line).
              Command-line  Symbols  can  be  tested for purposes of conditional tokenization, or
              their assigned values can be evaluated.

       -f, --flag [no]<FlagName>
              The tokenizer recognizes a specific set  of  special-feature  flag-names;  each  is
              associated  with a specific non-standard variant behavior. Pass the flag-name as an
              argument to the -f switch to enable the behavior; to disable it, precede the  flag-
              name with the optional string no.
              The  settings  of  the  special-feature flags can also be changed or displayed from
              within the source input file.
              The special-feature flags are all initially set to be enabled, except where noted.
              The flag-names and their associated special-features are as follows:

              Local-Values
                     Support IBM-style Local Values ("LV"s). Initially disabled.

              LV-Legacy-Separator
                     Allow Semicolon for Local Values Separator ("Legacy").

              LV-Legacy-Message
                     Display a Warning Message  when  Semicolon  is  used  as  the  Local  Values
                     Separator.

              ABORT-Quote
                     Allow ABORT" macro.

              Sun-ABORT-Quote
                     ABORT" with implicit IF ... THEN

              Abort-Quote-Throw
                     Use -2 THROW, rather than ABORT, in an Abort" phrase

              String-remark-escape
                     Allow "\ (Quote-Backslash) to interrupt string parsing.

              Hex-remark-escape
                     Allow \ (Backslash) to interrupt hex-sequence parsing within a string.

              C-Style-string-escape
                     Allow  the  C-style  String-Escape  pairs  \n  \t  and \xx\ to be treated as
                     special characters in string parsing.

              Always-Headers
                     Override occurrences of the standard directive headerless in the source with
                     -- effectively -- headers to make all definitions have a header. Occurrences
                     of the directive external will continue to behave in  the  standard  manner.
                     Initially disabled.

              Always-External
                     All  definitions  will  be  made  as  though  under  the external directive;
                     occurrences of either Standard directive headerless or headers in the source
                     will  be  overridden.  This  special-feature  flag  will  also  override the
                     Always-Headers special-feature  flag  in  the  event  that  both  have  been
                     specified. Initially disabled.

              Warn-if-Duplicate
                     Display  a  WARNING  message  whenever  a  definition  is  made  whose  name
                     duplicates that of an existing definition.  Disabling this flag will suspend
                     the  duplicate-names  test  globally, until it is re-enabled. A Directive is
                     supported that will suspend the test for  the  duration  of  only  a  single
                     definition, without affecting global behavior.

              Obsolete-FCode-Warning
                     Display  a  WARNING  message  whenever an FCode function is invoked that the
                     Standard identifies as obsolete.

              Trace-Conditionals
                     Issue  advisory  messages  about  the  state  of  conditional  tokenization.
                     (Remember that advisory messages are displayed only if the verbose option -v
                     is set.) Initially disabled.

              Upper-Case-Token-Names
                     Lower-Case-Token-Names When  outputting  the  names  of  headered  functions
                     (token-names)  to  the  binary  output  file, override the character-case in
                     which the names appeared in the source, and convert them to Upper- or Lower-
                     ‐Case, respectively. (These flags do not affect text string sequences, whose
                     character-case is always preserved.)  Initially disabled.

              Big-End-PCI-Rev-Level
                     Save the Revision Level of the Vendor's ROM field of the PCI Header in  big-
                     endian byte-order, rather than little-endian as per the general PCI Standard
                     convention. (This flag does not affect any other field
                      of the PCI Header). Initially disabled.

              Ret-Stk-Interp
                     Allow return-stack operations  during  interpretation.  While  the  standard
                     specifies  that  usage of the operators >r, r@, and r> while interpreting is
                     allowed, actual practice in the industry is  inconsistent.   Developers  who
                     wish to take a more cautious approach to this question can disable this flag
                     so that any attempt to use the operators >r, r@, and r> in the  interpreting
                     state will generate an ERROR Message.

       Also,  the  pseudo-flag-name help will cause a list of the flag-names and their associated
       special-features to be printed.

SEE ALSO

       detok(1).

AUTHORS

       Stefan Reinauer <stepan@openbios.org>
       David L. Paktor <dlpaktor@us.ibm.com>

       This manual page was  written  by  Aurelien  Jarno  <aurel32@debian.org>  for  the  Debian
       GNU/Linux system (but may be used by others).

                                           May 15, 2007                                   TOKE(1)