NAME

       "LaTeXML::Core::Token" - representation of a Token: a pair of character and category code (catcode); It
       extends LaTeXML::Common::Object.

   Exported functions
       "$catcode = CC_ESCAPE;"
           Constants for the category codes:

             CC_BEGIN, CC_END, CC_MATH, CC_ALIGN, CC_EOL,
             CC_PARAM, CC_SUPER, CC_SUB, CC_IGNORE,
             CC_SPACE, CC_LETTER, CC_OTHER, CC_ACTIVE,
             CC_COMMENT, CC_INVALID, CC_CS.

           [The last 2 are (apparent) extensions, with catcodes 16 and 17, respectively].

       "$token = Token($string,$cc);"
           Creates  a LaTeXML::Core::Token with the given content and catcode.  The following shorthand versions
           are also exported for convenience:

             T_BEGIN, T_END, T_MATH, T_ALIGN, T_PARAM,
             T_SUB, T_SUPER, T_SPACE, T_LETTER($letter),
             T_OTHER($char), T_ACTIVE($char),
             T_COMMENT($comment), T_CS($cs)

       "@tokens = Explode($string);"
           Returns a list of the tokens corresponding to the characters in $string.   All  tokens  have  catcode
           CC_OTHER, except for spaces which have catcode CC_SPACE.

       "@tokens = ExplodeText($string);"
           Returns  a  list  of the tokens corresponding to the characters in $string.  All (roman) letters have
           catcode CC_LETTER, all others have catcode CC_OTHER, except for spaces which have catcode CC_SPACE.

       "UnTeX($object, $suppress_linebreaks);"
           Converts $object to a string containing TeX that created it (or could have).  Note that this  is  not
           necessarily the original TeX code; expansions or other substitutions may have taken place.

           Line-breaking  of  the generated TeX can be explicitly requested or disabled by passing 0 or 1 as the
           second $suppress_linebreaks argument.  The default behavior of line-breaking  is  controlled  by  the
           global State value "SUPPRESS_UNTEX_LINEBREAKS".

   Methods
       "@tokens = $object->unlist;"
           Return a list of the tokens making up this $object.

       "$string = $object->toString;"
           Return a string representing $object.

       "$string = $token->getCSName;"
           Return  the  string  or  character  part  of  the $token; for the special category codes, returns the
           standard string (eg. "T_BEGIN->getCSName" returns "{").

       "$string = $token->getString;"
           Return the string or character part of the $token.

       "$code = $token->getCharcode;"
           Return the character code of the character part of the $token, or 256 if it is a control sequence.

       "$code = $token->getCatcode;"
           Return the catcode of the $token.

AUTHOR

       pBruce Miller <bruce.miller@nist.gov>

COPYRIGHT

       Public domain software, produced as part of work done by the United States Government &  not  subject  to
       copyright in the US.

perl v5.38.2                                       2024-02-27                          LaTeXML::Core::Token(3pm)