NAME

       tmx-tokenize - tokenizes translation units on a tmx file.

SYNOPSIS

          tmx-tokenize file.tmx  # creates t_file.tmx

          tmx-tokenize -o=out.tmx file.tmx

DESCRIPTION

       Although this script is bundled in "XML::TMX", it has a soft dependency on
       "Lingua::FreeLing3". Soft means that the dependency is not ensured at install time, and
       other features of the module can still be used without "Lingua::FreeLing3". Nevertheless,
       if you want to use this tool you should install that module.

       At the moment the supported languages are the same as supported by FreeLing3: English,
       Spanish, Russian, Portuguese and Italian.

       It your TMX file includes any other language, they will be maintained without a change.
       This behavior can change in the future, as a basic regexp based tokenizer might be
       implemented.

SEE ALSO

       XML::TMX, Lingua::FreeLing3

AUTHOR

       Alberto Manuel Brandão Simões, <ambs@cpan.org>

COPYRIGHT AND LICENSE

       Copyright (C) 2012 by Alberto Manuel Brandão Simões