Provided by: hunspell_1.1.9-1_i386 bug

NAME

       hunspell - format of Hunspell dictionaries and affix files

DESCRIPTION

       Hunspell(1)  requires  two  files  to  define  the  language that it is
       spellchecking.  The first file is a dictionary containing words for the
       language,  and  the  second is an "affix" file that defines the meaning
       of special flags in the dictionary.

       A dictionary file (*.dic) contains a list of words, one per line.   The
       first  line of the dictionaries (except personal dictionaries) contains
       the approximate word count (for optimal hash memory  size).  Each  word
       may  optionally  be  followed  by  a slash ("/") and one or more flags,
       which represents affixes or special attributes.  Dictionary  words  can
       contain  also  slashes  with  the  ""  syntax. Default flag format is a
       single (usually alphabetic) character. In a Hunspell  dictionary  file,
       there is also an optional morphological field separated by tabulator.

       Morphological desciptions have custom format.

       Personal  dictionaries  are  simple  word lists, but with optional word
       patterns for affixation, separated by a slash:

              foo
              Foo/Simpson

       In this example, "foo" and "Foo" are personal words, plus Foo  will  be
       recognised with affixes of Simpson (Foo’s etc.).

       An  affix  file  (*.aff) may contain a lot of optional attributes.  For
       example, SET is used for setting the character encodings of affixes and
       dictionary files.  TRY sets the change characters for suggestions.  REP
       sets  a  replacement  table  for  multiple  character  corrections   in
       suggestion  mode.   PFX and SFX defines prefix and suffix classes named
       with affix flags.

       The following affix file  example  defines  UTF-8  character  encoding.
       ‘TRY’ suggestions differ from the bad word with an English letter or an
       apostrophe. With these REP definitions, Hunspell can suggest the  right
       word  form,  when the misspelled word contains f instead of ph and vice
       versa.

              SET UTF-8
              TRY esianrtolcdugmphbyfvkwzESIANRTOLCDUGMPHBYFVKWZ’

              REP 2
              REP f ph
              REP ph f

              PFX A Y 1
              PFX A 0 re .

              SFX B Y 2
              SFX B 0 ed [^y]
              SFX B y ied y

       There are two affix classes in the dictionary. Class A defines an ‘re-’
       prefix.  Class  B defines two ‘-ed’ suffixes. First suffix can be added
       to a word if the last character of the word isn’t ‘y’.   Second  suffix
       can be added to words terminated with an ‘y’.  (See details later.) The
       following dictionary file uses these affix classes.

              3
              hello
              try/B
              work/AB

       All accepted words with this example: hello, try, tried, work,  worked,
       rework, reworked.

GENERAL OPTIONS

       SET encoding
              Set  character  encoding  of  words  and  morphemes in affix and
              dictionary  files.   Possible   values:   UTF-8,   ISO8859-1   -
              ISO8859-10,  ISO8859-13 - ISO8859-15, KOI8-R, KOI8-U, microsoft-
              cp1251, ISCII-DEVANAGARI.

       FLAG value
              Set flag type.  Default  type  is  the  extended  ASCII  (8-bit)
              character.    ‘UTF-8’   parameter  sets  UTF-8  encoded  Unicode
              character flags.  The ‘long’  value  sets  the  double  extended
              ASCII  character  flag  type,  the ‘num’ sets the decimal number
              flag type. Decimal flags numbered from 1 to 65535, and  in  flag
              fields  are  separated  by  comma.  BUG: UTF-8 flag type doesn’t
              work on ARM platform.

       COMPLEXPREFIXES
              Set twofold prefix stripping (but single suffix  stripping)  for
              agglutinative languages with right-to-left writing system.

       LANG langcode
              Set  language  code.  In Hunspell may be language specific codes
              enabled by LANG code. At present there are az_AZ,  hu_HU,  TR_tr
              specific codes in Hunspell (see the source code).

       IGNORE characters
              Ignore  characters  from  dictionary  words,  affixes  and input
              words.  Useful for optional characters,  as  Arabic  diacritical
              marks (Harakat).

       AF number_of_flag_vector_aliases

       AF flag_vector
              Hunspell  can substitue affix flag sets with a natural number in
              affix  rules  (alias  compression).  First  example  with  alias
              compression:

              3
              hello
              try/1
              work/2

       AF definitions in the affix file:

              SET UTF-8
              TRY esianrtolcdugmphbyfvkwzESIANRTOLCDUGMPHBYFVKWZ’
              AF 2
              AF A
              AF AB

       See also tests/alias* examples.

       Note:  If  affix file contains the FLAG parameter, define it before the
       AF definitions.

       Note II: Use makealias utility in Hunspell distribution to compress aff
       and dic files.

       AM number_of_morphological_description_aliases

       AM morphological_description
              Hunspell  can  substitue  also morphological descriptions with a
              natural  number  in  affix  rules  (alias   compression).    See
              tests/alias* examples.

OPTIONS FOR SUGGESTION

       TRY characters
              Hunspell  can  suggest right word forms, when those differs from
              the bad form by one TRY character. The parameter of TRY is  case
              sensitive.

       NOSUGGEST flag
              Words  signed  with  NOSUGGEST  flag are not suggested. Proposed
              flag for vulgar and obscene words.

       MAXNGRAMSUGS num
              Set number of n-gram suggestions. Value 0 switches  off  the  n-
              gram suggestions.

       NOSPLITSUGS
              Disable split-word suggestions.

       SUGSWITHDOTS
              Add  dot(s)  to suggestions, if input word terminates in dot(s).
              (Not for OpenOffice.org dictionaries, because OpenOffice.org has
              an automatic dot expansion mechanism.)

       REP number_of_replacement_definitions

       REP what replacement
              We  can  define  language-dependent  phonetic information in the
              affix file (.aff)  by a replacement table.   First  REP  is  the
              header of this table and one or more REP data line are following
              it. With this table, Hunspell can suggest the  right  forms  for
              the  typical  faults of spelling when the incorrect form differs
              by more, than 1 letter from  the  right  form.   For  example  a
              possible   English   replacement   table  definition  to  handle
              misspelled consonants:

              REP 8
              REP f ph
              REP ph f
              REP f gh
              REP gh f
              REP j dg
              REP dg j
              REP k ch
              REP ch k

       Note: It’s very useful to define replacements for the most typical one-
       character  mistakes,  too:  with  REP  you can add higher priority to a
       subset of the TRY suggestions (suggestion  list  begins  with  the  REP
       suggestions).

       Note  II:  Replacement  table  can be used for a stricter compound word
       checking (forbidding generated compound words, if they are also  simple
       words with typical fault, see CHECKCOMPOUNDREP).

       MAP number_of_map_definitions

       MAP string_of_related_chars
              We  can define language-dependent information on characters that
              should be considered related (ie. nearer than other chars not in
              the  set)  in  the  affix file (.aff)  by a character map table.
              With this table, Hunspell can suggest the right forms for words,
              which  incorrectly  choose  the  wrong letter from a related set
              more than once in a word.

              For example a possible mapping could be for the German  umlauted
              ü  versus  the  regular  u;  the word Frühstück really should be
              written with umlauted u’s and not regular ones

              MAP 1
              MAP uü

OPTIONS FOR COMPOUNDING

       BREAK number_of_break_definitions

       BREAK character_or_character_sequence
              Define break points for breaking words and checking  word  parts
              separately.   Rationale:  useful  for  compounding  with joining
              character or strings (for example, hyphen in English and  German
              or  hyphen and n-dash in Hungarian).  Dashes are often bad break
              points for  tokenization,  because  compounds  with  dashes  may
              contain  not  valid parts, too.)  With BREAK, Hunspell can check
              both side of these compounds, breaking the words at  dashes  and
              n-dashes:

              BREAK 2
              BREAK -
              BREAK --    # n-dash

       Breaking  are recursive, so foo-bar, bar-foo and foo-foo--bar-bar would
       be valid compounds.

       Note: COMPOUNDRULE is better (or will be better)  for  handling  dashes
       and other  compound joining characters or character strings. Use BREAK,
       if you want check words with dashes or  other  joining  characters  and
       there is no time or possibility to describe precise compound rules with
       COMPOUNDRULE (COMPOUNDRULE has handled only the last suffixation of the
       compound word yet).

       Note  II:  For  command  line spell checking, set WORDCHARS parameters:
       WORDCHARS --- (see tests/break.*) example

       COMPOUNDRULE number_of_compound_definitions

       COMPOUNDRULE compound_pattern
              Define custom compound patterns with a regex-like  syntax.   The
              first  COMPOUNDRULE is a header with the number of the following
              COMPOUNDRULE definitions.  Compound  patterns  consist  compound
              flags and star or question mark meta characters. A flag followed
              by a ‘*’ matches a word sequence of 0 or more matches  of  words
              signed  with  this  compound  flag.   A  flag  followed by a ‘?’
              matches a word sequence of 0 or 1 matches of a word signed  with
              this compound flag.  See tests/compound*.* examples.

              Note:  ‘*’  and  ‘?’  metacharacters  work only with the default
              8-bit character and the UTF-8 FLAG types.

              Note II: COMPOUNDRULE flags haven’t  been  compatible  with  the
              COMPOUNDFLAG,  COMPOUNDBEGIN, etc. compound flags yet (use these
              flags on different words).

       COMPOUNDMIN num
              Minimum length of words in compound words.  Default value  is  3
              letters.

       COMPOUNDFLAG flag
              Words  signed with COMPOUNDFLAG may be in compound words (except
              when word shorter than COMPOUNDMIN). Affixes  with  COMPOUNDFLAG
              also permits compounding of affixed words.

       COMPOUNDBEGIN flag
              Words  signed with COMPOUNDBEGIN (or with a signed affix) may be
              first elements in compound words.

       COMPOUNDLAST flag
              Words signed with COMPOUNDLAST (or with a signed affix)  may  be
              last elements in compound words.

       COMPOUNDMIDDLE flag
              Words signed with COMPOUNDMIDDLE (or with a signed affix) may be
              middle elements in compound words.

       ONLYINCOMPOUND flag
              Suffixes signed with ONLYINCOMPOUND flag may be only  inside  of
              compounds  (Fuge-elements  in German, fogemorphemes in Swedish).
              ONLYINCOMPOUND    flag    works    also    with    words    (see
              tests/onlyincompound.*).

       COMPOUNDPERMITFLAG flag
              Prefixes are allowed at the beginning of compounds, suffixes are
              allowed at the  end  of  compounds  by  default.   Affixes  with
              COMPOUNDPERMITFLAG may be inside of compounds.

       COMPOUNDFORBIDFLAG flag
              Suffixes  with this flag forbid compounding of the affixed word.

       COMPOUNDROOT flag
              COMPOUNDROOT flag signs the compounds in the dictionary (Now  it
              is used only in the Hungarian language specific code).

       COMPOUNDWORDMAX number
              Set   maximum  word  count  in  a  compound  word.  (Default  is
              unlimited.)

       CHECKCOMPOUNDDUP
              Forbid word duplication in compounds (eg. foofoo).

       CHECKCOMPOUNDREP
              Forbid compounding, if the (usually bad) compound word may be  a
              non  compound  word  with a REP fault. Useful for languages with
              ‘compound friendly’ orthography.

       CHECKCOMPOUNDCASE
              Forbid upper case characters at word bound in compounds.

       CHECKCOMPOUNDTRIPLE
              Forbid compounding, if compound  word  contains  triple  letters
              (eg.  foo|ox  or  xo|oof).   Bug:  missing  multi-byte character
              support  in  UTF-8  encoding  (works  only   for   7-bit   ASCII
              characters).

       CHECKCOMPOUNDPATTERN number_of_checkcompoundpattern_definitions

       CHECKCOMPOUNDPATTERN endchars beginchars
              Forbid   compounding,  if  first  word  in  compound  ends  with
              endchars, and next word begins with beginchars.

       COMPOUNDSYLLABLE max_syllable vowels
              Need  for  special  compounding  rules  in   Hungarian.    First
              parameter  is  the  maximum  syllable  number,  that may be in a
              compound, if words in compounds are more  than  COMPOUNDWORDMAX.
              Second   parameter  is  the  list  of  vowels  (for  calculating
              syllables).

       SYLLABLENUM flags
              Need for special compounding rules in Hungarian.

OPTIONS FOR AFFIX CREATION

       PFX flag cross_product number

       PFX flag stripping prefix condition morphological_description

       SFX flag cross_product number

       SFX flag stripping suffix condition morphological_description
              An affix is either a prefix or a suffix attached to  root  words
              to  make other words. We can define affix classes with arbitrary
              number affix rules.  Affix classes are signed with affix  flags.
              The  first  line of an affix class definition is the header. The
              fields of an affix class header:

              (0) Option name (PFX or SFX)

              (1) Flag (name of the affix class)

              (2) Cross product (permission to combine prefixes and suffixes).
              Possible values: Y (yes) or N (no)

              (3) Line count of the following rules.

              Fields of an affix rules:

              (0) Option name

              (1) Flag

              (2) stripping characters from beginning (at prefix rules) or end
              (at suffix rules) of the word

              (3)  affix  (optionally  with  flags  of  continuation  classes,
              separated by a slash)

              (4) condition.

              Zero stripping or affix are indicated by zero. Zero condition is
              indicated  by  dot.   Condition   is   a   simplified,   regular
              expression-like  pattern, which must be met before the affix can
              be applied. (Dot signs an  arbitrary  character.  Characters  in
              braces  sign  an  arbitrary character from the character subset.
              Dash hasn’t got special meaning, but  circumflex  (^)  next  the
              first brace sets the complementer character set.)

              (5) Custom morphological description.

OTHER OPTIONS

       CIRCUMFIX flag
              Affixes  signed  with  CIRCUMFIX flag may be on a word when this
              word also has a prefix with CIRCUMFIX flag and vice versa.

       FORBIDDENWORD flag
              This flag signs forbidden word form. Because affixed  forms  are
              also  forbidden,  we  can  substract  a  subset  from set of the
              accepted affixed and compound words.

       KEEPCASE flag
              Forbid uppercased and capitalized forms  of  words  signed  with
              KEEPCASE  flags.  Useful  for special ortographies (measurements
              and currency often keep their  case  in  uppercased  texts)  and
              writing systems (eg. keeping lower case of IPA characters).

              Note:  With  CHECKSHARPS  declaration,  words  with  sharp s and
              KEEPCASE flag may be capitalised and uppercased, but  uppercased
              forms  of  these  words  may  not  contain sharp s, only SS. See
              germancompounding example in the tests directory of the Hunspell
              distribution.

       LEMMA_PRESENT flag
              Generally,  there  are  dictionary  words as lemmas in output of
              morphological  analysis.  Sometimes  dictionary  words  are  not
              lemmas,  but affixed (not real) stems and virtual stems. In this
              case  lemmas  (real  stems)  need  to  put  into   morphological
              description,   and  forbid  not  real  lemmas  in  morphological
              analysis adding LEMMA_PRESENT flag to dictionary words.

       NEEDAFFIX flag
              This flag signs virtual stems in the dictionary.   Only  affixed
              forms  of  these words will be accepted by Hunspell.  Except, if
              the dictionary word has a homonym or a  zero  affix.   NEEDAFFIX
              works  also  with prefixes and prefix + suffix combinations (see
              tests/pseudoroot5.*).

       PSEUDOROOT flag
              Deprecated. (Former name of the NEEDAFFIX option.)

       WORDCHARS characters
              WORDCHARS extends tokenizer of Hunspell command  line  interface
              with  additional word character. For example, dot, dash, n-dash,
              numbers, percent sign are word character in Hungarian.

       CHECKSHARPS
              SS letter pair in uppercased (German) words may  be  upper  case
              sharp  s  (ß).  Hunspell can handle this special casing with the
              CHECKSHARPS   declaration   (see   also   KEEPCASE   flag    and
              tests/germancompounding    example)   in   both   spelling   and
              suggestion.

Morphological analysis

       Hunspell’s affix rules have got an optional  morphological  description
       field.  There is a similar optional field in dictionary file, separated
       by tabulator:

               word/flags    morphology

       We define a simple resource with morphological informations.

       Affix file:

               SFX X Y 1
               SFX X 0 able . +ABLE

       Dictionary file:

               drink/X   [VERB]

       Test file:

               drink
               drinkable

       Test:

               $ hunmorph test.aff test.dic test.txt
               drink:     drink[VERB]
               drinkable: drink[VERB]+ABLE

       You can  see  in  the  example,  that  the  analyzer  concatenates  the
       morphological fields in item and arrangement style.

Twofold suffix stripping

       Ispell’s  original algorithm strips only one suffix. Hunspell can strip
       another one yet.

       The twofold suffix stripping is a significant improvement  in  handling
       of   immense   number  of  suffixes,  that  characterize  agglutinative
       languages.

       Extending the previous example by adding a second suffix (affix class Y
       will be the continuation class of the suffix ‘able’):

               SFX Y Y 1
               SFX Y 0 s . +PLUR

               SFX X Y 1
               SFX X 0 able/Y . +ABLE

       Dictionary file:

               drink/X   [VERB]

       Test file:

               drink
               drinkable
               drinkables

       Test:

               $ hunmorph test.aff test.dic test.txt
               drink:      drink[VERB]
               drinkable:  drink[VERB]+ABLE
               drinkables: drink[VERB]+ABLE+PLUR

       Theoretically  with  the twofold suffix stripping needs only the square
       root  of  the  number  of  suffix  rules,  compared  with  a   Hunspell
       implementation. In our practice, we could have elaborated the Hungarian
       inflectional morphology with twofold suffix stripping.

       Note: In Hunlex preprocessor’s grammar can be use not only twofold, but
       multiple suffix slitting.

Extended affix classes

       Hunspell  can  handle more than 65000 affix classes.  There are two new
       syntax for giving flags in affix and dictionary files.

       FLAG long command sets 2-character flags:

                FLAG long
                SFX Y1 Y 1
                SFX Y1 0 s 1

       Dictionary record with the Y1, Z3, F? flags:

                foo/Y1Z3F?

       FLAG num command sets numerical flags separated by comma:

                FLAG num
                SFX 65000 Y 1
                SFX 65000 0 s 1

       Dictionary example:

                foo/65000,12,2756

Homonyms

       Hunspell’s dictionary can contain repeating elements that are homonyms:

               work/A    [VERB]
               work/B    [NOUN]

       An affix file:

               SFX A Y 1
               SFX A 0 s . +SG3

               SFX B Y 1
               SFX B 0 s . +PLUR

       Test file:

               works

       Test:

               > works
               work[VERB]+SG3
               work[NOUN]+PLUR

       This   feature  also  gives  a  way  to  forbid  illegal  prefix/suffix
       combinations in difficult cases.

Prefix--suffix dependencies

       An  interesting  side-effect  of  multi-step  stripping  is,  that  the
       appropriate treatment of circumfixes now comes for free.  For instance,
       in Hungarian, superlatives are formed by  simultaneous  prefixation  of
       leg-  and suffixation of -bb to the adjective base.  A problem with the
       one-level architecture is that  there  is  no  way  to  render  lexical
       licensing  of  particular  prefixes  and  suffixes  interdependent, and
       therefore incorrect forms are recognized as valid, i.e. *legvn = leg +
       vn  ‘old’.  Until the introduction of clusters, a special treatment of
       the superlative had to be hardwired in the earlier HunSpell code.  This
       may  have been legitimate for a single case, but in fact prefix--suffix
       dependences are ubiquitous in category-changing  derivational  patterns
       (cf.   English   payable,   non-payable   but  *non-pay  or  drinkable,
       undrinkable but *undrink). In simple words, here,  the  prefix  un-  is
       legitimate only if the base drink is suffixed with -able. If both these
       patters are handled by on-line affix rules and affix rules are  checked
       against  the  base only, there is no way to express this dependency and
       the system will necessarily over- or undergenerate.

       In next example, suffix class R have got a prefix ‘continuation’  class
       (class P).

              PFX P Y 1
              PFX P   0 un . [prefix_un]+

              SFX S Y 1
              SFX S   0 s . +PL

              SFX Q Y 1
              SFX Q   0 s . +3SGV

              SFX R Y 1
              SFX R   0 able/PS . +DER_V_ADJ_ABLE

       Dictionary:

              2
              drink/RQ  [verb]
              drink/S   [noun]

       Morphological analysis:

              > drink
              drink[verb]
              drink[noun]
              > drinks
              drink[verb]+3SGV
              drink[noun]+PL
              > drinkable
              drink[verb]+DER_V_ADJ_ABLE
              > drinkables
              drink[verb]+DER_V_ADJ_ABLE+PL
              > undrinkable
              [prefix_un]+drink[verb]+DER_V_ADJ_ABLE
              > undrinkables
              [prefix_un]+drink[verb]+DER_V_ADJ_ABLE+PL
              > undrink
              Unknown word.
              > undrinks
              Unknown word.

Circumfix

       Conditional  affixes implemented by a continuation class are not enough
       for circumfixes, because a circumfix is one  affix  in  morphology.  We
       also need CIRCUMFIX option for correct morphological analysis.

              # circumfixes: ~ obligate prefix/suffix combinations
              # superlative in Hungarian: leg- (prefix) AND -bb (suffix)
              # nagy, nagyobb, legnagyobb, legeslegnagyobb
              # (great, greater, greatest, most greatest)

              CIRCUMFIX X

              PFX A Y 1
              PFX A 0 leg/X .

              PFX B Y 1
              PFX B 0 legesleg/X .

              SFX C Y 3
              SFX C 0 obb . +COMPARATIVE
              SFX C 0 obb/AX . +SUPERLATIVE
              SFX C 0 obb/BX . +SUPERSUPERLATIVE

       Dictionary:

              1
              nagy/C    [MN]

       Analysis:

              > nagy
              nagy[MN]
              > nagyobb
              nagy[MN]+COMPARATIVE
              > legnagyobb
              nagy[MN]+SUPERLATIVE
              > legeslegnagyobb
              nagy[MN]+SUPERSUPERLATIVE

Compounds

       Allowing  free compounding yields decrease in precision of recognition,
       not to mention stemming and morphological analysis.   Although  lexical
       switches are introduced to license compounding of bases by Ispell, this
       proves not to be restrictive enough. For example:

              # affix file
              COMPOUNDFLAG X

              2
              foo/X
              bar/X

       With this resource, foobar and barfoo also are accepted words.

       This has  been  improved  upon  with  the  introduction  of  direction-
       sensitive  compounding,  i.e.,  lexical features can specify separately
       whether a base can  occur  as  leftmost  or  rightmost  constituent  in
       compounds.    This,  however,  is  still  insufficient  to  handle  the
       intricate patterns of compounding, not to  mention  idiosyncratic  (and
       language specific) norms of hyphenation.

       The  Hunspell  algorithm  currently  allows  any affixed form of words,
       which are lexically marked as potential members of compounds.  Hunspell
       improved  this,  and  its  recursive  compound  checking rules makes it
       possible to implement the intricate spelling conventions  of  Hungarian
       compounds.   For   example,  using  COMPOUNDWORDMAX,  COMPOUNDSYLLABLE,
       COMPOUNDROOT, SYLLABLENUM options can be set the  noteworthy  Hungarian
       ‘6--3’  rule.   Further  example  in Hungarian, derivate suffixes often
       modify compounding properties. Hunspell allows the compounding flags on
       the  affixes,  and  there are two special flags (COMPOUNDPERMITFLAG and
       (COMPOUNDFORBIDFLAG)  to  permit  or  prohibit   compounding   of   the
       derivations.

       Suffixes with this flag forbid compounding of the affixed word.

       We also need several Hunspell features for handling German compounding:

              # German compounding

              # set language to handle special casing of German sharp s

              LANG de_DE

              # compound flags

              COMPOUNDBEGIN U
              COMPOUNDMIDDLE V
              COMPOUNDEND W

              # Prefixes are allowed at the beginning of compounds,
              # suffixes are allowed at the end of compounds by default:
              # (prefix)?(root)+(affix)?
              # Affixes with COMPOUNDPERMITFLAG may be inside of compounds.
              COMPOUNDPERMITFLAG P

              # for German fogemorphemes (Fuge-element)
              # Hint: ONLYINCOMPOUND is not required everywhere, but the
              # checking will be a little faster with it.

              ONLYINCOMPOUND X

              # forbid uppercase characters at compound word bounds
              CHECKCOMPOUNDCASE

              # for handling Fuge-elements with dashes (Arbeits-)
              # dash will be a special word

              COMPOUNDMIN 1
              WORDCHARS -

              # compound settings and fogemorpheme for ‘Arbeit’

              SFX A Y 3
              SFX A 0 s/UPX .
              SFX A 0 s/VPDX .
              SFX A 0 0/WXD .

              SFX B Y 2
              SFX B 0 0/UPX .
              SFX B 0 0/VWXDP .

              # a suffix for ‘Computer’

              SFX C Y 1
              SFX C 0 n/WD .

              # for forbid exceptions (*Arbeitsnehmer)

              FORBIDDENWORD Z

              # dash prefix for compounds with dash (Arbeits-Computer)

              PFX - Y 1
              PFX - 0 -/P .

              # decapitalizing prefix
              # circumfix for positioning in compounds

              PFX D Y 29
              PFX D A a/PX A
              PFX D Ä ä/PX Ä
               .
               .
              PFX D Y y/PX Y
              PFX D Z z/PX Z

       Example dictionary:

              4
              Arbeit/A-
              Computer/BC-
              -/W
              Arbeitsnehmer/Z

       Accepted compound compound words with the previous resource:

              Computer
              Computern
              Arbeit
              Arbeits-
              Computerarbeit
              Computerarbeits-
              Arbeitscomputer
              Arbeitscomputern
              Computerarbeitscomputer
              Computerarbeitscomputern
              Arbeitscomputerarbeit
              Computerarbeits-Computer
              Computerarbeits-Computern

       Not accepted compoundings:

              computer
              arbeit
              Arbeits
              arbeits
              ComputerArbeit
              ComputerArbeits
              Arbeitcomputer
              ArbeitsComputer
              Computerarbeitcomputer
              ComputerArbeitcomputer
              ComputerArbeitscomputer
              Arbeitscomputerarbeits
              Computerarbeits-computer
              Arbeitsnehmer

       This solution is still not ideal, however, and will be  replaced  by  a
       pattern-based  compound-checking  algorithm which is closely integrated
       with input buffer tokenization. Patterns describing compounds come as a
       separate  input  resource  that  can  refer to high-level properties of
       constituent parts (e.g. the  number  of  syllables,  affix  flags,  and
       containment  of  hyphens).  The  patterns are matched against potential
       segmentations of compounds to assess wellformedness.

Character encoding

       Problems with the 8-bit encoding

       Both Ispell and Myspell use 8-bit ASCII character encoding, which is  a
       major  deficiency  when  it  comes to scalability.  Although a language
       like Hungarian has a standard ASCII  character  set  (ISO  8859-2),  it
       fails   to  allow  a  full  implementation  of  Hungarian  orthographic
       conventions.  For instance, the ’--’ symbol (n-dash)  is  missing  from
       this  character  set  contrary  to  the  fact  that  it is not only the
       official symbol to delimit parenthetic clauses in the language, but  it
       can be in compound words as a special ’big’ hyphen.

       MySpell  has  got  some  8-bit encoding tables, but there are languages
       without standard 8-bit encoding, too. For example,  a  lot  of  African
       languages have non-latin or extended latin characters.

       Similarly,  using  the  original spelling of certain foreign names like
       ngstrm or Molire is encouraged by the Hungarian spelling norm,  and,
       since  characters  ’Å’  and  ’è’  are not part of ISO 8859-2, when they
       combine with inflections containing characters only in ISO 8859-2 (like
       elative  -bo=l,  allative  -to=l  or delative -ro=l with double acute),
       these result in words (like ngstrmro=l or Molire-to=l.) that can not
       be encoded using any single ASCII encoding scheme.

       The  problems raised in relation to 8-bit ASCII encoding have long been
       recognized  by  proponents  of  Unicode.  Unfortunately,  switching  to
       Unicode  (e.g.,  UTF-16  encoding)  would  require a great deal of code
       optimization and  would  have  an  impact  on  the  efficiency  of  the
       algorithm.  The  Dömölki algorithm used in checking affixing conditions
       utilizes 256-byte character  arrays,  which  would  grow  to  64k  with
       Unicode  encoding.  Since  online  affixing  for a richly agglutinative
       language can easily have several hundred such arrays (in  the  case  of
       the standard Hungarian resources we use, this number is ca. 300 or more
       since  redundant  storage  of  structurally  identical  affix  patterns
       improves  efficiency),  switching  to Unicode would incur high resource
       costs.  Nonetheless, it is clear that trading efficiency for  encoding-
       independence  has  its  advantages  when it comes a truly multi-lingual
       application, therefore it was among our  plans  for  a  long  while  to
       extend the architecture in this direction.

       A hybrid solution

       Recently  we  implemented  successfully  a  memory  and  time efficient
       Unicode handling. In non-UTF-8 character encodings Hunspell works  with
       the  original  8-bit  algorithms, but with UTF-8 encoded dictionary and
       affix file Hunspell uses a hybrid  string  manipulation  and  condition
       checking to support Unicode:

       Affixes  and words are stored in UTF-8, during the analysis are handled
       in mostly UTF-8, in condition checking and suggestion are converted  to
       UTF-16.

       Dömölki-algorithm  is  used  for  storing and checking 7-bit ASCII (ISO
       646) condition characters, and sorted UTF-16 lists  for  other  Unicode
       characters of condition patterns.

Conversion of aspell dictionaries

       Aspell  dictionaries  can be easily converted into hunspell. Conversion
       steps:

       dictionary (xx.cwl -> xx.wl):

       preunzip xx.cwl
       wc -l < xx.wl > xx.dic
       cat xx.wl >> xx.dic

       affix file

       If the affix file exists, copy it:
       cp xx_affix.dat xx.aff
       If not, create it with the suitable character encoding (see xx.dat)
       echo "SET ISO8859-x" > xx.aff
       or
       echo "SET UTF-8" > xx.aff

       It’s useful to add a TRY option with the characters of  the  dictionary
       with frequency order to set edit distance suggestions:
       echo "TRY qwertzuiopasdfghjklyxcvbnmQWERTZUIOPASDFGHJKLYXCVBNM" >>xx.aff

SEE ALSO

       hunspell (1), ispell (1), ispell (4)

                                  2005-12-31                       hunspell(4)