Provided by: hunspell_1.1.2-1_i386 bug


       hunspell - format of Hunspell dictionaries and affix files


       Hunspell(1)  requires  two  files  to  define  the  language that it is
       spellchecking.  The first file is a dictionary containing words for the
       language,  and  the  second is an "affix" file that defines the meaning
       of special flags in the dictionary.

       A dictionary file (*.dic) contains a list of words, one per line.   The
       first  line of the dictionaries (except personal dictionaries) contains
       the word count. Each word may optionally be followed by a  slash  ("/")
       and  one or more flags, which represents affixes or special attributes.
       Default flag format is a single (usually alphabetic)  character.  In  a
       Hunspell dictionary file, there is also an optional morphological field
       separated by tabulator. Morphological desciptions have custom format.

       An affix file (*.aff) may contain a lot of  optional  attributes.   For
       example, SET is used for setting the character encodings of affixes and
       dictionary files.  TRY sets the change characters for suggestions.  REP
       sets   a  replacement  table  for  multiple  character  corrections  in
       suggestion mode.  PFX and SFX defines prefix and suffix  classes  named
       with affix flags.

       The  following  affix  file  example  defines UTF-8 character encoding.
       ‘TRY’ suggestions differ from the bad word with an English letter or an
       apostrophe.  With these REP definitions, Hunspell can suggest the right
       word form, when the misspelled word contains f instead of ph  and  vice

              SET UTF-8
              TRY esianrtolcdugmphbyfvkwzESIANRTOLCDUGMPHBYFVKWZ’

              REP 2
              REP f ph
              REP ph f

              PFX A Y 1
              PFX A 0 re .

              SFX B Y 2
              SFX B 0 ed [^y]
              SFX B y ied y

       There are two affix classes in the dictionary. Class A defines an ‘re-’
       prefix. Class B defines two ‘-ed’ suffixes. First suffix can  be  added
       to  a  word if the last character of the word isn’t ‘y’.  Second suffix
       can be added to words terminated with an ‘y’.  (See details later.) The
       following dictionary file uses these affix classes.


       All  accepted words with this example: hello, try, tried, work, worked,
       rework, reworked.


       SET encoding
              Set character encoding of  words  and  morphemes  in  affix  and
              dictionary   files.    Possible   values:   UTF-8,  ISO8859-1  -
              ISO8859-10, ISO8859-13 - ISO8859-15, KOI8-R, KOI8-U,  microsoft-
              cp1251, ISCII-DEVANAGARI.

       FLAG value
              Set  flag  type.  Default  type  is  the  extended ASCII (8-bit)
              character.   ‘UTF-8’  parameter  sets  UTF-8   encoded   Unicode
              character  flags.   The  ‘long’  value  sets the double extended
              ASCII character flag type, the ‘num’  sets  the  decimal  number
              flag  type.  Decimal flags numbered from 1 to 65535, and in flag
              fields are separated by comma.

              Set twofold prefix stripping (but single suffix  stripping)  for
              agglutinative languages with right-to-left writing system.

       LANG langcode
              Set  language  code.  In Hunspell may be language specific codes
              enabled by LANG code. At present there are az_AZ, de_DE,  hu_HU,
              TR_tr specific codes in Hunspell (see the source code).


       TRY characters
              Hunspell  can  suggest right word forms, when those differs from
              the bad form by one TRY character. The parameter of TRY is  case

       NOSUGGEST flag
              Words  signed  with  NOSUGGEST  flag are not suggested. Proposed
              flag for vulgar and obscene words.

              Set number of n-gram suggestions. Value 0 switches  off  the  n-
              gram suggestions.

              Disable split-word suggestions.

              Add  dot(s)  to suggestions, if input word terminates in dot(s).
              (Not for dictionaries, because has
              an automatic dot expansion mechanism.)

       REP number_of_replacement_definitions

       REP what replacement
              We  can  define  language-dependent  phonetic information in the
              affix file (.aff)  by a replacement table.   First  REP  is  the
              header of this table and one or more REP data line are following
              it. With this table, Hunspell can suggest the  right  forms  for
              the  typical  faults of spelling when the incorrect form differs
              by more, than 1 letter from  the  right  form.   For  example  a
              possible   English   replacement   table  definition  to  handle
              misspelled consonants:

              REP 8
              REP f ph
              REP ph f
              REP f gh
              REP gh f
              REP j dg
              REP dg j
              REP k ch
              REP ch k

       Replacement table is  also  usable  in  robust  morphological  analysis
       (accepting  bad  forms)  or  stricter  compound  word  support in spell
       checking (forbidding generated compound words, if they are also  simple
       words with typical fault).

       MAP number_of_map_definitions

       MAP string_of_related_chars
              We  can define language-dependent information on characters that
              should be considered related (ie. nearer than other chars not in
              the  set)  in  the  affix file (.aff)  by a character map table.
              With this table, Hunspell can suggest the right forms for words,
              which  incorrectly  choose  the  wrong letter from a related set
              more than once in a word.

              For example a possible mapping could be for the German  umlauted
              ü  versus  the  regular  u;  the word Frühstück really should be
              written with umlauted u’s and not regular ones

              MAP 1
              MAP uü


       COMPOUND number_of_compound_definitions

       COMPOUND compound_pattern
              Define custom compound patterns with a regex-like  syntax.   The
              first  COMPOUND  is  a  header  with the number of the following
              COMPOUND definitions. Compound patterns consist  compound  flags
              and  star or question mark meta characters. A flag followed by a
              ‘*’ matches a word sequence of 0 or more matches of words signed
              with  this  compound  flag.   A flag followed by a ‘?’ matches a
              word sequence of 0 or 1 matches  of  a  word  signed  with  this
              compound  flag.   See  tests/compound*.* examples. Note: ‘*’ and
              ‘?’ metacharacters work only with the  default  8-bit  character
              and  the UTF-8 FLAG types.  Note II: COMPOUND flags haven’t been
              compatible with the COMPOUNDFLAG, COMPOUNDBEGIN,  etc.  compound
              flags yet (use these flags on different words).

       COMPOUNDMIN num
              Minimum  length  of words in compound words.  Default value is 3

       COMPOUNDFLAG flag
              Words signed with COMPOUNDFLAG may be in compound words  (except
              when  word  shorter than COMPOUNDMIN). Affixes with COMPOUNDFLAG
              also permits compounding of affixed words.

              Words signed with COMPOUNDFLAG (or with a signed affix)  may  be
              first elements in compound words.

       COMPOUNDLAST flag
              Words  signed  with COMPOUNDFLAG (or with a signed affix) may be
              last elements in compound words.

              Words signed with COMPOUNDFLAG (or with a signed affix)  may  be
              middle elements in compound words.

              Suffixes  signed  with ONLYINCOMPOUND flag may be only inside of
              compounds (Fuge-elements in German, fogemorphemes  in  Swedish).
              ONLYINCOMPOUND    flag    works    also    with    words    (see

              Prefixes are allowed at the beginning of compounds, suffixes are
              allowed  at  the  end  of  compounds  by  default.  Affixes with
              COMPOUNDPERMITFLAG may be inside of compounds.

              Suffixes with this flag forbid compounding of the affixed  word.

       COMPOUNDROOT flag
              COMPOUNDROOT  flag signs the compounds in the dictionary (Now it
              is used only in the Hungarian language specific code).

              Set  maximum  word  count  in  a  compound  word.  (Default   is

              Forbid word duplication in compounds (eg. foofoo).

              Forbid  compounding, if the (usually bad) compound word may be a
              non compound word with a REP fault. Useful  for  languages  with
              ‘compound friendly’ orthography.

              Forbid upper case characters at word bound in compounds.

              Forbid  compounding,  if  compound  word contains triple letters
              (eg. foo|ox  or  xo|oof).   Bug:  missing  multi-byte  character
              support   in   UTF-8   encoding  (works  only  for  7-bit  ASCII

       CHECKCOMPOUNDPATTERN number_of_checkcompoundpattern_definitions

       CHECKCOMPOUNDPATTERN endchars beginchars
              Forbid  compounding,  if  first  word  in  compound  ends   with
              endchars, and next word begins with beginchars.

       COMPOUNDSYLLABLE max_syllable vowels
              Need   for   special  compounding  rules  in  Hungarian.   First
              parameter is the maximum syllable  number,  that  may  be  in  a
              compound,  if  words in compounds are more than COMPOUNDWORDMAX.
              Second  parameter  is  the  list  of  vowels  (for   calculating

       SYLLABLENUM flags
              Need for special compounding rules in Hungarian.


       PFX flag cross_product number

       PFX flag stripping prefix condition morphological_description

       SFX flag cross_product number

       SFX flag stripping suffix condition morphological_description
              An  affix  is either a prefix or a suffix attached to root words
              to make other words. We can define affix classes with  arbitrary
              number  affix rules.  Affix classes are signed with affix flags.
              The first line of an affix class definition is the  header.  The
              fields of an affix class header:

              (0) Option name (PFX or SFX)

              (1) Flag (name of the affix class)

              (2) Cross product (permission to combine prefixes and suffixes).
              Possible values: Y (yes) or N (no)

              (3) Line count of the following rules.

              Fields of an affix rules:

              (0) Option name

              (1) Flag

              (2) stripping characters from beginning (at prefix rules) or end
              (at suffix rules) of the word

              (3)  affix  (optionally  with  flags  of  continuation  classes,
              separated by a slash)

              (4) condition.

              Zero stripping or affix are indicated by zero. Zero condition is
              indicated   by   dot.    Condition   is  a  simplified,  regular
              expression-like pattern, which must be met before the affix  can
              be  applied.  (Dot  signs  an arbitrary character. Characters in
              braces sign an arbitrary character from  the  character  subset.
              Dash  hasn’t  got  special  meaning, but circumflex (^) next the
              first brace sets the complementer character set.)

              (5) Custom morphological description.


       CIRCUMFIX flag
              Affixes signed with CIRCUMFIX flag may be on a  word  when  this
              word also has a prefix with CIRCUMFIX flag and vice versa.

              This  flag  signs forbidden word form. Because affixed forms are
              also forbidden, we can  substract  a  subset  from  set  of  the
              accepted affixed and compound words.

       KEEPCASE flag
              Forbid  capitalized  and  uppercased  forms of words signed with
              KEEPCASE flags. Useful for  special  ortographies  (measurements
              and  currency  often  keep  their  case in uppercased texts) and
              writing systems (eg. keeping lower case of IPA characters).

       LEMMA_PRESENT flag
              Generally, there are dictionary words as  lemmas  in  output  of
              morphological  analysis.  Sometimes  dictionary  words  are  not
              lemmas, but affixed (not real) stems and virtual stems. In  this
              case   lemmas  (real  stems)  need  to  put  into  morphological
              description,  and  forbid  not  real  lemmas  in   morphological
              analysis adding LEMMA_PRESENT flag to dictionary words.

       NEEDAFFIX flag
              This  flag  signs virtual stems in the dictionary.  Only affixed
              forms of these words will be accepted by Hunspell.   Except,  if
              the  dictionary  word  has a homonym or a zero affix.  NEEDAFFIX
              works also with prefixes and prefix + suffix  combinations  (see

       PSEUDOROOT flag
              Deprecated. (Former name of the NEEDAFFIX option.)

       WORDCHARS characters
              WORDCHARS  extends  tokenizer of Hunspell command line interface
              with additional word character. For example, dot, dash,  n-dash,
              numbers,  percent  sign  are  word character in Hungarian. (This
              attribute haven’t worked with UTF-8 encoding yet.)

Morphological analysis

       Hunspell’s affix rules have got an optional  morphological  description
       field.  There is a similar optional field in dictionary file, separated
       by tabulator:

               word/flags    morphology

       We define a simple resource with morphological informations.

       Affix file:

               SFX X Y 1
               SFX X 0 able . +ABLE

       Dictionary file:

               drink/X   [VERB]

       Test file:



               $ hunmorph test.aff test.dic test.txt
               drink:     drink[VERB]
               drinkable: drink[VERB]+ABLE

       You can  see  in  the  example,  that  the  analyzer  concatenates  the
       morphological fields in item and arrangement style.

Twofold suffix stripping

       Ispell’s  original algorithm strips only one suffix. Hunspell can strip
       another one yet.

       The twofold suffix stripping is a significant improvement  in  handling
       of   immense   number  of  suffixes,  that  characterize  agglutinative

       Extending the previous example by adding a second suffix (affix class Y
       will be the continuation class of the suffix ‘able’):

               SFX Y Y 1
               SFX Y 0 s . +PLUR

               SFX X Y 1
               SFX X 0 able/Y . +ABLE

       Dictionary file:

               drink/X   [VERB]

       Test file:



               $ hunmorph test.aff test.dic test.txt
               drink:      drink[VERB]
               drinkable:  drink[VERB]+ABLE
               drinkables: drink[VERB]+ABLE+PLUR

       Theoretically  with  the twofold suffix stripping needs only the square
       root  of  the  number  of  suffix  rules,  compared  with  a   Hunspell
       implementation. In our practice, we could have elaborated the Hungarian
       inflectional morphology  with  twofold  suffix  stripping.   (Note:  In
       Hunlex preprocessor’s grammar can be use not only twofold, but multiple
       suffix slitting.)

Extended affix classes

       Hunspell can handle more than 65000 affix classes.  There are  two  new
       syntax for giving flags in affix and dictionary files.

       FLAG long command sets 2-character flags:

                FLAG long
                SFX Y1 Y 1
                SFX Y1 0 s 1

       Dictionary record with the Y1, Z3, F? flags:


       FLAG num command sets numerical flags separated by comma:

                FLAG num
                SFX 65000 Y 1
                SFX 65000 0 s 1

       Dictionary example:



       Hunspell’s dictionary can contain repeating elements that is homonyms:

               work/A    [VERB]
               work/B    [NOUN]

       An affix file:

               SFX A Y 1
               SFX A 0 s . +SG3

               SFX B Y 1
               SFX B 0 s . +PLUR

       Test file:



               > works

       This   feature  also  gives  a  way  to  forbid  illegal  prefix/suffix
       combinations in difficult cases.

Prefix--suffix dependencies

       An  interesting  side-effect  of  multi-step  stripping  is,  that  the
       appropriate treatment of circumfixes now comes for free.  For instance,
       in Hungarian, superlatives are formed by  simultaneous  prefixation  of
       leg-  and suffixation of -bb to the adjective base.  A problem with the
       one-level architecture is that  there  is  no  way  to  render  lexical
       licensing  of  particular  prefixes  and  suffixes  interdependent, and
       therefore incorrect forms are recognized as valid, i.e. *legvn = leg +
       vn  ‘old’.  Until the introduction of clusters, a special treatment of
       the superlative had to be hardwired in the earlier HunSpell code.  This
       may  have been legitimate for a single case, but in fact prefix--suffix
       dependences are ubiquitous in category-changing  derivational  patterns
       (cf.   English   payable,   non-payable   but  *non-pay  or  drinkable,
       undrinkable but *undrink). In simple words, here,  the  prefix  un-  is
       legitimate only if the base drink is suffixed with -able. If both these
       patters are handled by on-line affix rules and affix rules are  checked
       against  the  base only, there is no way to express this dependency and
       the system will necessarily over- or undergenerate.

       In next example, suffix class R have got a prefix ‘continuation’  class
       (class P).

              PFX P Y 1
              PFX P   0 un . [prefix_un]+

              SFX S Y 1
              SFX S   0 s . +PL

              SFX Q Y 1
              SFX Q   0 s . +3SGV

              SFX R Y 1
              SFX R   0 able/PS . +DER_V_ADJ_ABLE


              drink/RQ  [verb]
              drink/S   [noun]

       Morphological analysis:

              > drink
              > drinks
              > drinkable
              > drinkables
              > undrinkable
              > undrinkables
              > undrink
              Unknown word.
              > undrinks
              Unknown word.


       Conditional  affixes implemented by a continuation class are not enough
       for circumfixes, because a circumfix is one  affix  in  morphology.  We
       also need CIRCUMFIX option for correct morphological analysis.

              # circumfixes: ~ obligate prefix/suffix combinations
              # superlative in Hungarian: leg- (prefix) AND -bb (suffix)
              # nagy, nagyobb, legnagyobb, legeslegnagyobb
              # (great, greater, greatest, most greatest)

              CIRCUMFIX X

              PFX A Y 1
              PFX A 0 leg/X .

              PFX B Y 1
              PFX B 0 legesleg/X .

              SFX C Y 3
              SFX C 0 obb . +COMPARATIVE
              SFX C 0 obb/AX . +SUPERLATIVE
              SFX C 0 obb/BX . +SUPERSUPERLATIVE


              nagy/C    [MN]


              > nagy
              > nagyobb
              > legnagyobb
              > legeslegnagyobb


       Allowing  free compounding yields decrease in precision of recognition,
       not to mention stemming and morphological analysis.   Although  lexical
       switches are introduced to license compounding of bases by Ispell, this
       proves not to be restrictive enough. For example:

              # affix file
              COMPOUNDFLAG X


       With this resource, foobar and barfoo also are accepted words.

       This has  been  improved  upon  with  the  introduction  of  direction-
       sensitive  compounding,  i.e.,  lexical features can specify separately
       whether a base can  occur  as  leftmost  or  rightmost  constituent  in
       compounds.    This,  however,  is  still  insufficient  to  handle  the
       intricate patterns of compounding, not to  mention  idiosyncratic  (and
       language specific) norms of hyphenation.

       The  Hunspell  algorithm  currently  allows  any affixed form of words,
       which are lexically marked as potential members of compounds.  Hunspell
       improved  this,  and  its  recursive  compound  checking rules makes it
       possible to implement the intricate spelling conventions  of  Hungarian
       compounds.   For   example,  using  COMPOUNDWORDMAX,  COMPOUNDSYLLABLE,
       COMPOUNDROOT, SYLLABLENUM options can be set the  noteworthy  Hungarian
       ‘6--3’  rule.   Further  example  in Hungarian, derivate suffixes often
       modify compounding properties. Hunspell allows the compounding flags on
       the  affixes,  and  there are two special flags (COMPOUNDPERMITFLAG and
       (COMPOUNDFORBIDFLAG)  to  permit  or  prohibit   compounding   of   the

       Suffixes with this flag forbid compounding of the affixed word.

       We also need several Hunspell features for handling German compounding:

              # German compounding

              # set language to handle special casing of German sharp s

              LANG de_DE

              # compound flags

              COMPOUNDBEGIN U
              COMPOUNDMIDDLE V
              COMPOUNDEND W

              # Prefixes are allowed at the beginning of compounds,
              # suffixes are allowed at the end of compounds by default:
              # (prefix)?(root)+(affix)?
              # Affixes with COMPOUNDPERMITFLAG may be inside of compounds.

              # for German fogemorphemes (Fuge-element)
              # Hint: ONLYINCOMPOUND is not required everywhere, but the
              # checking will be a little faster with it.

              ONLYINCOMPOUND X

              # forbid uppercase characters at compound word bounds

              # for handling Fuge-elements with dashes (Arbeits-)
              # dash will be a special word

              COMPOUNDMIN 1
              WORDCHARS -

              # compound settings and fogemorpheme for ‘Arbeit’

              SFX A Y 3
              SFX A 0 s/UPX .
              SFX A 0 s/VPDX .
              SFX A 0 0/WXD .

              SFX B Y 2
              SFX B 0 0/UPX .
              SFX B 0 0/VWXDP .

              # a suffix for ‘Computer’

              SFX C Y 1
              SFX C 0 n/WD .

              # for forbid exceptions (*Arbeitsnehmer)

              FORBIDDENWORD Z

              # dash prefix for compounds with dash (Arbeits-Computer)

              PFX - Y 1
              PFX - 0 -/P .

              # decapitalizing prefix
              # circumfix for positioning in compounds

              PFX D Y 29
              PFX D A a/PX A
              PFX D Ä ä/PX Ä
              PFX D Y y/PX Y
              PFX D Z z/PX Z

       Example dictionary:


       Accepted compound compound words with the previous resource:


       Not accepted compoundings:


       This solution is still not ideal, however, and will be  replaced  by  a
       pattern-based  compound-checking  algorithm which is closely integrated
       with input buffer tokenization. Patterns describing compounds come as a
       separate  input  resource  that  can  refer to high-level properties of
       constituent parts (e.g. the  number  of  syllables,  affix  flags,  and
       containment  of  hyphens).  The  patterns are matched against potential
       segmentations of compounds to assess wellformedness.

Character encoding

       Problems with the 8-bit encoding

       Both Ispell and Myspell use 8-bit ASCII character encoding, which is  a
       major  deficiency  when  it  comes to scalability.  Although a language
       like Hungarian has a standard ASCII  character  set  (ISO  8859-2),  it
       fails   to  allow  a  full  implementation  of  Hungarian  orthographic
       conventions.  For instance, the ’--’ symbol (n-dash)  is  missing  from
       this  character  set  contrary  to  the  fact  that  it is not only the
       official symbol to delimit parenthetic clauses in the language, but  it
       can be in compound words as a special ’big’ hyphen.

       MySpell  has  got  some  8-bit encoding tables, but there are languages
       without standard 8-bit encoding, too. For example,  a  lot  of  African
       languages have non-latin or extended latin characters.

       Similarly,  using  the  original spelling of certain foreign names like
       ngstrm or Molire is encouraged by the Hungarian spelling norm,  and,
       since  characters  ’Å’  and  ’è’  are not part of ISO 8859-2, when they
       combine with inflections containing characters only in ISO 8859-2 (like
       elative  -bo=l,  allative  -to=l  or delative -ro=l with double acute),
       these result in words (like ngstrmro=l or Molire-to=l.) that can not
       be encoded using any single ASCII encoding scheme.

       The  problems raised in relation to 8-bit ASCII encoding have long been
       recognized  by  proponents  of  Unicode.  Unfortunately,  switching  to
       Unicode  (e.g.,  UTF-16  encoding)  would  require a great deal of code
       optimization and  would  have  an  impact  on  the  efficiency  of  the
       algorithm.  The  Dömölki algorithm used in checking affixing conditions
       utilizes 256-byte character  arrays,  which  would  grow  to  64k  with
       Unicode  encoding.  Since  online  affixing  for a richly agglutinative
       language can easily have several hundred such arrays (in  the  case  of
       the standard Hungarian resources we use, this number is ca. 300 or more
       since  redundant  storage  of  structurally  identical  affix  patterns
       improves  efficiency),  switching  to Unicode would incur high resource
       costs.  Nonetheless, it is clear that trading efficiency for  encoding-
       independence  has  its  advantages  when it comes a truly multi-lingual
       application, therefore it was among our  plans  for  a  long  while  to
       extend the architecture in this direction.

       A hybrid solution

       Recently  we  implemented  successfully  a  memory  and  time efficient
       Unicode handling. In non-UTF-8 character encodings Hunspell works  with
       the  original  8-bit  algorithms, but with UTF-8 encoded dictionary and
       affix file Hunspell uses a hybrid  string  manipulation  and  condition
       checking to support Unicode:

       Affixes  and words are stored in UTF-8, during the analysis are handled
       in mostly UTF-8, in condition checking and suggestion are converted  to

       Dömölki-algorithm  is  used  for  storing and checking 7-bit ASCII (ISO
       646) condition characters, and sorted UTF-16 lists  for  other  Unicode
       characters of condition patterns.

       Hunspell   have   supported  only  the  first  65534  character  (Basic
       Multilingual Plane) of Unicode Standard yet.


       hunspell (1), ispell (1), ispell (4)

                                2005. július 2.                    hunspell(4)