bionic (3) Lingua::Sentence.3pm.gz

Provided by: liblingua-sentence-perl_1.05-1_all bug

NAME
       Lingua::Sentence - Perl extension for breaking text paragraphs into sentences


SYNOPSIS
               use Lingua::Sentence;

               my $splitter = Lingua::Sentence->new("en");

               my $text = 'This is a paragraph. It contains several sentences. "But why," you ask?';

               print $splitter->split($text);


DESCRIPTION
       This module allows splitting of text paragraphs into sentences. It is based on scripts developed by
       Philipp Koehn and Josh Schroeder for processing the Europarl corpus (<http://www.statmt.org/europarl/>).

       The module uses punctuation and capitalization clues to split paragraphs into an newline-separated string
       with one sentence per line. For example:

               This is a paragraph. It contains several sentences. "But why," you ask?

       goes to:

               This is a paragraph.
               It contains several sentences.
               "But why," you ask?

       Languages currently supported by the module are:

       Catalan
       Czech
       Dutch
       English
       French
       German
       Greek
       Hungarian
       Icelandic
       Italian
       Latvian
       Polish
       Portuguese
       Russian
       Spanish
       Slovak
       Slovenian
       Swedish

   Nonbreaking Prefixes Files
       Nonbreaking prefixes are loosely defined as any word ending in a period that does NOT indicate an end of
       sentence marker. A basic example is Mr. and Ms. in English.

       The sentence splitter module uses the nonbreaking prefix files included in this distribution.

       To add a file for other languages, follow the naming convention nonbreaking_prefix.?? and use the two-
       letter language code you intend to use when creating a Lingua::Sentence object.

       The sentence splitter module will first look for a file for the language it is processing, and fall back
       to English if a file for that language is not found.

       For the splitter, normally a period followed by an uppercase word results in a sentence split. If the
       word preceeding the period is a nonbreaking prefix, this line break is not inserted.

       A special case of prefixes, NUMERIC_ONLY, is included for special cases where the prefix should be
       handled ONLY when before numbers.  For example, "Article No. 24 states this." the No. is a nonbreaking
       prefix. However, in "No. It is not true." No functions as a word.

       See the example prefix files included in the distribution for more examples.

       CREDITS

       Thanks for the following individuals for supplying nonbreaking prefix files: Bas Rozema (Dutch), Hilario
       Leal Fontes (Portuguese), Jesus Gimenez (Catalan & Spanish), Anne-Kathrin Schumann (Russian)

   EXPORT
       new($lang_id)
           Instantiate an object to split sentences in language $lang_id. If the language is not supported, a
           splitter object for English will be instantiated.

       new($lang_id,$nonbreaking_prefix_file)
           Instantiate an object to split sentences in language $lang_id and the nonbreaking prefix file
           $nonbreaking_prefix_file. If the file does not exist, a splitter object for English will be
           instantiated.

       split($text)
           Split sentences in $text by inserting newline characters at the sentence breaks. The resulting string
           is also terminated with a newline.

       split_array($text)
           Split sentences in $text into an array of sentences.


SUPPORT
       Bugs should always be submitted via the project hosting bug tracker

       <http://code.google.com/p/corpus-tools/issues/list>

       For other issues, contact the maintainer.


SEE ALSO
       Text::Sentence, Lingua::EN::Sentence, Lingua::DE::Sentence, Lingua::HE::Sentence


AUTHOR
       Achim Ruopp, <achimru@gmail.com>


       Copyright (C) 2010 by Digital Silk Road

       Portions Copyright (C) 2005 by Philip Koehn and Josh Schroeder (used with permission)

       This program is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser
       General Public License as published by the Free Software Foundation, either version 3 of the License, or
       (at your option) any later version.

       This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even
       the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General
       Public License for more details.

       You should have received a copy of the GNU Lesser General Public License along with this program.  If
       not, see <http://www.gnu.org/licenses/>.