NAME

       estwaver - command line interface of web crawler

SYNOPSIS

       estwaver init [-apn|-acc] [-xs|-xl|-xh] [-sv|-si|-sa] rootdir

       estwaver crawl [-restart|-revisit|-revcont] rootdir

       estwaver unittest rootdir

       estwaver fetch [-proxy hostr port] [-tout num] [-il lang] url

DESCRIPTION

       estwaver is an aggregation of sub commands.  The name of a sub command is specified by the
       first argument.  Other arguments are parsed according to each sub command.   The  argument
       rootdir specifies the crawler root directory which contains configuration file and so on.

       estwaver init [-apn|-acc] [-xs|-xl|-xh] [-sv|-si|-sa] rootdir
              Create the crawler root directory.
              If -apn is specified, N-gram analysis is performed against European text also.
              If  -acc  is  specified, character category analysis is performed instead of N-gram
              analysis.
              If -xs is specified, the index is tuned to register less than 50000 documents.
              If -xl is specified, the index is tuned to register more than 300000 documents.
              If -xh is specified, the index is tuned to register more than 1000000 documents.
              If -sv is specified, scores are stored as void.
              If -si is specified, scores are stored as 32-bit integer.
              If -sa is specified, scores are stored as-is  and  marked  not  to  be  tuned  when
              search.

       estwaver crawl [-restart|-revisit|-revcont] rootdir
              Start crawling.
              If -restart is specified, crawling is restarted from the seed documents.
              If -revisit is specified, collected documents are revisited.
              If  -revcont  is  specified, collected documents are revisited and then crawling is
              continued.</dd>

       estwaver unittest rootdir
              Perform unit tests.

       estwaver fetch [-proxy hostr port] [-tout num] [-il lang] url
              Fetch a document.
              url specifies the URL of a document.
              -proxy specifies the host name and the port number of the proxy server.
              -tout specifies timeout in seconds.
              -il specifies the preferred language.  By default, it is English.

       All sub commands return 0 if the operation is success, else return 1.  A  running  crawler
       finishes  with  closing  the database when it catches the signal 1 (SIGHUP), 2 (SIGINT), 3
       (SIGQUIT), or 15 (SIGTERM).

       When crawling finishes, there is a directory _index in the crawler root directory.  It  is
       an index available by estcmd and so on.

SEE ALSO

       estconfig(1), estcmd(1), estmaster(1), estcall(1), estraier(3), estnode(3)