NAME

       pymvpa2-crossval -  cross-validation of a learner's performance

SYNOPSIS

       pymvpa2  crossval  [--version] [-h] -i DATASET [DATASET ...] --learner LEARNER [--learner-
       space  LEARNER_SPACE]  --partitioner  PARTITIONER  [--errorfx  ERRORFX]   [--avg-datafold-
       results]        [--balance-training        BALANCE_TRAINING]       [--sampling-repetitions
       SAMPLING_REPETITIONS] [--permutations PERMUTATIONS] [--prob-tail {left,right}]  -o  OUTPUT
       [--hdf5-compression TYPE]

DESCRIPTION

       Cross-validation of a learner's performance

       A  learner  is  repeatedly  trained  and tested on partitions of an input dataset that are
       generated by a configurable partitioning scheme.  Partition  usually  constitute  training
       and  testing portions.  The learner is trained on training portion of the dataset and then
       learner's generalization is tested by comparing its predictions on the testing portion.

       A summary of a learner performance is written to STDOUT. Depending on the particular setup
       of  the  cross-validation  analysis,  either  the  learner's  raw  predictions  or summary
       statistics are returned in an output dataset.

       If Monte-Carlo permutation testing is enabled (see --permutations) a second output dataset
       with the corresponding p-values is stored as well (filename suffix '_nullprob').

OPTIONS

       --version
              show program's version and license information and exit

       -h, --help, --help-np
              show  this  help message and exit. --help-np forcefully disables the use of a pager
              for displaying the help.

       -i DATASET [DATASET ...], --input DATASET [DATASET ...]
              path(s) to one or more PyMVPA dataset files. All datasets will  be  merged  into  a
              single dataset (vstack'ed) in order of specification. In some cases this option may
              need to be specified more than once if multiple, but separate, input  datasets  are
              required.

   Options for cross-validation setup:
       --learner LEARNER
              select a learner (trainable node) via its description in the learner warehouse (see
              'info' command for a listing), a colon-separated list of capabilities, or by a file
              path to a Python script that creates a classifier instance (advanced).

       --learner-space LEARNER_SPACE
              name  of  a  sample attribute that defines the model to be learned by a learner. By
              default this is an attribute named 'targets'.

       --partitioner PARTITIONER
              select a data folding  scheme.  Supported  arguments  are:  'half'  for  split-half
              partitioning,  'oddeven' for partitioning into odd and even chunks, 'group-X' where
              X can be any positive integer for partitioning in X groups, 'n-X' where  X  can  be
              any  positive  integer for leave-X-chunks out partitioning. By default partitioners
              operate on dataset chunks that are defined by a 'chunks' sample attribute. The name
              of the "chunking" attribute can be changed by appending a colon and the name of the
              attribute (e.g.  'oddeven:run'). optionally an argument to this option can also  be
              a  file  path  to  a  Python  script  that  creates  a  custom partitioner instance
              (advanced).

       --errorfx ERRORFX
              error  function  to  be  applied  to  the   targets   and   predictions   of   each
              cross-validation  data  fold.  This  can  either be a name of any error function in
              PyMVPA's mvpa2.misc.errorfx module, or a file path to a Python script that  creates
              a custom error function (advanced).

       --avg-datafold-results
              average  result  values across data folds generated by the partitioner. For example
              to compute a mean prediction error across all folds of a crossvalidation procedure.

       --balance-training BALANCE_TRAINING
              If enabled, training samples are balanced within each data  fold.  If  the  keyword
              'equal'  is  given  as  argument  an equal number of random samples for each unique
              target value is chosen. The number of samples per category  is  determined  by  the
              category  with  the  least  number  of  samples  in the respective training set. An
              integer argument will cause the a corresponding number of samples per  category  to
              be  randomly  selected. A floating point number argument (interval [0,1]) indicates
              what fraction of the available samples shall be selected.

       --sampling-repetitions SAMPLING_REPETITIONS
              If training set balancing is enabled, how often should random sample  selection  be
              performed for each data fold. Default: 1

       --permutations PERMUTATIONS
              Number  of  Monte-Carlo  permutation  runs  to  be  computed  for  estimating an H0
              distribution for all  crossvalidation  results.  Enabling  this  option  will  make
              reports of corresponding p-values available in the result summary and output.

       --prob-tail {left,right}
              which  tail of the probability distribution to report p-values from when evaluating
              permutation test results. For example, a cross-validation computing mean prediction
              error could report left-tail p-value for a single-sided test.

   Output options:
       -o OUTPUT, --output OUTPUT
              output  filename ('.hdf5' extension is added automatically if necessary). NOTE: The
              output format is suitable for data exchange between PyMVPA  commands,  but  is  not
              recommended  for  long-term  storage  or  exchange as its specific content may vary
              depending on the  actual  software  environment.  For  long-term  storage  consider
              conversion into other data formats (see 'dump' command).

       --hdf5-compression TYPE
              compression  type  for  HDF5  storage. Available values depend on the specific HDF5
              installation. Typical values are: 'gzip', 'lzf', 'szip', or integers from  1  to  9
              indicating gzip compression levels.

AUTHOR

       Written by Michael Hanke & Yaroslav Halchenko, and numerous other contributors.

COPYRIGHT

       Copyright © 2006-2015 PyMVPA developers

       Permission  is  hereby  granted,  free  of  charge, to any person obtaining a copy of this
       software and associated documentation files (the "Software"),  to  deal  in  the  Software
       without  restriction, including without limitation the rights to use, copy, modify, merge,
       publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons
       to whom the Software is furnished to do so, subject to the following conditions:

       The  above  copyright notice and this permission notice shall be included in all copies or
       substantial portions of the Software.

       THE SOFTWARE IS PROVIDED "AS IS", WITHOUT  WARRANTY  OF  ANY  KIND,  EXPRESS  OR  IMPLIED,
       INCLUDING  BUT  NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
       PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE  LIABLE
       FOR  ANY  CLAIM,  DAMAGES  OR  OTHER  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
       OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR  THE  USE  OR  OTHER
       DEALINGS IN THE SOFTWARE.