Ubuntu Manpage: pycocci - Coccinelle wrapper for SmPL patch development

Provided by: coccinelle_1.1.1.deb-5_amd64

NAME

       pycocci - Coccinelle wrapper for SmPL patch development

       pycocci  [-h  | --help] [-p | --profile-cocci] [-j | --jobs <job-digit> ] [-v | --verbose]
       [-s | --show-proof] <SmPL-patch.cocci> <target> ]

DESCRIPTION

pycocci is wrapper around spatch, it enables a set of default arguments and also uses a
bit of heuristics to infers which arguments you likely want enabled. It takes two
arguments, the <SmPL-patch.cocci> and your <target> which can either be a directory or
file. The actual command run is always displayed on standard output.

Coccinelle spatch is a very rich tool, it provides a large set of features for use with
the Coccinelle engine. Typical day to day SmPL patch development may only require a base
set of features though, we can also infer a set of features depending on your target.
pycocci enables a set of flags which over time have been determined to be required for day
to day Linux kernel development, for instance through a cronjob, it also uses a set of
heuristics to infer a set of arguments to use depending on your release of Coccinelle.
pycocci also checks to see which version of Coccinelle you have installed and looks to
enable arguments depending on what version of Coccinelle you have installed.

pycocci can also be used to help write and verify SmPL patches when replacing a regular
patch series. pycocci checks the version of spatch installed and based on this enables a
set of features. This man page will document what options it always enables by default,
why it has done this and it will also document the heuristics used and logic behind them.

DEFAULT ARGUMENTS

       By default pycocci will always enable when running spatch:

       --in-place

       --recursive-includes

       --relax-include-path

       --timeout 120

       --dir <target>

       The  --in-place  option  is  enabled  by default as most development these days happens on
       version control systems  and  built-in  version  control  systems  can  typically  display
       differences better.

       --relax-include-path  and  --relax-include-path are enabled by default given that at least
       for Linux kernel development some C files tend to include headers in paths only  specified
       by  a  target's  Makefile  through utilities such as subdir-ccflags or ccflags-y making it
       difficult to ensure all required header files are read by Coccinelle.  We  are  aggressive
       with headers search and inclusion.

       A  default timeout of 120 seconds is used by default. Not using a timeout is typically not
       a good idea. The value of 120 is used by default and  is  considered  significantly  large
       enough to support most SmPL patches.

       We  make  use  of  --dir  to  enable  use  of  a  target  directory and its subdirectories
       recursively.

MULTITHREAD ARGUMENT HEURISTICS

       Coccinelle spatch prior to release 1.0.0 provided support for multithreading but  required
       the  developer  to  spawn  each thread on their own, and provide an index reference number
       identifying each thread. Coccinelle would divide the amount of work required  to  be  done
       and  based on the index grant the thread a specific set of work.  Some shell scripts could
       be used to help split the work out for you.  pycocci was originally written  to  supersede
       these  scripts  and  use Python's multithreaded support, while also enabling some sensible
       arguments by default.

       If you have a version of spatch older than 1.0.2 pycocci will  provide  its  own  built-in
       multithreaded  batched solution, the spatch --jobs argument is enabled on spatch >= 1.0.2.
       The spatch --jobs arguments takes advantage of Coccinelle's built-in paramap support,  and
       performance-wise  yields  better results than pycocci's multithreaded solution. The number
       of threads used will always default to the number of number of CPUs on your  system,  this
       is taken from what Python multiprocessing.cpu_count() returns. You can override the number
       of threads pycocci will use with the --jobs argument.

INDEXING ARGUMENT HEURISTICS

Coccinelle pycocci supports using a series of indexing alternatives:

--use-glimpse

--use-gitgrep

--use-coccigrep

Coccinelle puts the onus of which indexing feature to enable on the developer. pycocci
will figure things out for you and make sure that only supported options are used in
specific supported releases of coccinelle. So for instance, although --use-gitgrep has
been available on 1.0.1 pycocci will only it on 1.0.2. The order of preference for what
indexing option to uses the following heuristics: If your target directory has a
.glimpse_index file we assume you will want to use --use-glimpse. Glimpse is now released
under the ISC license and performance-wise is known to work as the best indexing
alternative, this is why we always check for a glimpse index first. This heuristic is
however limited, you need the target path to be the base directory where the
.glimpse_index file exists, otherwise pycocci will not recurse below on sub-directories.
If we determine glimpse cannot be used then pycocci will use git rev-parse --toplevel on
the specified path to determine if the provided path is part of a git directory to enable
--use-gitgrep. Although coccinelle provides a fail-safe mechanism to enable use of --use-
gitgrep and fall back onto --use-coccigrep if a git tree is not used, we avoid the failure
case ahead of time. pycocci will be tuned through each release to infer the best indexing
option known to use on your target path.

Further information about spatch is available at
https://coccinelle.gitlabpages.inria.fr/website/.

OPTIONS

       Here is a summary of the options available on pycocci:

SYNOPSIS

       pycocci [-h | --help] [-p |  --profile-cocci]  [-j  |  --jobs]  [-v  |  --verbose]  [-s  |
       --show-proof] <SmPL patch> <target> ]

       -h | --help
              print short help menu

       -p | --profile-cocci
              Profile the <SmPL patch> when run against <target>

       -j | --jobs <job-digit>
              Override  the  default number of jobs to use with <job-digit>. You really only need
              this if the heuristics  for  the  default  number  of  jobs  is  determined  to  be
              incorrect.

       -v | --verbose
              Output all possible information when run. By default pycocci will provide no output
              unless an error is found.

       -s | --show-proof
              Show that the <SmPL-patch.cocci> can  replace  a  series  of  patches.  The  <SmPL-
              patch.cocci>  must have a .cocci postfix. A series of patches are expected to exist
              in the directory  named  after  the  <SmPL-patch.cocci>  with  the  .cocci  postfix
              removed.   pycocci will use git in order to show a proof of equivalence between the
              two. A new git branch will be created and used  to  apply  the  series  of  patches
              ("pycocci-patch-8-random-chars")  onto  the  <target>,  another  git branch will be
              created and used to apply the  <SmPL-patch.cocci>  ("pycocci-smpl-8-random-chars").
              pycocci  will  display  name  of the newly created branches and finally provide the
              output of git diff  --stat  <pycocci-patch-8-random-chars>..<pycocci-smpl-8-random-
              chars>.  You should see no changes, and therefore a perfect SmPL patch equivalence,
              when the diffstat on the output shows no changes. The way to interpret  a  positive
              diffstat  is that the <SmPL-patch.cocci> is introducing more changes than the patch
              series. The way to interpret a negative diffstat is that the <SmPL-patch.cocci>  is
              lacking  some  changes in place on the patch series. pycocci will use git rev-parse
              --toplevel against the <target> to see if the <target> is part of a  git  tree.  If
              the <target> is not part of a git tree a git tree is created for you on <target> if
              <target> is a directory or on the base directory of <target> if <target> is a file.
              pycocci  will  return  you  to the branch previously set on the <target> if one was
              previously set, otherwise if a new git tree was created for you  pycocci  will  set
              the  tree  back  to the master branch. Each run of pycocci --show-proof will create
              two new git branches. 8 random characters are  postixed  to  each  new  git  branch
              created to avoid conflicts with previous runs.

              You must run this option with a clean git tree, if you have any pending changes you
              must commit them or discard them.

              c | --clean-proof This does what --show-proof does but this  is  completely  silent
              unless  an  error occurs. It will also remove the git branches used to test for the
              equivalence proof, unless an error is found. If an error is found you  can  inspect
              the two branches used to test for proof, refer to the documentation on --show-proof
              for details about these branches. This option is useful if  you  want  to  automate
              tests with proofs.

              Note  that  using  this  method  will  have  created and subsequently if successful
              deleted two git branches on your git tree. As a consequence of using  git  branches
              your  git reflog will show these branches, if you push your tree out these branches
              will not be pushed as they were  deleted,  your  reflog  however  will  keep  these
              references  locally  until git expires them, by default this is 30 days. If this is
              too chatty for you, you can run:

                   git reflog expire --all --expire=now

              This will immediately clear old entries from your reflog.

AUTHOR

       pycocci and this man page was written by Luis R. Rodriguez <mcgrof@kernel.org>

REPORTING BUGS

       Send a mail to <cocci@systeme.lip6.fr>

COPYRIGHT

       Copyright 2015, Luis R. Rodriguez pycocci is free software: you can redistribute it and/or
       modify  it  under  the  terms  of  the GNU General Public License as published by the Free
       Software Foundation, according to version 2 of the License.