Provided by: dgit_11.3_all bug

NAME

       dgit - principles of operation

SUMMARY

       dgit  treats  the Debian archive as a version control system, and bidirectionally gateways
       between the archive and git.  The git view of the package can contain the  usual  upstream
       git  history,  and  will  be  augmented  by  commits  representing  uploads  done by other
       developers not using dgit.  This git history is stored in a canonical  location  known  as
       dgit-repos which lives on a dedicated git server.

       git  branches  suitable for use with dgit can be edited directly in git, and used directly
       for building binary packages.  They can be shared using all conventional means for sharing
       git  branches.   It  is  not  necessary  to  use  dgit  to work with dgitish git branches.
       However, dgit is (usually) needed in order to convert  to  or  from  Debian-format  source
       packages.

SEE ALSO

       dgit(1)
              Reference manual and documentation catalogue.

       dgit-*(7)
              Tutorials and workflow guides.  See dgit(1) for a list.

MODEL

       You may use any suitable git workflow with dgit, provided you satisfy dgit's requirements:

       dgit maintains a pseudo-remote called dgit, with one branch per suite.  This remote cannot
       be used with plain git.

       The  dgit-repos  repository  for  each  package  contains  one   ref   per   suite   named
       refs/dgit/suite.  These should be pushed to only by dgit.  They are fast forwarding.  Each
       push on this branch corresponds to an upload (or attempted upload).

       However, it is perfectly fine to have other branches in  dgit-repos;  normally  the  dgit-
       repos repo for the package will be accessible via the remote name `origin'.

       dgit push-* will also make signed tags called archive/debian/version (with version encoded
       a la DEP-14) and push them to dgit-repos.  These are used at the  server  to  authenticate
       pushes.

       Uploads  made  by dgit contain an additional field Dgit in the source package .dsc.  (This
       is added by dgit push-*.)  This specifies: a commit (an ancestor of the dgit/suite branch)
       whose  tree is identical to the unpacked source upload; the distro to which the upload was
       made; a tag name which can be used to fetch the git commits; and a url to use  as  a  hint
       for the dgit git server for that distro.

       Uploads  not made by dgit are represented in git by commits which are synthesised by dgit.
       The tree of each such commit corresponds to the unpacked source; there is  a  commit  with
       the  contents,  and  a pseudo-merge from last known upload - that is, from the contents of
       the dgit/suite branch.  Depending on the source package format, the  contents  commit  may
       have  a more complex structure, but ultimately it will be a convergence of stubby branches
       from origin commits representing the components of the source package.

       dgit expects trees that it works with to have a dgit (pseudo) remote.  This refers to  the
       dgit-created git view of the corresponding archive.

       The  dgit  archive  tracking view is synthesised locally, on demand, by each copy of dgit.
       The tracking view is always a descendant of the dgit-repos suite branch (if  one  exists),
       but  may be ahead of it if uploads have been done without dgit.  The archive tracking view
       is always fast forwarding within each suite.

       dgit push-* can operate on any commit which is a descendant of the suite tracking branch.

       dgit does not make a systematic record of its imports of orig tarball(s).  So it does  not
       work  by finding git tags or branches referring to orig tarball(s).  The orig tarballs are
       downloaded (by dgit clone) into the parent directory, as with a  traditional  (non-gitish)
       dpkg-source  workflow.  You need to retain these tarballs in the parent directory for dgit
       build and dgit push-*.  (They are not needed for purely-git-based workflows.)

       dgit repositories could  be  cloned  with  standard  (git)  methods.   However,  the  dgit
       repositories  do  not  contain  uploads  not  made  with dgit.  And for sourceful builds /
       uploads the orig tarball(s) will need to be present in the parent directory.

       To a user looking at the archive, changes pushed in a simple  NMU  using  dgit  look  like
       reasonable  changes made in an NMU: in a `3.0 (quilt)' package the delta from the previous
       upload is recorded in new patch(es) constructed by dpkg-source.

COMBINED SUITES

       dgit can synthesize a combined view of several underlying suites.  This  is  requested  by
       specifying, for suite, a comma-separated list:

              mainsuite,subsuite...

       This facility is available with dgit clone, fetch and pull, only.

       dgit  will  fetch the same package from each specified underlying suite, separately (as if
       with dgit fetch).  dgit will then generate a pseudomerge commit  on  the  tracking  branch
       remotes/dgit/dgit/suite which has the tip of each of the underlying suites as an ancestor,
       and which contains the same as the suite which has the highest version of the package.

       The package must exist in mainsuite, but need not exist in the subsuites.

       If a specified subsuite starts with - then mainsuite is prepended.

       So, for example, stable,-security means to look for the package  in  stable,  and  stable-
       security,  taking  whichever  is  newer.   If stable is currently jessie, dgit clone would
       leave you on the branch dgit/jessie,-security.

       Combined suites are not supported by the dgit build operations.   This  is  because  those
       options are intended for building for uploading source packages, and look in the changelog
       to find the relevant suite.  It does not make sense to name  a  dgit-synthesised  combined
       suite in a changelog, or to try to upload to it.

       When  using  this  facility, it is important to always specify the same suites in the same
       order: dgit will not make a coherent fast-forwarding history view otherwise.

       The history generated by this feature is not  normally  suitable  for  merging  back  into
       upstreams, as it necessarily contains unattractive pseudomerges.

LIMITATIONS

       Because  the  synthesis  of  the suite tracking branches is done locally based only on the
       current archive state, it will not necessarily see every upload not done with dgit.  Also,
       different  versions  of  dgit  (or  the  software  it  calls)  might import the same .dscs
       differently (although we try to minimise this).  As a consequence, the dgit tracking views
       of the same suite, made by different instances of dgit, may vary.  They will have the same
       contents, but may have different history.

       There is no uniform linkage between the  tracking  branches  for  different  suites.   The
       Debian  infrastructure  does not do any automatic import of uploads made without dgit.  It
       would be possible for a distro's infrastructure to do this; in that case,  different  dgit
       client instances would see exactly the same history.

       There has been no bulk import of historical uploads into Debian's dgit infrastructure.  To
       do this it would be necessary to decide whether to  import  existing  vcs  history  (which
       might  not be faithful to dgit's invariants) or previous non-Dgit uploads (which would not
       provide a very rich history).

       git represents only file executability.  git does not represent empty directories, or  any
       leaf  objects other than plain files and symlinks.  The behaviour of Debian source package
       formats on objects with unusual permissions  is  complicated.   Some  pathological  Debian
       source  packages  will no longer build if empty directories are pruned (or if other things
       not reproduced by git are changed).  Such sources cannot be worked with properly  in  git,
       and therefore not with dgit either.

READ-ONLY DISTROS

       Distros  which do not maintain a set of dgit history git repositories can still be used in
       a read-only mode with dgit.  Currently Ubuntu is configured this way.

GITATTRIBUTES

       git has features which can automatically transform files as they are being copied  between
       the  working tree and the git history.  The attributes can be specified in the source tree
       itself, in .gitattributes.  See gitattributes(5).

       These transformations are context-sensitive and  not,  in  general,  reversible,  so  dgit
       operates  on  the  principle that the dgit git history contains the actual contents of the
       package.  (When dgit is manipulating a .dsc, it does so  in  a  private  area,  where  the
       transforming gitattributes are defused, to achieve this.)

       If  transforming  gitattributes are used, they can cause trouble, because the working tree
       files can differ from the git revision history (and therefore from the  source  packages).
       dgit  warns  if  it  finds a .gitattributes file (in a package being fetched or imported),
       unless the transforming gitattributes have been defused.

       dgit clone and dgit setup-new-tree  disable  transforming  gitattributes  by  default,  by
       creating  a  suitable  .git/info/attributes.   See  dgit  setup-new-tree  and  dgit setup-
       gitattributes in dgit(1).

       Note that dgit does not disable gitattributes unless they would  actually  interfere  with
       your work on dgit branches.  In particular, gitattributes which affect git archive are not
       disabled, so .origs you generate by hand can be wrong.  You  should  consider  using  git-
       deborig (1) which gets this right, suppressing the attributes.

PACKAGE SOURCE FORMATS

       If  you  are  not  the maintainer, you do not need to worry about the source format of the
       package.  You can just make changes as you like in git.  If the package is a `3.0 (quilt)'
       package, the patch stack will usually not be represented in the git history.

FILE EXECUTABILITY

       Debian source package formats do not always faithfully reproduce changes to executability.
       But dgit insists that the result of dgit clone is identical (as far as git can represent -
       see Limitations, above) to the result of dpkg-source -x.

       So  files  that  are executable in your git tree must be executable in the result of dpkg-
       source -x (but often aren't).  If a package has such troublesome files, they  have  to  be
       non-executable in dgit-compatible git branches.

FORMAT 3.0 (QUILT)

       For  a format `3.0 (quilt)' source package, dgit may have to make a commit on your current
       branch to contain metadata used by quilt and dpkg-source.

       This is because `3.0 (quilt)' source  format  represents  the  patch  stack  as  files  in
       debian/patches/  actually  inside the source tree.  This means that, taking the whole tree
       (as seen by git or ls) (i) dpkg-source cannot represent certain trees, and (ii) packing up
       a tree in `3.0 (quilt)' and then unpacking it does not always yield the same tree.

       dgit  will  automatically  work  around  this for you when building and pushing.  The only
       thing you need to know is that dgit build, sbuild, etc., may  make  new  commits  on  your
       HEAD.   If you're not a quilt user this commit won't contain any changes to files you care
       about.

       Simply committing to source files (whether in debian/ or not, but  not  to  patches)  will
       result in a branch that dgit quilt-fixup can linearise.  Other kinds of changes, including
       editing patches or merging, cannot be handled this way.

       You can explicitly request that dgit do just this fixup, by running dgit quilt-fixup.

       If you are a quilt user you need to know  that  dgit's  git  trees  are  `patches  applied
       packaging branches' and do not contain the .pc directory (which is used by quilt to record
       which patches are applied).  If you want to manipulate the patch stack you  probably  want
       to be looking at tools like git-debrebase, gbp pq, or git-dpm.

   quilt fixup error messages
       When dgit's quilt fixup fails, it prints messages like this:

       dgit: base trees orig=5531f03d8456b702eab6 o+d/p=135338e9cc253cc85f84
       dgit: quilt differences: src:  == orig ##     gitignores:  == orig ##
       dgit: quilt differences:      HEAD ## o+d/p               HEAD ## o+d/p
       starting quiltify (multiple patches, linear mode)

       dgit: error: quilt fixup cannot be linear.  Stopped at:
       dgit:  696c9bd5..84ae8f96: changed debian/patches/test-gitignore

       orig   is an import of the .orig tarballs dgit found, with the debian/ directory from your
              HEAD substituted.  This is a git tree object, not a commit: you can pass  its  hash
              to git-diff but not git-log.

       o+d/p  is  another  tree  object,  which  is  the  same  as orig but with the patches from
              debian/patches applied.

       HEAD   is of course your own git HEAD.

       quilt differences
              shows whether each of the these trees differs from the others (i) in upstream files
              excluding  .gitignore  files;  (ii)  in  upstream  .gitignore  files.  == indicates
              equality; ## indicates inequality.

       dgit quilt-fixup --quilt=linear walks commits backwards from your HEAD trying to construct
       a  linear  set of additional patches, starting at the end.  It hopes to eventually find an
       ancestor whose tree is identical to o+d/p in all upstream files.

       In the error message, 696c9bd5..84ae8f96 is  the  first  commit  child-parent  edge  which
       cannot  sensibly  be  either  ignored,  or turned into a patch in debian/patches.  In this
       example, this is because it  itself  changes  files  in  debian/patches,  indicating  that
       something  unusual  is  going  on and that continuing is not safe.  But you might also see
       other kinds of troublesome commit or edge.

       Your appropriate response depends on the cause and the context.  If you have  been  freely
       merging  your  git  branch  and  do not need need a pretty linear patch queue, you can use
       --quilt=single or --quilt=smash.  (Don't use the single-debian-patch  dpkg  source  format
       option;  it has strange properties.)  If you want a pretty linear series, and this message
       is unexpected, it can mean that you  have  unwittingly  committed  changes  that  are  not
       representable  by  dpkg-source  (such  as  some mode changes).  Or maybe you just forgot a
       necessary --quilt= option.

       Finally, this problem can occur if you have provided  Debian  git  tooling  such  as  git-
       debrebase, git-dpm or git-buildpackage with upstream git commit(s) or tag(s) which are not
       100% identical to your orig tarball(s).

SPLIT VIEW AND SPLITTING QUILT MODES

       When working with git branches intended for use with the `3.0 (quilt)' source format  dgit
       can  automatically  convert a suitable maintainer-provided git branch (in one of a variety
       of formats) into a dgit branch.

       When a splitting quilt mode is selected dgit build commands and dgit push-* will, on  each
       invocation,  convert  the  user's  HEAD into the dgit view, so that it can be built and/or
       uploaded.

       Split view mode can also be enabled explicitly with the --split-view command  line  option
       and the .split-view access configuration key.

       When  split  view  is  in  operation,  regardless  of  the  quilt mode, any dgit-generated
       pseudomerges and any quilt fixup commits will appear only in the dgit view.   dgit  push-*
       will  push  the dgit view to the dgit git server.  The dgit view is always a descendant of
       the maintainer view.  dgit push-* will also make a maintainer view tag according to DEP-14
       and push that to the dgit git server.

       Splitting  quilt  modes  must  be enabled explicitly (by the use of the applicable command
       line options, subcommands, or configuration).  This is  because  it  is  not  possible  to
       reliably  tell (for example) whether a git tree for a dpkg-source `3.0 (quilt)' package is
       a patches-applied or patches-unapplied tree.

       Split view conversions are cached in the ref dgit-intern/quilt-cache.  This should not  be
       manipulated directly.

FILES IN THE ORIG TARBALL BUT NOT IN GIT - AUTOTOOLS ETC.

       This section is mainly of interest to maintainers who want to use dgit with their existing
       git history for the Debian package.

       Some developers like to have an extra-clean git tree which lacks files which are  normally
       found  in  source  tarballs  and  therefore in Debian source packages.  For example, it is
       conventional to ship ./configure in the source tarball, but some people prefer not to have
       it present in the git view of their project.

       dgit  requires that the source package unpacks to exactly the same files as are in the git
       commit on which dgit push-* operates.  So if you just try to dgit push-* directly from one
       of these extra-clean git branches, it will fail.

       As the maintainer you therefore have the following options:

       •      Delete the files from your git branches, and your Debian source packages, and carry
              the deletion as a delta from upstream.  (With `3.0 (quilt)' this means representing
              the  deletions  as  patches.  You may need to pass --include-removal to dpkg-source
              --commit, or pass corresponding options to other tools.)  This can make the  Debian
              source package less useful for people without Debian build infrastructure.

       •      Persuade  upstream  that  the  source code in their git history and the source they
              ship as tarballs should be identical.  Of course simply removing the files from the
              tarball may make the tarball hard for people to use.

              One  answer  is to commit the (maybe autogenerated) files, perhaps with some simple
              automation to deal with conflicts and spurious changes.   This  has  the  advantage
              that  someone who clones the git repository finds the program just as easy to build
              as someone who uses the tarball.

       Of course it may also be that the differences are due to build system  bugs,  which  cause
       unintended  files  to  end  up in the source package.  dgit will notice this and complain.
       You may have to fix these bugs before you can unify your existing git history with dgit's.

FILES IN THE SOURCE PACKAGE BUT NOT IN GIT - DOCS, BINARIES ETC.

       Some upstream tarballs contain build artifacts which upstream expects some  users  not  to
       want  to  rebuild  (or  indeed  to  find  hard  to rebuild), but which in Debian we always
       rebuild.

       Examples sometimes include crossbuild  firmware  binaries  and  documentation.   To  avoid
       problems  when  building  updated  source  packages  (in  particular,  to  avoid trying to
       represent as changes in  the  source  package  uninteresting  or  perhaps  unrepresentable
       changes  to  such  files)  many maintainers arrange for the package clean target to delete
       these files.

       dpkg-source does not (with any of the commonly used source formats) represent deletion  of
       binaries (outside debian/) present in upstream.  Thus deleting such files in a dpkg-source
       working tree does not actually result in them being deleted from the source package.  Thus
       deleting the files in rules clean sweeps this problem under the rug.

       However,  git  does  always properly record file deletion.  Since dgit's principle is that
       the dgit git tree is the same of dpkg-source -x, that means  that  a  dgit-compatible  git
       tree always contains these files.

       For the non-maintainer, this can be observed in the following suboptimal occurrences:

       •      The  package  clean  target  often  deletes  these files, making the git tree dirty
              trying to build the source package, etc.  This can be fixed by using dgit  -wg  aka
              --clean=git, so that the package clean target is never run.

       •      The  package  build  modifies  these files, so that builds make the git tree dirty.
              This can be worked around by using `git reset --hard' after each build (or at least
              before each commit or push).

       From  the  maintainer's  point  of  view,  the  main  consequence  is that to make a dgit-
       compatible git branch it is necessary to commit these files to git.  The maintainer has  a
       few  additional options for mitigation: for example, it may be possible for the rules file
       to arrange to do the build in a temporary area,  which  avoids  updating  the  troublesome
       files; they can then be left in the git tree without seeing trouble.

PROBLEMS WITH PACKAGE CLEAN TARGETS ETC.

       A  related  problem  is  other  unexpected  behaviour  by  a package's clean target.  If a
       package's rules modify files which are distributed in the package,  or  simply  forget  to
       remove certain files, dgit will complain that the tree is dirty.

       Again,  the  solution  is to use dgit -wg aka --clean=git, which instructs dgit to use git
       clean instead of the package's build target, along with perhaps git  reset  --hard  before
       each build.

       This  is  100%  reliable, but has the downside that if you forget to git add or to commit,
       and then use dgit -wg or git reset --hard, your changes may be lost.