Provided by: dgit_4.3_all bug

NAME

       dgit - principles of operation

SUMMARY

       dgit  treats  the Debian archive as a version control system, and bidirectionally gateways
       between the archive and git.  The git view of the package can contain the  usual  upstream
       git  history,  and  will  be  augmented  by  commits  representing  uploads  done by other
       developers not using dgit.  This git history is stored in a canonical  location  known  as
       dgit-repos which lives on a dedicated git server.

       git  branches  suitable for use with dgit can be edited directly in git, and used directly
       for building binary packages.  They can be shared using all conventional means for sharing
       git  branches.   It  is  not  necessary  to  use  dgit  to work with dgitish git branches.
       However, dgit is (usually) needed in order to convert  to  or  from  Debian-format  source
       packages.

SEE ALSO

       dgit(1)
              Reference manual and documentation catalogue.

       dgit-*(7)
              Tutorials and workflow guides.  See dgit(1) for a list.

MODEL

       You may use any suitable git workflow with dgit, provided you satisfy dgit's requirements:

       dgit maintains a pseudo-remote called dgit, with one branch per suite.  This remote cannot
       be used with plain git.

       The  dgit-repos  repository  for  each  package  contains  one   ref   per   suite   named
       refs/dgit/suite.  These should be pushed to only by dgit.  They are fast forwarding.  Each
       push on this branch corresponds to an upload (or attempted upload).

       However, it is perfectly fine to have other branches in  dgit-repos;  normally  the  dgit-
       repos repo for the package will be accessible via the remote name `origin'.

       dgit push will also make signed tags called archive/debian/version (with version encoded a
       la DEP-14) and push them to dgit-repos.  These are used  at  the  server  to  authenticate
       pushes.

       Uploads  made  by dgit contain an additional field Dgit in the source package .dsc.  (This
       is added by dgit push.)  This specifies: a commit (an ancestor of the  dgit/suite  branch)
       whose  tree is identical to the unpacked source upload; the distro to which the upload was
       made; a tag name which can be used to fetch the git commits; and a url to use  as  a  hint
       for the dgit git server for that distro.

       Uploads  not made by dgit are represented in git by commits which are synthesised by dgit.
       The tree of each such commit corresponds to the unpacked source; there is  a  commit  with
       the  contents,  and  a pseudo-merge from last known upload - that is, from the contents of
       the dgit/suite branch.  Depending on the source package format, the  contents  commit  may
       have  a more complex structure, but ultimately it will be a convergence of stubby branches
       from origin commits representing the components of the source package.

       dgit expects trees that it works with to have a dgit (pseudo) remote.  This refers to  the
       dgit-created git view of the corresponding archive.

       The  dgit  archive  tracking view is synthesised locally, on demand, by each copy of dgit.
       The tracking view is always a descendant of the dgit-repos suite branch (if  one  exists),
       but  may be ahead of it if uploads have been done without dgit.  The archive tracking view
       is always fast forwarding within each suite.

       dgit push can operate on any commit which is a descendant of the suite tracking branch.

       dgit does not make a systematic record of its imports of orig tarball(s).  So it does  not
       work  by finding git tags or branches referring to orig tarball(s).  The orig tarballs are
       downloaded (by dgit clone) into the parent directory, as with a  traditional  (non-gitish)
       dpkg-source  workflow.  You need to retain these tarballs in the parent directory for dgit
       build and dgit push.  (They are not needed for purely-git-based workflows.)

       dgit repositories could  be  cloned  with  standard  (git)  methods.   However,  the  dgit
       repositories  do  not  contain  uploads  not  made  with dgit.  And for sourceful builds /
       uploads the orig tarball(s) will need to be present in the parent directory.

       To a user looking at the archive, changes pushed in a simple  NMU  using  dgit  look  like
       reasonable  changes made in an NMU: in a `3.0 (quilt)' package the delta from the previous
       upload is recorded in new patch(es) constructed by dpkg-source.

COMBINED SUITES

       dgit can synthesize a combined view of several underlying suites.  This  is  requested  by
       specifying, for suite, a comma-separated list:

              mainsuite,subsuite...

       This facility is available with dgit clone, fetch and pull, only.

       dgit  will  fetch the same package from each specified underlying suite, separately (as if
       with dgit fetch).  dgit will then generate a pseudomerge commit  on  the  tracking  branch
       remotes/dgit/dgit/suite which has the tip of each of the underlying suites as an ancestor,
       and which contains the same as the suite which has the highest version of the package.

       The package must exist in mainsuite, but need not exist in the subsuites.

       If a specified subsuite starts with - then mainsuite is prepended.

       So, for example, stable,-security means to look for the package  in  stable,  and  stable-
       security,  taking  whichever  is  newer.   If stable is currently jessie, dgit clone would
       leave you on the branch dgit/jessie,-security.

       Combined suites are not supported by the dgit build operations.   This  is  because  those
       options are intended for building for uploading source packages, and look in the changelog
       to find the relevant suite.  It does not make sense to name  a  dgit-synthesised  combined
       suite in a changelog, or to try to upload to it.

       When  using  this  facility, it is important to always specify the same suites in the same
       order: dgit will not be make a coherent fast-forwarding history view otherwise.

       The history generated by this feature is not  normally  suitable  for  merging  back  into
       upstreams, as it necessarily contains unattractive pseudomerges.

LIMITATIONS

       Because  the  synthesis  of  the suite tracking branches is done locally based only on the
       current archive state, it will not necessarily see every upload not done with dgit.  Also,
       different  versions  of  dgit  (or  the  software  it  calls)  might import the same .dscs
       differently (although we try to minimise this).  As a consequence, the dgit tracking views
       of the same suite, made by different instances of dgit, may vary.  They will have the same
       contents, but may have different history.

       There is no uniform linkage between the  tracking  branches  for  different  suites.   The
       Debian  infrastructure  does not do any automatic import of uploads made without dgit.  It
       would be possible for a distro's infrastructure to do this; in that case,  different  dgit
       client instances would see exactly the same history.

       There has been no bulk import of historical uploads into Debian's dgit infrastructure.  To
       do this it would be necessary to decide whether to  import  existing  vcs  history  (which
       might  not be faithful to dgit's invariants) or previous non-Dgit uploads (which would not
       provide a very rich history).

       git represents only file executability.  git does not represent empty directories, or  any
       leaf  objects other than plain files and symlinks.  The behaviour of Debian source package
       formats on objects with unusual permissions  is  complicated.   Some  pathological  Debian
       source  packages  will no longer build if empty directories are pruned (or if other things
       not reproduced by git are changed).  Such sources cannot be worked with properly  in  git,
       and therefore not with dgit either.

READ-ONLY DISTROS

       Distros  which do not maintain a set of dgit history git repositories can still be used in
       a read-only mode with dgit.  Currently Ubuntu is configured this way.

GITATTRIBUTES

       git has features which can automatically transform files as they are being copied  between
       the  working tree and the git history.  The attributes can be specified in the source tree
       itself, in .gitattributes.  See gitattributes(5).

       These transformations are context-sensitive and  not,  in  general,  reversible,  so  dgit
       operates  on  the  principle that the dgit git history contains the actual contents of the
       package.  (When dgit is manipulating a .dsc, it does so  in  a  private  area,  where  the
       transforming gitattributes are defused, to achieve this.)

       If  transforming  gitattributes are used, they can cause trouble, because the working tree
       files can differ from the git revision history (and therefore from the  source  packages).
       dgit  warns  if  it  finds a .gitattributes file (in a package being fetched or imported),
       unless the transforming gitattributes have been defused.

       dgit clone and dgit setup-new-tree  disable  transforming  gitattributes  by  default,  by
       creating  a  suitable  .git/info/attributes.   See  dgit  setup-new-tree  and  dgit setup-
       gitattributes in dgit(1).

PACKAGE SOURCE FORMATS

       If you are not the maintainer, you do not need to worry about the  source  format  of  the
       package.  You can just make changes as you like in git.  If the package is a `3.0 (quilt)'
       package, the patch stack will usually not be represented in the git history.

FILE EXECUTABILITY

       Debian source package formats do not always faithfully reproduce changes to executability.
       But dgit insists that the result of dgit clone is identical (as far as git can represent -
       see Limitations, above) to the result of dpkg-source -x.

       So files that are executable in your git tree must be executable in the  result  of  dpkg-
       source  -x  (but  often aren't).  If a package has such troublesome files, they have to be
       non-executable in dgit-compatible git branches.

FORMAT 3.0 (QUILT)

       For a format `3.0 (quilt)' source package, dgit may have to make a commit on your  current
       branch to contain metadata used by quilt and dpkg-source.

       This  is  because  `3.0  (quilt)'  source  format  represents  the patch stack as files in
       debian/patches/ actually inside the source tree.  This means that, taking the  whole  tree
       (as seen by git or ls) (i) dpkg-source cannot represent certain trees, and (ii) packing up
       a tree in `3.0 (quilt)' and then unpacking it does not always yield the same tree.

       dgit will automatically work around this for you when  building  and  pushing.   The  only
       thing  you  need  to  know  is that dgit build, sbuild, etc., may make new commits on your
       HEAD.  If you're not a quilt user this commit won't contain any changes to files you  care
       about.

       You can explicitly request that dgit do just this fixup, by running dgit quilt-fixup.

       If  you  are  a  quilt  user  you  need to know that dgit's git trees are `patches applied
       packaging branches' and do not contain the .pc directory (which is used by quilt to record
       which  patches  are applied).  If you want to manipulate the patch stack you probably want
       to be looking at tools like git-dpm.

SPLIT VIEW QUILT MODE

       When working with git branches intended for use with the `3.0 (quilt)' source format  dgit
       can  automatically  convert a suitable maintainer-provided git branch (in one of a variety
       of formats) into a dgit branch.

       When a split view mode is engaged  dgit  build  commands  and  dgit  push  will,  on  each
       invocation,  convert  the  user's  HEAD into the dgit view, so that it can be built and/or
       uploaded.

       dgit push in split view mode will push the dgit view to the dgit  git  server.   The  dgit
       view is always a descendant of the maintainer view.  dgit push will also make a maintainer
       view tag according to DEP-14 and push that to the dgit git server.

       Split view mode must be enabled explicitly (by the use  of  the  applicable  command  line
       options,  subcommands,  or configuration).  This is because it is not possible to reliably
       tell (for example) whether a git tree  for  a  dpkg-source  `3.0  (quilt)'  package  is  a
       patches-applied or patches-unapplied tree.

       Split  view conversions are cached in the ref dgit-intern/quilt-cache.  This should not be
       manipulated directly.

FILES IN THE ORIG TARBALL BUT NOT IN GIT - AUTOTOOLS ETC.

       This section is mainly of interest to maintainers who want to use dgit with their existing
       git history for the Debian package.

       Some  developers like to have an extra-clean git tree which lacks files which are normally
       found in source tarballs and therefore in Debian source  packages.   For  example,  it  is
       conventional to ship ./configure in the source tarball, but some people prefer not to have
       it present in the git view of their project.

       dgit requires that the source package unpacks to exactly the same files as are in the  git
       commit  on which dgit push operates.  So if you just try to dgit push directly from one of
       these extra-clean git branches, it will fail.

       As the maintainer you therefore have the following options:

       •      Delete the files from your git branches, and your Debian source packages, and carry
              the  deletion as a delta from upstream.  (With `3.0 (quilt)' this means represeting
              the deletions as patches.  You may need to pass  --include-removal  to  dpkg-source
              --commit,  or pass corresponding options to other tools.)  This can make the Debian
              source package less useful for people without Debian build infrastructure.

       •      Persuade upstream that the source code in their git history  and  the  source  they
              ship as tarballs should be identical.  Of course simply removing the files from the
              tarball may make the tarball hard for people to use.

              One answer is to commit the (maybe autogenerated) files, perhaps with  some  simple
              automation  to  deal  with  conflicts and spurious changes.  This has the advantage
              that someone who clones the git repository finds the program just as easy to  build
              as someone who uses the tarball.

       Of  course  it  may also be that the differences are due to build system bugs, which cause
       unintended files to end up in the source package.  dgit will  notice  this  and  complain.
       You may have to fix these bugs before you can unify your existing git history with dgit's.

FILES IN THE SOURCE PACKAGE BUT NOT IN GIT - DOCS, BINARIES ETC.

       Some  upstream  tarballs  contain build artifacts which upstream expects some users not to
       want to rebuild (or indeed to find hard  to  rebuild),  but  which  in  Debian  we  always
       rebuild.

       Examples  sometimes  include  crossbuild  firmware  binaries  and documentation.  To avoid
       problems when building  updated  source  packages  (in  particular,  to  avoid  trying  to
       represent  as  changes  in  the  source  package  uninteresting or perhaps unrepresentable
       changes to such files) many maintainers arrange for the package  clean  target  to  delete
       these files.

       dpkg-source  does not (with any of the commonly used source formats) represent deletion of
       binaries (outside debian/) present in upstream.  Thus deleting such files in a dpkg-source
       working tree does not actually result in them being deleted from the source package.  Thus
       deleting the files in rules clean sweeps this problem under the rug.

       However, git does always properly record file deletion.  Since dgit's  principle  is  that
       the  dgit  git  tree  is the same of dpkg-source -x, that means that a dgit-compatible git
       tree always contains these files.

       For the non-maintainer, this can be observed in the following suboptimal occurrences:

       •      The package clean target often deletes these  files,  making  the  git  tree  dirty
              trying  to  build the source package, etc.  This can be fixed by using dgit -wg aka
              --clean=git, so that the package clean target is never run.

       •      The package build modifies these files, so that builds make  the  git  tree  dirty.
              This can be worked around by using `git reset --hard' after each build (or at least
              before each commit or push).

       From the maintainer's point of view,  the  main  consequence  is  that  to  make  a  dgit-
       compatible  git branch it is necessary to commit these files to git.  The maintainer has a
       few additional options for mitigation: for example, it may be possible for the rules  file
       to  arrange  to  do  the  build in a temporary area, which avoids updating the troublesome
       files; they can then be left in the git tree without seeing trouble.

PROBLEMS WITH PACKAGE CLEAN TARGETS ETC.

       A related problem is other unexpected  behaviour  by  a  package's  clean  target.   If  a
       package's  rules  modify  files  which are distributed in the package, or simply forget to
       remove certain files, dgit will complain that the tree is dirty.

       Again, the solution is to use dgit -wg aka --clean=git, which instructs dgit  to  use  git
       clean  instead  of  the package's build target, along with perhaps git reset --hard before
       each build.

       This is 100% reliable, but has the downside that if you forget to git add  or  to  commit,
       and then use dgit -wg or git reset --hard, your changes may be lost.