Provided by: dgit_9.10_all 
NAME
dgit - principles of operation
SUMMARY
dgit treats the Debian archive as a version control system, and bidirectionally gateways
between the archive and git. The git view of the package can contain the usual upstream
git history, and will be augmented by commits representing uploads done by other
developers not using dgit. This git history is stored in a canonical location known as
dgit-repos which lives on a dedicated git server.
git branches suitable for use with dgit can be edited directly in git, and used directly
for building binary packages. They can be shared using all conventional means for sharing
git branches. It is not necessary to use dgit to work with dgitish git branches.
However, dgit is (usually) needed in order to convert to or from Debian-format source
packages.
SEE ALSO
dgit(1)
Reference manual and documentation catalogue.
dgit-*(7)
Tutorials and workflow guides. See dgit(1) for a list.
MODEL
You may use any suitable git workflow with dgit, provided you satisfy dgit's requirements:
dgit maintains a pseudo-remote called dgit, with one branch per suite. This remote cannot
be used with plain git.
The dgit-repos repository for each package contains one ref per suite named
refs/dgit/suite. These should be pushed to only by dgit. They are fast forwarding. Each
push on this branch corresponds to an upload (or attempted upload).
However, it is perfectly fine to have other branches in dgit-repos; normally the dgit-
repos repo for the package will be accessible via the remote name `origin'.
dgit push will also make signed tags called archive/debian/version (with version encoded a
la DEP-14) and push them to dgit-repos. These are used at the server to authenticate
pushes.
Uploads made by dgit contain an additional field Dgit in the source package .dsc. (This
is added by dgit push.) This specifies: a commit (an ancestor of the dgit/suite branch)
whose tree is identical to the unpacked source upload; the distro to which the upload was
made; a tag name which can be used to fetch the git commits; and a url to use as a hint
for the dgit git server for that distro.
Uploads not made by dgit are represented in git by commits which are synthesised by dgit.
The tree of each such commit corresponds to the unpacked source; there is a commit with
the contents, and a pseudo-merge from last known upload - that is, from the contents of
the dgit/suite branch. Depending on the source package format, the contents commit may
have a more complex structure, but ultimately it will be a convergence of stubby branches
from origin commits representing the components of the source package.
dgit expects trees that it works with to have a dgit (pseudo) remote. This refers to the
dgit-created git view of the corresponding archive.
The dgit archive tracking view is synthesised locally, on demand, by each copy of dgit.
The tracking view is always a descendant of the dgit-repos suite branch (if one exists),
but may be ahead of it if uploads have been done without dgit. The archive tracking view
is always fast forwarding within each suite.
dgit push can operate on any commit which is a descendant of the suite tracking branch.
dgit does not make a systematic record of its imports of orig tarball(s). So it does not
work by finding git tags or branches referring to orig tarball(s). The orig tarballs are
downloaded (by dgit clone) into the parent directory, as with a traditional (non-gitish)
dpkg-source workflow. You need to retain these tarballs in the parent directory for dgit
build and dgit push. (They are not needed for purely-git-based workflows.)
dgit repositories could be cloned with standard (git) methods. However, the dgit
repositories do not contain uploads not made with dgit. And for sourceful builds /
uploads the orig tarball(s) will need to be present in the parent directory.
To a user looking at the archive, changes pushed in a simple NMU using dgit look like
reasonable changes made in an NMU: in a `3.0 (quilt)' package the delta from the previous
upload is recorded in new patch(es) constructed by dpkg-source.
COMBINED SUITES
dgit can synthesize a combined view of several underlying suites. This is requested by
specifying, for suite, a comma-separated list:
mainsuite,subsuite...
This facility is available with dgit clone, fetch and pull, only.
dgit will fetch the same package from each specified underlying suite, separately (as if
with dgit fetch). dgit will then generate a pseudomerge commit on the tracking branch
remotes/dgit/dgit/suite which has the tip of each of the underlying suites as an ancestor,
and which contains the same as the suite which has the highest version of the package.
The package must exist in mainsuite, but need not exist in the subsuites.
If a specified subsuite starts with - then mainsuite is prepended.
So, for example, stable,-security means to look for the package in stable, and stable-
security, taking whichever is newer. If stable is currently jessie, dgit clone would
leave you on the branch dgit/jessie,-security.
Combined suites are not supported by the dgit build operations. This is because those
options are intended for building for uploading source packages, and look in the changelog
to find the relevant suite. It does not make sense to name a dgit-synthesised combined
suite in a changelog, or to try to upload to it.
When using this facility, it is important to always specify the same suites in the same
order: dgit will not make a coherent fast-forwarding history view otherwise.
The history generated by this feature is not normally suitable for merging back into
upstreams, as it necessarily contains unattractive pseudomerges.
LIMITATIONS
Because the synthesis of the suite tracking branches is done locally based only on the
current archive state, it will not necessarily see every upload not done with dgit. Also,
different versions of dgit (or the software it calls) might import the same .dscs
differently (although we try to minimise this). As a consequence, the dgit tracking views
of the same suite, made by different instances of dgit, may vary. They will have the same
contents, but may have different history.
There is no uniform linkage between the tracking branches for different suites. The
Debian infrastructure does not do any automatic import of uploads made without dgit. It
would be possible for a distro's infrastructure to do this; in that case, different dgit
client instances would see exactly the same history.
There has been no bulk import of historical uploads into Debian's dgit infrastructure. To
do this it would be necessary to decide whether to import existing vcs history (which
might not be faithful to dgit's invariants) or previous non-Dgit uploads (which would not
provide a very rich history).
git represents only file executability. git does not represent empty directories, or any
leaf objects other than plain files and symlinks. The behaviour of Debian source package
formats on objects with unusual permissions is complicated. Some pathological Debian
source packages will no longer build if empty directories are pruned (or if other things
not reproduced by git are changed). Such sources cannot be worked with properly in git,
and therefore not with dgit either.
READ-ONLY DISTROS
Distros which do not maintain a set of dgit history git repositories can still be used in
a read-only mode with dgit. Currently Ubuntu is configured this way.
GITATTRIBUTES
git has features which can automatically transform files as they are being copied between
the working tree and the git history. The attributes can be specified in the source tree
itself, in .gitattributes. See gitattributes(5).
These transformations are context-sensitive and not, in general, reversible, so dgit
operates on the principle that the dgit git history contains the actual contents of the
package. (When dgit is manipulating a .dsc, it does so in a private area, where the
transforming gitattributes are defused, to achieve this.)
If transforming gitattributes are used, they can cause trouble, because the working tree
files can differ from the git revision history (and therefore from the source packages).
dgit warns if it finds a .gitattributes file (in a package being fetched or imported),
unless the transforming gitattributes have been defused.
dgit clone and dgit setup-new-tree disable transforming gitattributes by default, by
creating a suitable .git/info/attributes. See dgit setup-new-tree and dgit setup-
gitattributes in dgit(1).
Note that dgit does not disable gitattributes unless they would actually interfere with
your work on dgit branches. In particular, gitattributes which affect git archive are not
disabled, so .origs you generate by hand can be wrong. You should consider using git-
deborig (1) which gets this right, suppressing the attributes.
PACKAGE SOURCE FORMATS
If you are not the maintainer, you do not need to worry about the source format of the
package. You can just make changes as you like in git. If the package is a `3.0 (quilt)'
package, the patch stack will usually not be represented in the git history.
FILE EXECUTABILITY
Debian source package formats do not always faithfully reproduce changes to executability.
But dgit insists that the result of dgit clone is identical (as far as git can represent -
see Limitations, above) to the result of dpkg-source -x.
So files that are executable in your git tree must be executable in the result of dpkg-
source -x (but often aren't). If a package has such troublesome files, they have to be
non-executable in dgit-compatible git branches.
FORMAT 3.0 (QUILT)
For a format `3.0 (quilt)' source package, dgit may have to make a commit on your current
branch to contain metadata used by quilt and dpkg-source.
This is because `3.0 (quilt)' source format represents the patch stack as files in
debian/patches/ actually inside the source tree. This means that, taking the whole tree
(as seen by git or ls) (i) dpkg-source cannot represent certain trees, and (ii) packing up
a tree in `3.0 (quilt)' and then unpacking it does not always yield the same tree.
dgit will automatically work around this for you when building and pushing. The only
thing you need to know is that dgit build, sbuild, etc., may make new commits on your
HEAD. If you're not a quilt user this commit won't contain any changes to files you care
about.
Simply committing to source files (whether in debian/ or not, but not to patches) will
result in a branch that dgit quilt-fixup can linearise. Other kinds of changes, including
editing patches or merging, cannot be handled this way.
You can explicitly request that dgit do just this fixup, by running dgit quilt-fixup.
If you are a quilt user you need to know that dgit's git trees are `patches applied
packaging branches' and do not contain the .pc directory (which is used by quilt to record
which patches are applied). If you want to manipulate the patch stack you probably want
to be looking at tools like git-debrebase, gbp pq, or git-dpm.
quilt fixup error messages
When dgit's quilt fixup fails, it prints messages like this:
dgit: base trees orig=5531f03d8456b702eab6 o+d/p=135338e9cc253cc85f84
dgit: quilt differences: src: == orig ## gitignores: == orig ##
dgit: quilt differences: HEAD ## o+d/p HEAD ## o+d/p
starting quiltify (multiple patches, linear mode)
dgit: error: quilt fixup cannot be linear. Stopped at:
dgit: 696c9bd5..84ae8f96: changed debian/patches/test-gitignore
orig is an import of the .orig tarballs dgit found, with the debian/ directory from your
HEAD substituted. This is a git tree object, not a commit: you can pass its hash
to git-diff but not git-log.
o+d/p is another tree object, which is the same as orig but with the patches from
debian/patches applied.
HEAD is of course your own git HEAD.
quilt differences
shows whether each of the these trees differs from the others (i) in upstream files
excluding .gitignore files; (ii) in upstream .gitignore files. == indicates
equality; ## indicates inequality.
dgit quilt-fixup --quilt=linear walks commits backwards from your HEAD trying to construct
a linear set of additional patches, starting at the end. It hopes to eventually find an
ancestor whose tree is identical to o+d/p in all upstream files.
In the error message, 696c9bd5..84ae8f96 is the first commit child-parent edge which
cannot sensibly be either ignored, or turned into a patch in debian/patches. In this
example, this is because it itself changes files in debian/patches, indicating that
something unusual is going on and that continuing is not safe. But you might also see
other kinds of troublesome commit or edge.
Your appropriate response depends on the cause and the context. If you have been freely
merging your git branch and do not need need a pretty linear patch queue, you can use
--quilt=smash (or use the 1.0 or single-debian-patch source formats; see dpkg-source(1).)
If you want a pretty linear series, and this message is unexpected, it can mean that you
have unwittingly committed changes that are not representable by dpkg-source (such as some
mode changes). Or maybe you just forgot a necessary --quilt= option.
Finally, this problem can occur if you have provided Debian git tooling such as git-
debrebase, git-dpm or git-buildpackage with upstream git commit(s) or tag(s) which are not
100% identical to your orig tarball(s).
SPLIT VIEW AND SPLITTING QUILT MODES
When working with git branches intended for use with the `3.0 (quilt)' source format dgit
can automatically convert a suitable maintainer-provided git branch (in one of a variety
of formats) into a dgit branch.
When a splitting quilt mode is selected dgit build commands and dgit push will, on each
invocation, convert the user's HEAD into the dgit view, so that it can be built and/or
uploaded.
Split view mode can also be enabled explicitly with the --split-view command line option
and the .split-view access configuration key.
When split view is in operation, regardless of the quilt mode, any dgit-generated
pseudomerges and any quilt fixup commits will appear only in the dgit view. dgit push
will push the dgit view to the dgit git server. The dgit view is always a descendant of
the maintainer view. dgit push will also make a maintainer view tag according to DEP-14
and push that to the dgit git server.
Splitting quilt modes must be enabled explicitly (by the use of the applicable command
line options, subcommands, or configuration). This is because it is not possible to
reliably tell (for example) whether a git tree for a dpkg-source `3.0 (quilt)' package is
a patches-applied or patches-unapplied tree.
Split view conversions are cached in the ref dgit-intern/quilt-cache. This should not be
manipulated directly.
FILES IN THE ORIG TARBALL BUT NOT IN GIT - AUTOTOOLS ETC.
This section is mainly of interest to maintainers who want to use dgit with their existing
git history for the Debian package.
Some developers like to have an extra-clean git tree which lacks files which are normally
found in source tarballs and therefore in Debian source packages. For example, it is
conventional to ship ./configure in the source tarball, but some people prefer not to have
it present in the git view of their project.
dgit requires that the source package unpacks to exactly the same files as are in the git
commit on which dgit push operates. So if you just try to dgit push directly from one of
these extra-clean git branches, it will fail.
As the maintainer you therefore have the following options:
• Delete the files from your git branches, and your Debian source packages, and carry
the deletion as a delta from upstream. (With `3.0 (quilt)' this means representing
the deletions as patches. You may need to pass --include-removal to dpkg-source
--commit, or pass corresponding options to other tools.) This can make the Debian
source package less useful for people without Debian build infrastructure.
• Persuade upstream that the source code in their git history and the source they
ship as tarballs should be identical. Of course simply removing the files from the
tarball may make the tarball hard for people to use.
One answer is to commit the (maybe autogenerated) files, perhaps with some simple
automation to deal with conflicts and spurious changes. This has the advantage
that someone who clones the git repository finds the program just as easy to build
as someone who uses the tarball.
Of course it may also be that the differences are due to build system bugs, which cause
unintended files to end up in the source package. dgit will notice this and complain.
You may have to fix these bugs before you can unify your existing git history with dgit's.
FILES IN THE SOURCE PACKAGE BUT NOT IN GIT - DOCS, BINARIES ETC.
Some upstream tarballs contain build artifacts which upstream expects some users not to
want to rebuild (or indeed to find hard to rebuild), but which in Debian we always
rebuild.
Examples sometimes include crossbuild firmware binaries and documentation. To avoid
problems when building updated source packages (in particular, to avoid trying to
represent as changes in the source package uninteresting or perhaps unrepresentable
changes to such files) many maintainers arrange for the package clean target to delete
these files.
dpkg-source does not (with any of the commonly used source formats) represent deletion of
binaries (outside debian/) present in upstream. Thus deleting such files in a dpkg-source
working tree does not actually result in them being deleted from the source package. Thus
deleting the files in rules clean sweeps this problem under the rug.
However, git does always properly record file deletion. Since dgit's principle is that
the dgit git tree is the same of dpkg-source -x, that means that a dgit-compatible git
tree always contains these files.
For the non-maintainer, this can be observed in the following suboptimal occurrences:
• The package clean target often deletes these files, making the git tree dirty
trying to build the source package, etc. This can be fixed by using dgit -wg aka
--clean=git, so that the package clean target is never run.
• The package build modifies these files, so that builds make the git tree dirty.
This can be worked around by using `git reset --hard' after each build (or at least
before each commit or push).
From the maintainer's point of view, the main consequence is that to make a dgit-
compatible git branch it is necessary to commit these files to git. The maintainer has a
few additional options for mitigation: for example, it may be possible for the rules file
to arrange to do the build in a temporary area, which avoids updating the troublesome
files; they can then be left in the git tree without seeing trouble.
PROBLEMS WITH PACKAGE CLEAN TARGETS ETC.
A related problem is other unexpected behaviour by a package's clean target. If a
package's rules modify files which are distributed in the package, or simply forget to
remove certain files, dgit will complain that the tree is dirty.
Again, the solution is to use dgit -wg aka --clean=git, which instructs dgit to use git
clean instead of the package's build target, along with perhaps git reset --hard before
each build.
This is 100% reliable, but has the downside that if you forget to git add or to commit,
and then use dgit -wg or git reset --hard, your changes may be lost.