Provided by: borgbackup2_2.0.0b5-1_amd64 
NAME
borg-patterns - Details regarding patterns
DESCRIPTION
When specifying one or more file paths in a Borg command that supports patterns for the
respective option or argument, you can apply the patterns described here to include only
desired files and/or exclude unwanted ones. Patterns can be used
• for --exclude option,
• in the file given with --exclude-from option,
• for --pattern option,
• in the file given with --patterns-from option and
• for PATH arguments that explicitly support them.
Borg always stores all file paths normalized and relative to the current recursion root.
The recursion root is also named PATH in Borg commands like borg create that do a file
discovery, so do not confuse the root with the PATH argument of e.g. borg extract.
Starting with Borg 1.2, paths that are matched against patterns always appear relative. If
you give /absolute/ as root, the paths going into the matcher will look relative like
absolute/.../file.ext. If you give ../some/path as root, the paths will look like
some/path/.../file.ext.
File patterns support five different styles. If followed by a colon ':', the first two
characters of a pattern are used as a style selector. Explicit style selection is
necessary if a non-default style is desired or when the desired pattern starts with two
alphanumeric characters followed by a colon (i.e. aa:something/*).
Fnmatch, selector fm:
This is the default style for --exclude and --exclude-from. These patterns use a
variant of shell pattern syntax, with '*' matching any number of characters, '?'
matching any single character, '[...]' matching any single character specified,
including ranges, and '[!...]' matching any character not specified. For the
purpose of these patterns, the path separator (backslash for Windows and '/' on
other systems) is not treated specially. Wrap meta-characters in brackets for a
literal match (i.e. [?] to match the literal character '?'). For a path to match a
pattern, the full path must match, or it must match from the start of the full path
to just before a path separator. Except for the root path, paths will never end in
the path separator when matching is attempted. Thus, if a given pattern ends in a
path separator, a '*' is appended before matching is attempted. A leading path
separator is always removed.
Shell-style patterns, selector sh:
This is the default style for --pattern and --patterns-from. Like fnmatch patterns
these are similar to shell patterns. The difference is that the pattern may include
**/ for matching zero or more directory levels, * for matching zero or more
arbitrary characters with the exception of any path separator. A leading path
separator is always removed.
Regular expressions, selector re:
Unlike shell patterns, regular expressions are not required to match the full path
and any substring match is sufficient. It is strongly recommended to anchor
patterns to the start ('^'), to the end ('$') or both. Path separators (backslash
for Windows and '/' on other systems) in paths are always normalized to a forward
slash '/' before applying a pattern.
Path prefix, selector pp:
This pattern style is useful to match whole sub-directories. The pattern
pp:root/somedir matches root/somedir and everything therein. A leading path
separator is always removed.
Path full-match, selector pf:
This pattern style is (only) useful to match full paths. This is kind of a pseudo
pattern as it can not have any variable or unspecified parts - the full path must
be given. pf:root/file.ext matches root/file.ext only. A leading path separator is
always removed.
Implementation note: this is implemented via very time-efficient O(1) hashtable
lookups (this means you can have huge amounts of such patterns without impacting
performance much). Due to that, this kind of pattern does not respect any context
or order. If you use such a pattern to include a file, it will always be included
(if the directory recursion encounters it). Other include/exclude patterns that
would normally match will be ignored. Same logic applies for exclude.
NOTE:
re:, sh: and fm: patterns are all implemented on top of the Python SRE engine. It is
very easy to formulate patterns for each of these types which requires an inordinate
amount of time to match paths. If untrusted users are able to supply patterns, ensure
they cannot supply re: patterns. Further, ensure that sh: and fm: patterns only contain
a handful of wildcards at most.
Exclusions can be passed via the command line option --exclude. When used from within a
shell, the patterns should be quoted to protect them from expansion.
The --exclude-from option permits loading exclusion patterns from a text file with one
pattern per line. Lines empty or starting with the hash sign '#' after removing whitespace
on both ends are ignored. The optional style selector prefix is also supported for
patterns loaded from a file. Due to whitespace removal, paths with whitespace at the
beginning or end can only be excluded using regular expressions.
To test your exclusion patterns without performing an actual backup you can run borg
create --list --dry-run ....
Examples:
# Exclude '/home/user/file.o' but not '/home/user/file.odt':
$ borg create -e '*.o' archive /
# Exclude '/home/user/junk' and '/home/user/subdir/junk' but
# not '/home/user/importantjunk' or '/etc/junk':
$ borg create -e 'home/*/junk' archive /
# Exclude the contents of '/home/user/cache' but not the directory itself:
$ borg create -e home/user/cache/ archive /
# The file '/home/user/cache/important' is *not* backed up:
$ borg create -e home/user/cache/ archive / /home/user/cache/important
# The contents of directories in '/home' are not backed up when their name
# ends in '.tmp'
$ borg create --exclude 're:^home/[^/]+\.tmp/' archive /
# Load exclusions from file
$ cat >exclude.txt <<EOF
# Comment line
home/*/junk
*.tmp
fm:aa:something/*
re:^home/[^/]+\.tmp/
sh:home/*/.thumbnails
# Example with spaces, no need to escape as it is processed by borg
some file with spaces.txt
EOF
$ borg create --exclude-from exclude.txt archive /
A more general and easier to use way to define filename matching patterns exists with the
--pattern and --patterns-from options. Using these, you may specify the backup roots,
default pattern styles and patterns for inclusion and exclusion.
Root path prefix R
A recursion root path starts with the prefix R, followed by a path (a plain path,
not a file pattern). Use this prefix to have the root paths in the patterns file
rather than as command line arguments.
Pattern style prefix P
To change the default pattern style, use the P prefix, followed by the pattern
style abbreviation (fm, pf, pp, re, sh). All patterns following this line will use
this style until another style is specified.
Exclude pattern prefix -
Use the prefix -, followed by a pattern, to define an exclusion. This has the same
effect as the --exclude option.
Exclude no-recurse pattern prefix !
Use the prefix !, followed by a pattern, to define an exclusion that does not
recurse into subdirectories. This saves time, but prevents include patterns to
match any files in subdirectories.
Include pattern prefix +
Use the prefix +, followed by a pattern, to define inclusions. This is useful to
include paths that are covered in an exclude pattern and would otherwise not be
backed up.
The first matching pattern is used, so if an include pattern matches before an exclude
pattern, the file is backed up. Note that a no-recurse exclude stops examination of
subdirectories so that potential includes will not match - use normal excludes for such
use cases.
Tip: You can easily test your patterns with --dry-run and --list:
$ borg create --dry-run --list --patterns-from patterns.txt archive
This will list the considered files one per line, prefixed with a character that indicates
the action (e.g. 'x' for excluding, see Item flags in borg create usage docs).
NOTE:
It's possible that a sub-directory/file is matched while parent directories are not. In
that case, parent directories are not backed up and thus their user, group, permission,
etc. cannot be restored.
Patterns (--pattern) and excludes (--exclude) from the command line are considered first
(in the order of appearance). Then patterns from --patterns-from are added. Exclusion
patterns from --exclude-from files are appended last.
Examples:
# back up pics, but not the ones from 2018, except the good ones:
# note: using = is essential to avoid cmdline argument parsing issues.
borg create --pattern=+pics/2018/good --pattern=-pics/2018 archive pics
# back up only JPG/JPEG files (case insensitive) in all home directories:
borg create --pattern '+ re:\.jpe?g(?i)$' archive /home
# back up homes, but exclude big downloads (like .ISO files) or hidden files:
borg create --exclude 're:\.iso(?i)$' --exclude 'sh:home/**/.*' archive /home
# use a file with patterns (recursion root '/' via command line):
borg create --patterns-from patterns.lst archive /
The patterns.lst file could look like that:
# "sh:" pattern style is the default
# exclude caches
- home/*/.cache
# include susans home
+ home/susan
# also back up this exact file
+ pf:home/bobby/specialfile.txt
# don't back up the other home directories
- home/*
# don't even look in /dev, /proc, /run, /sys, /tmp (note: would exclude files like /device, too)
! re:^(dev|proc|run|sys|tmp)
You can specify recursion roots either on the command line or in a patternfile:
# these two commands do the same thing
borg create --exclude home/bobby/junk archive /home/bobby /home/susan
borg create --patterns-from patternfile.lst archive
patternfile.lst:
# note that excludes use fm: by default and patternfiles use sh: by default.
# therefore, we need to specify fm: to have the same exact behavior.
P fm
R /home/bobby
R /home/susan
- home/bobby/junk
This allows you to share the same patterns between multiple repositories without needing
to specify them on the command line.
AUTHOR
The Borg Collective
2023-03-01 BORG-PATTERNS(1)