This is the configuration file used by the proxy server and other
proxy middlewares.
The configuration file follows the python-pastedeploy syntax. The
file is divided into sections, which are enclosed by square brackets. Each
section will contain a certain number of key/value parameters which are
described later.
Any line that begins with a '#' symbol is ignored.
You can find more information about python-pastedeploy
configuration format at
https://docs.pylonsproject.org/projects/pastedeploy/en/latest/#config-format
Any section that has its name prefixed by
"filter:" indicates a filter section. Filters are used to specify
configuration parameters for specific swift middlewares. Below are the filters
available and respective acceptable parameters.
- [filter:healthcheck]
- use
- Entry point for paste.deploy for the healthcheck middleware. This is the
reference to the installed python egg. This is normally
egg:swift#healthcheck.
- disable_path
- An optional filesystem path which, if present, will cause the healthcheck
URL to return "503 Service Unavailable" with a body of
"DISABLED BY FILE".
- use
- Entry point for paste.deploy for the tempauth middleware. This is the
reference to the installed python egg. This is normally
egg:swift#tempauth.
- set log_name
- Label used when logging. The default is tempauth.
- set
log_facility
- Syslog log facility. The default is LOG_LOCAL0.
- set log_level
- Logging level. The default is INFO.
- set
log_address
- Logging address. The default is /dev/log.
- set log_headers
- Enables the ability to log request headers. The default is False.
- reseller_prefix
- The reseller prefix will verify a token begins with this prefix before
even attempting to validate it. Also, with authorization, only Swift
storage accounts with this prefix will be authorized by this middleware.
Useful if multiple auth systems are in use for one Swift cluster. The
default is AUTH.
- auth_prefix
- The auth prefix will cause requests beginning with this prefix to be
routed to the auth subsystem, for granting tokens, etc. The default is
/auth/.
- require_group
- The require_group parameter names a group that must be presented by either
X-Auth-Token or X-Service-Token. Usually this parameter is used only with
multiple reseller prefixes (e.g., SERVICE_require_group=blah). By default,
no group is needed. Do not use .admin.
- token_life
- This is the time in seconds before the token expires. The default is
86400.
- allow_overrides
- This allows middleware higher in the WSGI pipeline to override auth
processing, useful for middleware such as tempurl and formpost. If you
know you're not going to use such middleware and you want a bit of extra
security, you can set this to false. The default is true.
- storage_url_scheme
- This specifies what scheme to return with storage urls: http, https, or
default (chooses based on what the server is running as) This can be
useful with an SSL load balancer in front of a non-SSL server.
- user_<account>_<user>
- Lastly, you need to list all the accounts/users you want here. The format
is: user_<account>_<user> = <key> [group] [group] [...]
[storage_url] or if you want underscores in <account> or
<user>, you can base64 encode them (with no equal signs) and use
this format: user64_<account_b64>_<user_b64> = <key>
[group] [group] [...] [storage_url]
There are special groups of: .reseller_admin who can do
anything to any account for this auth and also .admin who can do
anything within the account.
If neither of these groups are specified, the user can only
access containers that have been explicitly allowed for them by a
.admin or .reseller_admin. The trailing optional
storage_url allows you to specify an alternate URL to hand back to the
user upon authentication. If not specified, this defaults to
http[s]://<ip>:<port>/v1/<reseller_prefix>_<account>
where http or https depends on whether cert_file is specified in the
[DEFAULT] section, <ip> and <port> are based on the
[DEFAULT] section's bind_ip and bind_port (falling back to 127.0.0.1 and
8080), <reseller_prefix> is from this section, and <account>
is from the user_<account>_<user> name.
Here are example entries, required for running the tests:
To enable Keystone authentication you need to have the auth token
middleware first to be configured. Here is an example below, please refer to
the keystone's documentation for details about the different settings.
You'll need to have as well the keystoneauth middleware enabled
and have it in your main pipeline so instead of having tempauth in there you
can change it to: authtoken keystoneauth
The auth credentials ("project_domain_name",
"user_domain_name", "username",
"project_name", "password") must match the Keystone
credentials for the Swift service. The example values shown here assume a
user named "swift" with admin role on a project named
"service", both being in the Keystone domain with id
"default". Refer to the KeystoneMiddleware documentation at
https://docs.openstack.org/keystonemiddleware/latest/middlewarearchitecture.html#configuration
for other examples.
Keystone authentication middleware.
- use
- Entry point for paste.deploy for the keystoneauth middleware. This is the
reference to the installed python egg. This is normally
egg:swift#keystoneauth.
- reseller_prefix
- The reseller_prefix option lists account namespaces that this middleware
is responsible for. The prefix is placed before the Keystone project id.
For example, for project 12345678, and prefix AUTH, the account is named
AUTH_12345678 (i.e., path is /v1/AUTH_12345678/...). Several prefixes are
allowed by specifying a comma-separated list as in: "reseller_prefix
= AUTH, SERVICE". The empty string indicates a single blank/empty
prefix. If an empty prefix is required in a list of prefixes, a value of
'' (two single quote characters) indicates a blank/empty prefix. Except
for the blank/empty prefix, an underscore ('_') character is appended to
the value unless already present.
- operator_roles
- The user must have at least one role named by operator_roles on a project
in order to create, delete and modify containers and objects and to set
and read privileged headers such as ACLs. If there are several reseller
prefix items, you can prefix the parameter so it applies only to those
accounts (for example the parameter SERVICE_operator_roles applies to the
/v1/SERVICE_<project> path). If you omit the prefix, the option
applies to all reseller prefix items. For the blank/empty prefix, prefix
with '' (do not put underscore after the two single quote
characters).
- reseller_admin_role
- The reseller admin role has the ability to create and delete
accounts.
- allow_overrides
- This allows middleware higher in the WSGI pipeline to override auth
processing, useful for middleware such as tempurl and formpost. If you
know you're not going to use such middleware and you want a bit of extra
security, you can set this to false.
- service_roles
- If the service_roles parameter is present, an X-Service-Token must be
present in the request that when validated, grants at least one role
listed in the parameter. The X-Service-Token may be scoped to any project.
If there are several reseller prefix items, you can prefix the parameter
so it applies only to those accounts (for example the parameter
SERVICE_service_roles applies to the /v1/SERVICE_<project> path). If
you omit the prefix, the option applies to all reseller prefix items. For
the blank/empty prefix, prefix with '' (do not put underscore after the
two single quote characters). By default, no service_roles are
required.
- default_domain_id
- For backwards compatibility, keystoneauth will match names in cross-tenant
access control lists (ACLs) when both the requesting user and the tenant
are in the default domain i.e the domain to which existing tenants are
migrated. The default_domain_id value configured here should be the same
as the value used during migration of tenants to keystone domains.
- allow_names_in_acls
- For a new installation, or an installation in which keystone projects may
move between domains, you should disable backwards compatible name
matching in ACLs by setting allow_names_in_acls to false:
Caching middleware that manages caching in swift.
- use
- Entry point for paste.deploy for the memcache middleware. This is the
reference to the installed python egg. This is normally
egg:swift#memcache.
- set log_name
- Label used when logging. The default is memcache.
- set
log_facility
- Syslog log facility. The default is LOG_LOCAL0.
- set log_level
- Logging level. The default is INFO.
- set
log_address
- Logging address. The default is /dev/log.
- set
log_headers
- Enables the ability to log request headers. The default is False.
- memcache_max_connections
- Sets the maximum number of connections to each memcached server per
worker.
- memcache_servers
- If not set in the configuration file, the value for memcache_servers will
be read from /etc/swift/memcache.conf (see memcache.conf-sample) or
lacking that file, it will default to 127.0.0.1:11211. You can specify
multiple servers separated with commas, as in:
10.1.2.3:11211,10.1.2.4:11211. (IPv6 addresses must follow rfc3986
section-3.2.2, i.e. [::1]:11211)
Rate limits requests on both an Account and Container level.
Limits are configurable.
- use
- Entry point for paste.deploy for the ratelimit middleware. This is the
reference to the installed python egg. This is normally
egg:swift#ratelimit.
- set
log_name
- Label used when logging. The default is ratelimit.
- set
log_facility
- Syslog log facility. The default is LOG_LOCAL0.
- set log_level
- Logging level. The default is INFO.
- set
log_address
- Logging address. The default is /dev/log.
- set log_headers
- Enables the ability to log request headers. The default is False.
- clock_accuracy
- This should represent how accurate the proxy servers' system clocks are
with each other. 1000 means that all the proxies' clock are accurate to
each other within 1 millisecond. No ratelimit should be higher than the
clock accuracy. The default is 1000.
- max_sleep_time_seconds
- App will immediately return a 498 response if the necessary sleep time
ever exceeds the given max_sleep_time_seconds. The default is 60
seconds.
- log_sleep_time_seconds
- To allow visibility into rate limiting set this value > 0 and all
sleeps greater than the number will be logged. If set to 0 means disabled.
The default is 0.
- rate_buffer_seconds
- Number of seconds the rate counter can drop and be allowed to catch up (at
a faster than listed rate). A larger number will result in larger spikes
in rate but better average accuracy. The default is 5.
- account_ratelimit
- If set, will limit PUT and DELETE requests to
/account_name/container_name. Number is in requests per second. If set to
0 means disabled. The default is 0.
- container_ratelimit_size
- When set with container_limit_x = r: for containers of size x, limit
requests per second to r. Will limit PUT, DELETE, and POST requests to
/a/c/o. The default is ''.
- container_listing_ratelimit_size
- Similarly to the above container-level write limits, the following will
limit container GET (listing) requests.
Middleware that translates container and account parts of a domain
to path parameters that the proxy server understands. The
container.account.storageurl/object gets translated to
container.account.storageurl/path_root/account/container/object and
account.storageurl/path_root/container/object gets translated to
account.storageurl/path_root/account/container/object
- use
- Entry point for paste.deploy for the domain_remap middleware. This is the
reference to the installed python egg. This is normally
egg:swift#domain_remap.
- set
log_name
- Label used when logging. The default is domain_remap.
- set
log_facility
- Syslog log facility. The default is LOG_LOCAL0.
- set log_level
- Logging level. The default is INFO.
- set
log_address
- Logging address. The default is /dev/log.
- set log_headers
- Enables the ability to log request headers. The default is False.
- storage_domain
- The domain to be used by the middleware. Multiple domains can be specified
separated by a comma.
- path_root
- The path root value for the storage URL. The default is v1.
- reseller_prefixes
- Browsers can convert a host header to lowercase, so check that reseller
prefix on the account is the correct case. This is done by comparing the
items in the reseller_prefixes config option to the found prefix. If they
match except for case, the item from reseller_prefixes will be used
instead of the found reseller prefix. When none match, the default
reseller prefix is used. When no default reseller prefix is configured,
any request with an account prefix not in that list will be ignored by
this middleware. Defaults to 'AUTH'.
- default_reseller_prefix
- The default reseller prefix. This is used when none of the configured
reseller_prefixes match. When not set, no reseller prefix is added.
- use
- Entry point for paste.deploy for the catch_errors middleware. This is the
reference to the installed python egg. This is normally
egg:swift#catch_errors.
- set
log_name
- Label used when logging. The default is catch_errors.
- set
log_facility
- Syslog log facility. The default is LOG_LOCAL0.
- set log_level
- Logging level. The default is INFO.
- set log_address
- Logging address. The default is /dev/log.
- set
log_headers
- Enables the ability to log request headers. The default is False.
Note: this middleware requires python-dnspython
- use
- Entry point for paste.deploy for the cname_lookup middleware. This is the
reference to the installed python egg. This is normally
egg:swift#cname_lookup.
- set
log_name
- Label used when logging. The default is cname_lookup.
- set
log_facility
- Syslog log facility. The default is LOG_LOCAL0.
- set log_level
- Logging level. The default is INFO.
- set
log_address
- Logging address. The default is /dev/log.
- set
log_headers
- Enables the ability to log request headers. The default is False.
- storage_domain
- The domain to be used by the middleware.
- lookup_depth
- How deep in the CNAME chain to look for something that matches the storage
domain. The default is 1.
- nameservers
- Specify the nameservers to use to do the CNAME resolution. If unset, the
system configuration is used. Multiple nameservers can be specified
separated by a comma. Default is unset.
Note: Put staticweb just after your auth filter(s) in the
pipeline
- use
- Entry point for paste.deploy for the staticweb middleware. This is the
reference to the installed python egg. This is normally
egg:swift#staticweb.
- set
log_name
- Label used when logging. The default is staticweb.
- set
log_facility
- Syslog log facility. The default is LOG_LOCAL0.
- set log_level
- Logging level. The default is INFO.
- set log_address
- Logging address. The default is /dev/log.
- set
log_headers
- Enables the ability to log request headers. The default is False.
Note: Put tempurl before slo, dlo, and your auth filter(s) in the
pipeline
- use
- Entry point for paste.deploy for the tempurl middleware. This is the
reference to the installed python egg. This is normally
egg:swift#tempurl.
- methods
- The methods allowed with Temp URLs. The default is 'GET HEAD PUT POST
DELETE'.
- The headers to remove from incoming requests. Simply a whitespace
delimited list of header names and names can optionally end with '*' to
indicate a prefix match. incoming_allow_headers is a list of exceptions to
these removals.
- The headers allowed as exceptions to incoming_remove_headers. Simply a
whitespace delimited list of header names and names can optionally end
with '*' to indicate a prefix match.
- The headers to remove from outgoing responses. Simply a whitespace
delimited list of header names and names can optionally end with '*' to
indicate a prefix match. outgoing_allow_headers is a list of exceptions to
these removals.
- The headers allowed as exceptions to outgoing_remove_headers. Simply a
whitespace delimited list of header names and names can optionally end
with '*' to indicate a prefix match.
Note: Put formpost just before your auth filter(s) in the
pipeline
- use
- Entry point for paste.deploy for the formpost middleware. This is the
reference to the installed python egg. This is normally
egg:swift#formpost.
Note: Just needs to be placed before the proxy-server in the
pipeline.
- use
- Entry point for paste.deploy for the name_check middleware. This is the
reference to the installed python egg. This is normally
egg:swift#name_check.
- forbidden_chars
- Characters that will not be allowed in a name. The default is
'"`<>.
- maximum_length
- Maximum number of characters that can be in the name. The default is
255.
- forbidden_regexp
- Python regular expressions of substrings that will not be allowed in a
name. The default is /./|/../|/.$|/..$.
- [filter:list-endpoints]
- use
- Entry point for paste.deploy for the list_endpoints middleware. This is
the reference to the installed python egg. This is normally
egg:swift#list_endpoints.
- list_endpoints_path
- The default is '/endpoints/'.
Logging for the proxy server now lives in this middleware. If the
access_* variables are not set, logging directives from [DEFAULT] without
"access_" will be used.
- use
- Entry point for paste.deploy for the proxy_logging middleware. This is the
reference to the installed python egg. This is normally
egg:swift#proxy_logging.
- access_log_name
- Label used when logging. The default is proxy-server.
- access_log_facility
- Syslog log facility. The default is LOG_LOCAL0.
- access_log_level
- Logging level. The default is INFO.
- access_log_address
- Default is /dev/log.
- access_log_udp_host
- If set, access_log_udp_host will override access_log_address. Default is
unset.
- access_log_udp_port
- Default is 514.
- access_log_statsd_host
- You can use log_statsd_* from [DEFAULT], or override them here. StatsD
server. IPv4/IPv6 addresses and hostnames are supported. If a hostname
resolves to an IPv4 and IPv6 address, the IPv4 address will be used.
- access_log_statsd_port
- Default is 8125.
- access_log_statsd_default_sample_rate
- Default is 1.
- access_log_statsd_sample_rate_factor
- The default is 1.
- access_log_statsd_metric_prefix
- Default is "" (empty-string)
- Default is False.
- If access_log_headers is True and access_log_headers_only is set only
these headers are logged. Multiple headers can be defined as comma
separated list like this: access_log_headers_only = Host,
X-Object-Meta-Mtime
- reveal_sensitive_prefix
- By default, the X-Auth-Token is logged. To obscure the value, set
reveal_sensitive_prefix to the number of characters to log. For example,
if set to 12, only the first 12 characters of the token appear in the log.
An unauthorized access of the log file won't allow unauthorized usage of
the token. However, the first 12 or so characters is unique enough that
you can trace/debug token usage. Set to 0 to suppress the token completely
(replaced by '...' in the log). The default is 16 chars. Note:
reveal_sensitive_prefix will not affect the value logged with
access_log_headers=True.
- log_statsd_valid_http_methods
- What HTTP methods are allowed for StatsD logging (comma-sep); request
methods not in this list will have "BAD_METHOD" for the
<verb> portion of the metric. Default is
"GET,HEAD,POST,PUT,DELETE,COPY,OPTIONS".
- log_anonymization_method
- Hashing algorithm for anonymization. Must be one of algorithms supported
by Python's hashlib. Default is MD5.
- log_anonymization_salt
- Salt added as prefix before hashing the value to anonymize. Default is
empty (no salt).
- log_msg_template
- Template used to format access logs. All words surrounded by curly
brackets will be substituted with the appropriate values.
- Some keywords map to
timestamps and can be converted to standard dates formats using the matching
transformers: 'datetime', 'asctime' or 'iso8601'.
- Other transformers for
timestamps are 's', 'ms', 'us' and 'ns' for seconds, milliseconds,
microseconds and nanoseconds.
- Python's strftime
directives can also be used as tranformers (a, A, b, B, c, d, H, I, j, m, M,
p, S, U, w, W, x, X, y, Y, Z).
- Some keywords map to user
data that could be anonymized by using the transformer
'anonymized'.
- Keywords availables
are:
- Example:
'{client_ip.anonymized} {remote_addr.anonymized} {start_time.iso8601}
{end_time.H}:{end_time.M} {method} acc:{account} cnt:{container}
obj:{object.anonymized}'
- Default: '{client_ip}
{remote_addr} {end_time.datetime} {method} {path} {protocol} {status_int}
{referer} {user_agent} {auth_token} {bytes_recvd} {bytes_sent} {client_etag}
{transaction_id} {headers} {request_time} {source} {log_info} {start_time}
{end_time} {policy_index}'
- Warning: A bad log
message template will raise an error in initialization.
Note: Put before both ratelimit and auth in the pipeline.
- use
- Entry point for paste.deploy for the bulk middleware. This is the
reference to the installed python egg. This is normally
egg:swift#bulk.
- The default is 10000.
- The default is 1000.
- max_deletes_per_request
- The default is 10000.
- max_failed_deletes
- The default is 1000.
In order to keep a connection active during a potentially long
bulk request, Swift may return whitespace prepended to the actual
response body. This whitespace will be yielded no more than every
yield_frequency seconds. The default is 10.
- yield_frequency
-
- delete_container_retry_count
- Note: This parameter is used during a bulk delete of objects and their
container. This would frequently fail because it is very likely that all
replicated objects have not been deleted by the time the middleware got a
successful response. It can be configured the number of retries. And the
number of seconds to wait between each retry will be 1.5**retry The
default is 0.
Note: Put after auth and staticweb in the pipeline.
- use
- Entry point for paste.deploy for the slo middleware. This is the reference
to the installed python egg. This is normally egg:swift#slo.
- max_manifest_segments
- The default is 1000.
- max_manifest_size
- The default is 2097152.
- min_segment_size
- The default is 1048576
- rate_limit_after_segment
- Start rate-limiting object segments after the Nth segment of a segmented
object. The default is 10 segments.
- rate_limit_segments_per_sec
- Once segment rate-limiting kicks in for an object, limit segments served
to N per second. The default is 1.
- max_get_time
- Time limit on GET requests (seconds). The default is 86400.
Note: Put after auth and staticweb in the pipeline. If you don't
put it in the pipeline, it will be inserted for you.
- use
- Entry point for paste.deploy for the dlo middleware. This is the reference
to the installed python egg. This is normally egg:swift#dlo.
- rate_limit_after_segment
- Start rate-limiting object segments after the Nth segment of a segmented
object. The default is 10 segments.
- rate_limit_segments_per_sec
- Once segment rate-limiting kicks in for an object, limit segments served
to N per second. The default is 1.
- max_get_time
- Time limit on GET requests (seconds). The default is 86400.
- [filter:container-quotas]
Note: Put after auth in the pipeline.
- use
- Entry point for paste.deploy for the container_quotas middleware. This is
the reference to the installed python egg. This is normally
egg:swift#container_quotas.
Note: Put after auth in the pipeline.
- use
- Entry point for paste.deploy for the account_quotas middleware. This is
the reference to the installed python egg. This is normally
egg:swift#account_quotas.
Note: this middleware requires python-dnspython
- use
- Entry point for paste.deploy for the gatekeeper middleware. This is the
reference to the installed python egg. This is normally
egg:swift#gatekeeper.
- set
log_name
- Label used when logging. The default is gatekeeper.
- set
log_facility
- Syslog log facility. The default is LOG_LOCAL0.
- set log_level
- Logging level. The default is INFO.
- set
log_address
- Logging address. The default is /dev/log.
- set
log_headers
- Enables the ability to log request headers. The default is False.
Note: this middleware requires python-dnspython
- use
- Entry point for paste.deploy for the container_sync middleware. This is
the reference to the installed python egg. This is normally
egg:swift#container_sync.
- allow_full_urls
- Set this to false if you want to disallow any full URL values to be set
for any new X-Container-Sync-To headers. This will keep any new full urls
from coming in, but won't change any existing values already in the
cluster. Updating those will have to be done manually, as knowing what the
true realm endpoint should be cannot always be guessed. The default is
true.
- current
- Set this to specify this clusters //realm/cluster as "current"
in /info
Note: Put it at the beginning of the pipeline to profile all
middleware. But it is safer to put this after healthcheck.
- use
- Entry point for paste.deploy for the xprofile middleware. This is the
reference to the installed python egg. This is normally
egg:swift#xprofile.
- profile_module
- This option enable you to switch profilers which should inherit from
python standard profiler. Currently the supported value can be 'cProfile',
'eventlet.green.profile' etc.
- log_filename_prefix
- This prefix will be used to combine process ID and timestamp to name the
profile data file. Make sure the executing user has permission to write
into this path (missing path segments will be created, if necessary). If
you enable profiling in more than one type of daemon, you must override it
with an unique value like, the default is
/var/log/swift/profile/account.profile.
- dump_interval
- The profile data will be dumped to local disk based on above naming rule
in this interval. The default is 5.0.
- dump_timestamp
- Be careful, this option will enable profiler to dump data into the file
with time stamp which means there will be lots of files piled up in the
directory. The default is false
- path
- This is the path of the URL to access the mini web UI. The default is
__profile__.
- flush_at_shutdown
- Clear the data when the wsgi server shutdown. The default is false.
- unwind
- Unwind the iterator of applications. Default is false.
- [filter:versioned_writes]
Note: Put after slo, dlo in the pipeline. If you don't put it in
the pipeline, it will be inserted automatically.
- use
- Entry point for paste.deploy for the versioned_writes middleware. This is
the reference to the installed python egg. This is normally
egg:swift#versioned_writes.
- allow_versioned_writes
- Enables using versioned writes middleware and exposing configuration
settings via HTTP GET /info. WARNING: Setting this option bypasses the
"allow_versions" option in the container configuration file,
which will be eventually deprecated. See documentation for more
details.