Provided by: netcdf-bin_4.9.2-2ubuntu1_amd64 bug

NAME

       nccopy  -  Copy a netCDF file, optionally changing format, compression, or chunking in the
       output.

SYNOPSIS

       nccopy [-k  kind_name ] [-kind_code] [-d  n ] [-s] [-c   chunkspec  ]  [-u]  [-w]  [-[v|V]
              var1,...]   [-[g|G] grp1,...]  [-m  bufsize ] [-h  chunk_cache ] [-e  cache_elems ]
              [-r] [-F  filterspec ] [-L  n ] [-M  n ]  infile  outfile

DESCRIPTION

       The nccopy utility copies an input netCDF file in  any  supported  format  variant  to  an
       output  netCDF  file,  optionally  converting  the  output to any compatible netCDF format
       variant, compressing the data, or rechunking the data.  For example,  if  built  with  the
       netCDF-3  library,  a  netCDF  classic  file may be copied to a netCDF 64-bit offset file,
       permitting larger variables.  If built with the netCDF-4 library, a  netCDF  classic  file
       may  be  copied to a netCDF-4 file or to a netCDF-4 classic model file as well, permitting
       data compression, efficient schema changes,  larger  variable  sizes,  and  use  of  other
       netCDF-4 features.

       If  no output format is specified, with either -k kind_name or -kind_code, then the output
       will use the same format as the input, unless the input is classic or  64-bit  offset  and
       either  chunking  or  compression  is specified, in which case the output will be netCDF-4
       classic model format.  Attempting some kinds of format conversion will result in an error,
       if  the  conversion is not possible.  For example, an attempt to copy a netCDF-4 file that
       uses features of the enhanced model, such as groups or variable-length strings, to any  of
       the other kinds of netCDF formats that use the classic model will result in an error.

       nccopy  also  serves as an example of a generic netCDF-4 program, with its ability to read
       any valid netCDF file and handle nested groups, strings, and user-defined types, including
       arbitrarily  nested  compound types, variable-length types, and data of any valid netCDF-4
       type.

       If DAP support was enabled when nccopy was built, the file name may  specify  a  DAP  URL.
       This may be used to convert data on DAP servers to local netCDF files.

OPTIONS

        -k   kind_name
              Use  format  name  to specify the kind of file to be created and, by inference, the
              data  model  (i.e.  netcdf-3  (classic)  or  netcdf-4  (enhanced)).   The  possible
              arguments are:

                     'nc3' or 'classic' => netCDF classic format

                     'nc6' or '64-bit offset' => netCDF 64-bit format

                     'nc4' or 'netCDF-4' => netCDF-4 format (enhanced data model)

                     'nc7' or 'netCDF-4 classic model' => netCDF-4 classic model format

              Note:  The  old  format  numbers '1', '2', '3', '4', equivalent to the format names
              'nc3', 'nc6', 'nc4', or 'nc7' respectively, are also still accepted but deprecated,
              due to easy confusion between format numbers and format names.

       [-kind_code]
              Use  format numeric code (instead of format name) to specify the kind of file to be
              created and, by inference, the data model (i.e. netcdf-3 (classic) versus  netcdf-4
              (enhanced)).  The numeric codes are:

                     3 => netcdf classic format

                     6 => netCDF 64-bit format

                     4 => netCDF-4 format (enhanced data model)

                     7 => netCDF-4 classic model format
       The numeric code "7" is used because "7=3+4", specifying the format that uses the netCDF-3
       data model for compatibility with the netCDF-4 storage format for performance.  Credit  is
       due to NCO for use of these numeric codes instead of the old and confusing format numbers.

        -d   n
              For  netCDF-4  output,  including  netCDF-4  classic model, specify deflation level
              (level of compression) for variable data output.  0 corresponds to  no  compression
              and  9  to  maximum  compression,  with  higher  levels  of  compression  requiring
              marginally more time to compress or uncompress than lower levels. As a side  effect
              specifying  a  compression  level  of  0  (via "-d 0") actually turns off deflation
              altogether.  Compression achieved may also depend on  output  chunking  parameters.
              If  this  option  is  specified  for a classic format or 64-bit offset format input
              file, it is not necessary to also  specify  that  the  output  should  be  netCDF-4
              classic  model,  as  that will be the default.  If this option is not specified and
              the input file has compressed variables, the compression will still be preserved in
              the output, using the same chunking as in the input by default.

              Note that nccopy requires all variables to be compressed using the same compression
              level, but the API has no such restriction.   With  a  program  you  can  customize
              compression for each variable independently.

        -s    For  netCDF-4  output,  including  netCDF-4  classic  model,  specify  shuffling of
              variable data bytes before compression or after decompression.  Shuffling refers to
              interlacing  of  bytes  in  a  chunk  so  that  the  first  bytes of all values are
              contiguous in storage, followed by all the second bytes, and  so  on,  which  often
              improves  compression.  This option is ignored unless a non-zero deflation level is
              specified.  Using -d0  to  specify  no  deflation  on  input  data  that  has  been
              compressed and shuffled turns off both compression and shuffling in the output.

        -u    Convert  any unlimited size dimensions in the input to fixed size dimensions in the
              output.  This can speed up variable-at-a-time access, but  slow  down  record-at-a-
              time access to multiple variables along an unlimited dimension.

        -w    Keep  output in memory (as a diskless netCDF file) until output is closed, at which
              time output file is written to disk.  This can greatly speedup operations  such  as
              converting  unlimited dimension to fixed size (-u option), chunking, rechunking, or
              compressing the input.  It requires that available memory is large enough  to  hold
              the  output  file.  This option may provide a larger speedup than careful tuning of
              the -m, -h, or -e options, and it's certainly a lot simpler.

        -c  chunkspec
              For  netCDF-4  output,  including  netCDF-4   classic   model,   specify   chunking
              (multidimensional  tiling)  for  variable  data  in  the output.  This is useful to
              specify the units of disk access, compression, or other filters such as  checksums.
              Changing the chunking in a netCDF file can also greatly speedup access, by choosing
              chunk shapes that are appropriate for the most common access patterns.

              The chunkspec  argument  has  several  forms.  The  first  form  is  the  original,
              deprecated  form and is a string of comma-separated associations, each specifying a
              dimension name, a '/' character, and optionally the corresponding chunk length  for
              that  dimension.   No blanks should appear in the chunkspec string, except possibly
              escaped blanks that are part of a dimension name.  A chunkspec names at  least  one
              dimension,  and  may  omit  dimensions which are not to be chunked or for which the
              default chunk length is desired.   If  a  dimension  name  is  followed  by  a  '/'
              character  but  no subsequent chunk length, the actual dimension length is assumed.
              If copying a classic model file to a  netCDF-4  output  file  and  not  naming  all
              dimensions  in the chunkspec, unnamed dimensions will also use the actual dimension
              length for the chunk length.  An example of a chunkspec for variables that use  'm'
              and  'n' dimensions might be 'm/100,n/200' to specify 100 by 200 chunks. To see the
              chunking resulting from copying with a chunkspec, use the '-s' option of ncdump  on
              the output file.

              The  chunkspec  '/'  that omits all dimension names and corresponding chunk lengths
              specifies that no chunking is to occur in the output, so can be used to unchunk all
              the  chunked  variables.   To  see  the  chunking  resulting  from  copying  with a
              chunkspec, use the '-s' option of ncdump on the output file.

              As an I/O optimization, nccopy has a threshold for the minimum size  of  non-record
              variables  that  get  chunked,  currently  8192  bytes.  The -M flag can be used to
              override this value.

              Note that nccopy requires variables that share a dimension to also share the  chunk
              size  associated  with  that  dimension,  but the programming interface has no such
              restriction.  If you need to customize chunking for  variables  independently,  you
              will  need  to  use the second form of chunkspec. This second form of chunkspec has
              this syntax:  var:n1,n2,...,nn . This assumes that the  variable  named  "var"  has
              rank  n.  The chunking to be applied to each dimension of the variable is specified
              by the values of n1 through nn. This second form of chunking specification  can  be
              repeated  multiple times to specify the exact chunking for different variables.  If
              the variable is specified but no chunk sizes are specified (i.e.  -c  var:  )  then
              chunking  is  disabled  for  that variable.  If the same variable is specified more
              than once, the second and later specifications  are  ignored.   Also,  this  second
              form,  per-variable  chunking,  takes  precedence  over  any per-dimension chunking
              except the bare "/" case.

              The third form of the chunkspec has the syntax:   var:compact  or   var:contiguous.
              This explicitly attempts to set the variable storage type as compact or contiguous,
              respectively. These may be overridden if other flags require  the  variable  to  be
              chunked.

        -v   var1,...
              The output will include data values for the specified variables, in addition to the
              declarations of all dimensions, variables, and attributes. One  or  more  variables
              must  be  specified  by name in the comma-delimited list following this option. The
              list must be a single argument to  the  command,  hence  cannot  contain  unescaped
              blanks  or  other  white space characters. The named variables must be valid netCDF
              variables in the input-file. A variable within a group in a netCDF-4  file  may  be
              specified  with  an  absolute  path  name, such as "/GroupA/GroupA2/var".  Use of a
              relative path name such as 'var' or "grp/var" specifies all matching variable names
              in  the file.  The default, without this option, is to include data values for  all
              variables in the output.

        -V   var1,...
              The output will include the specified variables only but all dimensions and  global
              or  group attributes. One or more variables must be specified by name in the comma-
              delimited list following this option. The list must be a  single  argument  to  the
              command, hence cannot contain unescaped blanks or other white space characters. The
              named variables must be valid netCDF variables in the input-file. A variable within
              a  group  in  a  netCDF-4 file may be specified with an absolute path name, such as
              '/GroupA/GroupA2/var'.  Use of a relative path name  such  as  'var'  or  'grp/var'
              specifies  all  matching  variable  names  in  the file.  The default, without this
              option, is to include  all  variables in the output.

        -g   grp1,...
              The output will include data values only for the specified  groups.   One  or  more
              groups must be specified by name in the comma-delimited list following this option.
              The list must be a single argument to the command. The named groups must  be  valid
              netCDF  groups  in  the input-file. The default, without this option, is to include
              data values for all groups in the output.

        -G   grp1,...
              The output will include only the specified groups.  One  or  more  groups  must  be
              specified  by name in the comma-delimited list following this option. The list must
              be a single argument to the command. The named groups must be valid  netCDF  groups
              in  the  input-file.  The default, without this option, is to include all groups in
              the output.

        -m   bufsize
              An integer or floating-point number that specifies the size, in bytes, of the  copy
              buffer used to copy large variables.  A suffix of K, M, G, or T multiplies the copy
              buffer size by one thousand, million,  billion,  or  trillion,  respectively.   The
              default  is 5 Mbytes, but will be increased if necessary to hold at least one chunk
              of netCDF-4 chunked variables in the input file.  You may want to specify  a  value
              larger  than the default for copying large files over high latency networks.  Using
              the '-w' option may provide better performance, if the output fits in memory.

        -h   chunk_cache
              For netCDF-4 output, including netCDF-4 classic model, an integer or floating-point
              number  that  specifies the size in bytes of chunk cache allocated for each chunked
              variable.  This is not a property of the file,  but  merely  a  performance  tuning
              parameter  for  avoiding  compressing or decompressing the same data multiple times
              while copying and changing chunk shapes.  A suffix of K, M, G, or T multiplies  the
              chunk cache size by one thousand, million, billion, or trillion, respectively.  The
              default is 4.194304 Mbytes  (or  whatever  was  specified  for  the  configure-time
              constant  CHUNK_CACHE_SIZE when the netCDF library was built).  Ideally, the nccopy
              utility should accept only one memory buffer size and divide it optimally between a
              copy  buffer  and  chunk  cache, but no general algorithm for computing the optimum
              chunk cache size has been implemented yet. Using the '-w' option may provide better
              performance, if the output fits in memory.

        -e   cache_elems
              For  netCDF-4  output, including netCDF-4 classic model, specifies number of chunks
              that the chunk cache can hold. A suffix of K, M, G, or T multiplies the  number  of
              chunks  that  can  be  held  in  the  cache  by  one thousand, million, billion, or
              trillion, respectively.  This  is  not  a  property  of  the  file,  but  merely  a
              performance  tuning  parameter  for  avoiding compressing or decompressing the same
              data multiple times while copying and changing chunk shapes.  The default  is  1009
              (or  whatever was specified for the configure-time constant CHUNK_CACHE_NELEMS when
              the netCDF library was built).  Ideally, the nccopy  utility  should  determine  an
              optimum  value  for  this  parameter,  but  no  general algorithm for computing the
              optimum number of chunk cache elements has been implemented yet.

        -r    Read netCDF classic or 64-bit offset input file into  a  diskless  netCDF  file  in
              memory  before  copying.   Requires  that  input  file  be small enough to fit into
              memory.  For nccopy, this doesn't seem to provide any significant speedup,  so  may
              not be a useful option.

        -L  n Set the log level; only usable if nccopy supports netCDF-4 (enhanced).

        -M  n Set the minimum chunk size; only usable if nccopy supports netCDF-4 (enhanced).

        -F  filterspec
              For netCDF-4 output, including netCDF-4 classic model, specify a filter to apply to
              a specified  set  of  variables  in  the  output.  As  a  rule,  the  filter  is  a
              compression/decompression  algorithm  with  a unique numeric identifier assigned by
              the HDF Group (see https://support.hdfgroup.org/services/filters.html).

              The filterspec argument has this general form.
              fqn1|fqn2...,filterid,param1,param2...paramn or *,filterid,param1,param2...paramn
       An fqn (fully qualified name) is the name of a variable prefixed by its containing  groups
       with  the  group  names separated by forward slash ('/').  An example might be /g1/g2/var.
       Alternatively, just the variable name can be given if it is in the root group:  e.g.  var.
       Backslash  escapes  may be used as needed.  A note of warning: the '|' separator is a bash
       reserved character, so you will probably need to put the  filter  spec  in  some  kind  of
       quotes or otherwise escape it.

              The  filterid  is  an unsigned positive integer representing the id assigned by the
              HDFgroup to the filter. Following the id is a sequence of parameters  defining  the
              operation of the filter. Each parameter is a 32-bit unsigned integer.

              This parameter may be repeated multiple times with different variable names.

EXAMPLES

       Make  a  copy of foo1.nc, a netCDF file of any type, to foo2.nc, a netCDF file of the same
       type:

              nccopy foo1.nc foo2.nc

       Note that the above copy will not be as fast as use of cp or other  simple  copy  utility,
       because  the  file is copied using only the netCDF API.  If the input file has extra bytes
       after the end of the netCDF  data,  those  will  not  be  copied,  because  they  are  not
       accessible  through the netCDF interface.  If the original file was generated in "No fill"
       mode so that fill values are not stored for padding for data alignment,  the  output  file
       may have different padding bytes.

       Convert a netCDF-4 classic model file, compressed.nc, that uses compression, to a netCDF-3
       file classic.nc:

              nccopy -k classic compressed.nc classic.nc

       Note that 'nc3' could be used instead of 'classic'.

       Download the variable 'time_bnds' and its associated attributes from an OPeNDAP server and
       copy the result to a netCDF file named 'tb.nc':

              nccopy 'http://test.opendap.org/opendap/data/nc/sst.mnmean.nc.gz?time_bnds' tb.nc

       Note  that URLs that name specific variables as command-line arguments should generally be
       quoted, to avoid the shell interpreting special characters such as '?'.

       Compress all the variables in the input file foo.nc, a netCDF file of  any  type,  to  the
       output file bar.nc:

              nccopy -d1 foo.nc bar.nc

       If  foo.nc  was  a classic or 64-bit offset netCDF file, bar.nc will be a netCDF-4 classic
       model netCDF file, because the classic and 64-bit offset  format  variants  don't  support
       compression.   If  foo.nc was a netCDF-4 file with some variables compressed using various
       deflation levels, the output will also be a netCDF-4 file of the same type,  but  all  the
       variables, including any uncompressed variables in the input, will now use deflation level
       1.

       Assume the input data includes gridded variables that use time, lat, lon dimensions,  with
       1000  times  by 1000 latitudes by 1000 longitudes, and that the time dimension varies most
       slowly.  Also assume that users want quick access to data at all times for a small set  of
       lat-lon points.  Accessing data for 1000 times would typically require accessing 1000 disk
       blocks, which may be slow.

       Reorganizing the data into chunks on disk that have all the time in each chunk for  a  few
       lat  and  lon  coordinates  would  greatly speed up such access.  To chunk the data in the
       input file slow.nc, a netCDF file of any type, to the output file fast.nc, you could use;

              nccopy -c time/1000,lat/40,lon/40 slow.nc fast.nc

       to specify data chunks of 1000 times, 40 latitudes, and 40 longitudes.  If you had  enough
       memory  to  contain  the  output  file,  you  could  speed  up  the  rechunking  operation
       significantly by creating the output in memory before writing it to disk on  close  (using
       the -w flag):

              nccopy -w -c time/1000,lat/40,lon/40 slow.nc fast.nc
       Alternatively,  one  could  write  this  using  the  alternate, variable-specific chunking
       specification and assuming that times, lat, and lon are variables.

              nccopy -c time:1000 -c lat:40 -c lon:40 slow.nc fast.nc

Chunking Rules

       The complete set of chunking rules is captured here.  As  a  rough  summary,  these  rules
       preserve  all  chunking  properties  from  the input file. These rules apply only when the
       selected output format supports chunking, i.e. for the netcdf-4 variants.

       The variable specific chunking specification should be obvious and translates directly  to
       the corresponding "nc_def_var_chunking" API call.

       The original per-dimension, chunking specification requires some interpretation by nccopy.
       The following rules are applied in the given order independently for each variable  to  be
       copied from input to output. The rules are written assuming we are trying to determine the
       chunking for a given output variable Vout that comes from an input variable Vin.

       1.     If there is no '-c' option that applies to a variable and the  corresponding  input
              variable is contiguous or the input is some netcdf-3 variant, then let the netcdf-c
              library make all chunking decisions.

       2.     For each dimension of Vout explicitly specified on the command line (using the '-c'
              option),  apply the chunking value for that dimension regardless of input format or
              input properties.

       3.     For dimensions of Vout not named on the command line in  a  '-c'  option,  preserve
              chunk sizes from the corresponding input variable, if it is chunked.

       4.     If Vin is contiguous, and none of its dimensions are named on the command line, and
              chunking is not mandated by other options, then make Vout be contiguous.

       5.     If the input variable is contiguous (or is some netcdf-3 variant) and there are  no
              options  requiring  chunking,  or  the  '/'  special  case  for  the '-c' option is
              specified, then the output variable V is marked as contiguous.

       6.     Final, default case: some or all chunk sizes are not determined by the command line
              or  the input variable. This includes the non-chunked input cases such as netcdf-3,
              cdf5, and DAP. In these cases retain all chunk sizes determined by previous  rules,
              and  use  the  full  dimension  size  as  the  default.  The exception is unlimited
              dimensions, where the default is 4 megabytes.

SEE ALSO

       ncdump(1),ncgen(1),netcdf(3)