Provided by: gridsite-clients_2.2.6-1build1_amd64 bug

NAME

       htcp,  htmv,  htrm,  htls,  htll, htmkdir, htfind, htping - file transfers and queries via
       HTTP/HTTPS/SiteCast

SYNOPSIS

       htcp, htmv [options] Source-URL[s] Destination-URL

       htrm, htls, htll, htmkir, htfind [options] Target-URL[s]

       htping [options]

DESCRIPTION

       htcp is a client to fetch files or directory listings from remote servers  using  HTTP  or
       HTTPS,  or  to put or delete files or directories onto remote servers using HTTPS. htcp is
       similar to scp(1), but uses HTTP/HTTPS rather than ssh as its transfer protocol. htcp  can
       also use the HTCP protocol to query HTTP(S) fileservers via SiteCast.

       When talking to a fileserver with HTTPS, htcp can run "anonymously", with a standard X.509
       user certificate and key, or with a GSI  Proxy.  This  makes  htcp  very  useful  in  Grid
       environments  where  many  users have certificates and where jobs and users have access to
       GSI proxies.

URLs

       htcp supports the file:, http: and https: URL schemes as sources and destinations.  If  no
       scheme  is  given,  the  URL  scheme  is  assumed  to be file: and relative to the current
       directory if not an absolute path.

       If multiple sources are given during a copy, they will be used in turn and the destination
       must  be  a  directory  (directories  are  indicated  by a trailing /) However, source and
       destination cannot both refer to remote servers.

OPTIONS

       -v/--verbose
              Turn on debugging information. Used once, this option will enable  htcp's  messages
              to stderr. Used twice, will also enable the underlying libcurl messages.

       --delete
              Instead  of  copying files, delete all the URLs given on the command line.  Calling
              the program as htrm has the same effect.

       --list Instead of copying files, output lists of  files  located  in  the  URL-directories
              given on the command line. Calling the program as htls has the same effect.

       --long-list
              Instead  of  copying  files,  output  long  listings  of  files located in the URL-
              directories given on the  command  line.  If  available,  the  size  in  bytes  and
              modification  time  of each file is given. Calling the program as htll has the same
              effect.

       --mkdir
              Instead of copying files, attempt to create a directory on  a  remote  server  with
              HTTP  PUT. The server must support the convention that PUT to a URL with a trailing
              slash means create a directory. No file  body  is  sent.  Calling  the  program  as
              htmkdir has the same effect.

       --move Move/rename  files  on  a single remote server, given the two, absolute URLs of the
              remote file names. Server must support HTTP/WebDAV MOVE.  Calling  the  program  as
              htmv has the same effect.

       --ping Query specified multicast groups with the HTCP NOP ("No Operation") code.  SiteCast
              enabled servers will respond immediately with a NOP reply, and all of the responses
              will  be  listed,  with  the  round  trip  time in milliseconds.  Any waiting times
              specified in the --groups option will be ignored. Calling the program as htping has
              the same effect.  (--groups must be used for this option to work.)

       --find Query  specified  multicast groups with the HTCP TST code. SiteCast enabled servers
              will respond with TST replies if they have the files  corresponding  to  the  given
              SiteCast  target  URL(s). All of the transfer URLs returned will be listed. Waiting
              times specified in the --groups option will be used  to  space  out  the  multicast
              queries, but the program listens for responses continuously. Calling the program as
              htfind has the same effect.  (--groups must be used for this option to work.)

       --groups <IP Groups>
              IP multicast groups to use for SiteCast queries. IP Groups  is  a  comma  separated
              list  of  groups, in the format: nnn.nnn.nnn.nnn:port[:ttl[:seconds]] The IP number
              and port must be specified. The IP time-to-live, ttl, controls  how  many  networks
              the  multicast  packets  may  pass  through - the default, 1, limits packets to the
              local network. Multiple groups may be specified, separated by commas.  If  multiple
              groups  are  specified,  then  seconds  is  the time to wait before making the next
              multicast - 1 second is the default.

       --timeout <seconds>
              A request timeout used for multicast ping.

       --anon Do not attempt to use X.509 user certificates or GSI proxies to authenticate to the
              remote  HTTPS server. This means you are "anonymous", but the server's identity may
              still be verified and the connection is still encrypted.

       --cert <X.509 cert path>  and  --key <X.509 key path>
              Path to the PEM-encoded X.509 or GSI Proxy user certificate  and  key  to  use  for
              HTTPS  connections,  intead  of "anonymous mode." If only one of --key or --cert is
              given, then that will be tried for both. If neither is given,  then  the  following
              order  of  precedence  is used: the file name held by the variable X509_USER_PROXY;
              the file /tmp/x509up_uID (with Unix UID equal  to  ID);  the  file  names  held  by
              X509_USER_CERT    /    X509_USER_KEY;    the   files   ~/.globus/usercert.pem   and
              ~/.globus/userkey.pem (where ~/ is the home directory of the user.)

       --capath <X.509 CA root certs directory or file>
              Path to the PEM-encoded CA root certificates to use when verifying remote  servers'
              host  certificates  in  HTTPS  connections.  Ideally  this should be a directory of
              hash.0 files as described in the OpenSSL verify(1) man page, but a file may be used
              instead.  If  --capath  is  not  given,  the  value  of  the  environment  variable
              X509_CERT_DIR  will  be  tried.   If   this   is   not   valid,   then   /etc/grid-
              security/certificates will be used.

       --no-verify
              Do  not use CA root certificates to verify remote servers' host certificates.  This
              is useful for testing sites before their certificate is set up properly, but leaves
              you  vulnerable  to  "man in the middle" attacks by hostile servers masquerading as
              your target.

       --grid-http
              Try to use GridHTTP redirection for HTTPS URLs.  Compatible  servers  will  perform
              authentication  and authorization on the HTTPS connection and then redirect to HTTP
              for the  GET  or  PUT  file  transfer.  htcp  makes  the  HTTP  request  using  the
              GRID_AUTH_PASSCODE  single-use  passcode obtained via HTTPS. The --grid-http option
              will be ignored for directory operations or HTTP URLs.  If  a  redirected  transfer
              isn't possible, a normal HTTPS data transfer will be attempted.

       --sitecast
              Try  to  use SiteCast to locate remote files which are to be copied (currently only
              for the fetching of remote files.) If no location is found  via  SiteCast,  then  a
              direct  request  for the given URL is tried. (--groups must be used for this option
              to work.)

       --domain <SiteCast domain>
              Try to use SiteCast to locate remote files which are to be copied  (currently  only
              for  the  fetching  of remote files) if the domain component of the URL matches the
              SiteCast domain given.  If no location is found via SiteCast, then a direct request
              for the given URL is tried. (--groups must be used for this option to work.)

FILES

       /tmp/x509up_uID
              Default GSI Proxy file for Unix UID equal to ID.

       /etc/grid-security/certificates
              Default  location for trusted Certification Authority root certificates to use when
              checking server certificates.

       /tmp/.ca-roots-XXXXXX
              Prior  to  7.9.8,  the  underlying  curl  library  did  not  support  the  CA  root
              certificates  directory.   If  built  with  an  old  version  of libcurl, htcp will
              concatenate the certificates in the CA roots directory into a unique temporary file
              and use that.

ENVIRONMENT

       X509_CERT_DIR
              Holds  directory  to  search  for  Certification  Authority  root certificates when
              verifying server certificates. (Tried if --capath  is  not  given  on  the  command
              line.)

       X509_USER_PROXY
              Holds  file  name  of  a  GSI Proxy to use as user certificate. (Tried if --cert or
              --key are not given on the command line.)

       X509_USER_CERT and X509_USER_KEY
              Holds file name of X.509 user certificate and key. (Tried if X509_USER_PROXY is not
              valid.)

EXIT CODES

       0  is  returned  on  complete  success. Curl error codes are returned when reported by the
       underlying curl library, and CURLE_HTTP_RETURNED_ERROR (22) is returned when  the  HTTP(S)
       server  returns a code outside the range 200-299.  The manpage libcurl-errors(3) lists all
       the curl error codes.

TO DO

       Recursive copying. Server-side wildcards. Parallel streams. Better error recovery.

AUTHOR

       Andrew McNab <Andrew.McNab@manchester.ac.uk>

       htcp is part of GridSite: http://www.gridsite.org/

SEE ALSO

       scp(1), curl(1), wget(1), verify(1), libcurl-errors(3)