NAME

       ossl-guide-tls-introduction - OpenSSL Guide: An introduction to SSL/TLS in OpenSSL

INTRODUCTION

       This page will provide an introduction to some basic SSL/TLS concepts and background and
       how it is used within OpenSSL. It assumes that you have a basic understanding of TCP/IP
       and sockets.

WHAT IS TLS?

       TLS stands for Transport Layer Security. TLS allows applications to securely communicate
       with each other across a network such that the confidentiality of the information
       exchanged is protected (i.e. it prevents eavesdroppers from listening in to the
       communication). Additionally it protects the integrity of the information exchanged to
       prevent an attacker from changing it. Finally it provides authentication so that one or
       both parties can be sure that they are talking to who they think they are talking to and
       not some imposter.

       Sometimes TLS is referred to by its predecessor's name SSL (Secure Sockets Layer). OpenSSL
       dates from a time when the SSL name was still in common use and hence many of the
       functions and names used by OpenSSL contain the "SSL" abbreviation. Nonetheless OpenSSL
       contains a fully fledged TLS implementation.

       TLS is based on a client/server model. The application that initiates a communication is
       known as the client. The application that responds to a remotely initiated communication
       is the server. The term "endpoint" refers to either of the client or the server in a
       communication. The term "peer" refers to the endpoint at the other side of the
       communication that we are currently referring to. So if we are currently talking about the
       client then the peer would be the server.

       TLS is a standardised protocol and there are numerous different implementations of it. Due
       to the standards an OpenSSL client or server is able to communicate seamlessly with an
       application using some different implementation of TLS. TLS (and its predecessor SSL) have
       been around for a significant period of time and the protocol has undergone various
       changes over the years. Consequently there are different versions of the protocol
       available. TLS includes the ability to perform version negotiation so that the highest
       protocol version that the client and server share in common is used.

       TLS acts as a security layer over some lower level transport protocol. Typically the
       transport layer will be TCP.

SSL AND TLS VERSIONS

       SSL was initially developed by Netscape Communications and its first publicly released
       version was SSLv2 in 1995. Note that SSLv1 was never publicly released.  SSLv3 came along
       quickly afterwards in 1996. Subsequently development of the protocol moved to the IETF
       which released the first version of TLS (TLSv1.0) in 1999 as RFC2246. TLSv1.1 was released
       in 2006 as RFC4346 and TLSv1.2 came along in 2008 as RFC5246. The most recent version of
       the standard is TLSv1.3 which was released in 2018 as RFC8446.

       Today TLSv1.3 and TLSv1.2 are the most commonly deployed versions of the protocol. The
       IETF have formally deprecated TLSv1.1 and TLSv1.0, so anything below TLSv1.2 should be
       avoided since the older protocol versions are susceptible to security problems.

       OpenSSL does not support SSLv2 (it was removed in OpenSSL 1.1.0). Support for SSLv3 is
       available as a compile time option - but it is not built by default.  Support for TLSv1.0,
       TLSv1.1, TLSv1.2 and TLSv1.3 are all available by default in a standard build of OpenSSL.
       However special run-time configuration is required in order to make TLSv1.0 and TLSv1.1
       work successfully.

       OpenSSL will always try to negotiate the highest protocol version that it has been
       configured to support. In most cases this will mean either TLSv1.3 or TLSv1.2 is chosen.

CERTIFICATES

       In order for a client to establish a connection to a server it must authenticate the
       identify of that server, i.e. it needs to confirm that the server is really the server
       that it claims to be and not some imposter. In order to do this the server will send to
       the client a digital certificate (also commonly referred to as an X.509 certificate). The
       certificate contains various information about the server including its full DNS hostname.
       Also within the certificate is the server's public key. The server operator will have a
       private key which is linked to the public key and must not be published.

       Along with the certificate the server will also send to the client proof that it knows the
       private key associated with the public key in the certificate. It does this by digitally
       signing a message to the client using that private key. The client can verify the
       signature using the public key from the certificate. If the signature verifies
       successfully then the client knows that the server is in possession of the correct private
       key.

       The certificate that the server sends will also be signed by a Certificate Authority. The
       Certificate Authority (commonly known as a CA) is a third party organisation that is
       responsible for verifying the information in the server's certificate (including its DNS
       hostname). The CA should only sign the certificate if it has been able to confirm that the
       server operator does indeed have control of the server associated with its DNS hostname
       and that the server operator has control of the private key.

       In this way, if the client trusts the CA that has signed the server's certificate and it
       can verify that the server has the right private key then it can trust that the server
       truly does represent the DNS hostname given in the certificate. The client must also
       verify that the hostname given in the certificate matches the hostname that it originally
       sent the request to.

       Once all of these checks have been done the client has successfully verified the identify
       of the server. OpenSSL can perform all of these checks automatically but it must be
       provided with certain information in order to do so, i.e. the set of CAs that the client
       trusts as well as the DNS hostname for the server that this client is trying to connect
       to.

       Note that it is common for certificates to be built up into a chain. For example a
       server's certificate may be signed by a key owned by a an intermediate CA.  That
       intermediate CA also has a certificate containing its public key which is in turn signed
       by a key owned by a root CA. The client may only trust the root CA, but if the server
       sends both its own certificate and the certificate for the intermediate CA then the client
       can still successfully verify the identity of the server. There is a chain of trust
       between the root CA and the server.

       By default it is only the client that authenticates the server using this method. However
       it is also possible to set things up such that the server additionally authenticates the
       client. This is known as "client authentication".  In this approach the client will still
       authenticate the server in the same way, but the server will request a certificate from
       the client. The client sends the server its certificate and the server authenticates it in
       the same way that the client does.

TRUSTED CERTIFICATE STORE

       The system described above only works if a chain of trust can be built between the set of
       CAs that the endpoint trusts and the certificate that the peer is using. The endpoint must
       therefore have a set of certificates for CAs that it trusts before any communication can
       take place. OpenSSL itself does not provide such a set of certificates. Therefore you will
       need to make sure you have them before you start if you are going to be verifying
       certificates (i.e. always if the endpoint is a client, and only if client authentication
       is in use for a server).

       Fortunately other organisations do maintain such a set of certificates. If you have
       obtained your copy of OpenSSL from an Operating System (OS) vendor (e.g. a Linux
       distribution) then normally the set of CA certificates will also be distributed with that
       copy.

       You can check this by running the OpenSSL command line application like this:

        openssl version -d

       This will display a value for OPENSSLDIR. Look in the certs sub directory of OPENSSLDIR
       and check its contents. For example if OPENSSLDIR is "/usr/local/ssl", then check the
       contents of the "/usr/local/ssl/certs" directory.

       You are expecting to see a list of files, typically with the suffix ".pem" or ".0". If
       they exist then you already have a suitable trusted certificate store.

       If you are running your version of OpenSSL on Windows then OpenSSL (from version 3.2
       onwards) will use the default Windows set of trusted CAs.

       If you have built your version of OpenSSL from source, or obtained it from some other
       location and it does not have a set of trusted CA certificates then you will have to
       obtain them yourself. One such source is the Curl project. See the page
       <https://curl.se/docs/caextract.html> where you can download trusted certificates in a
       single file. Rename the file to "cert.pem" and store it directly in OPENSSLDIR. For
       example if OPENSSLDIR is "/usr/local/ssl", then save it as "/usr/local/ssl/cert.pem".

       You can also use environment variables to override the default location that OpenSSL will
       look for its trusted certificate store. Set the SSL_CERT_PATH environment variable to give
       the directory where OpenSSL should looks for its certificates or the SSL_CERT_FILE
       environment variable to give the name of a single file containing all of the certificates.
       See openssl-env(7) for further details about OpenSSL environment variables. For example
       you could use this capability to have multiple versions of OpenSSL all installed on the
       same system using different values for OPENSSLDIR but all using the same trusted
       certificate store.

       You can test that your trusted certificate store is setup correctly by using it via the
       OpenSSL command line. Use the following command to connect to a TLS server:

        openssl s_client www.openssl.org:443

       Once the command has connected type the letter "Q" followed by "<enter>" to exit the
       session. This will print a lot of information on the screen about the connection. Look for
       a block of text like this:

        SSL handshake has read 4584 bytes and written 403 bytes
        Verification: OK

       Hopefully if everything has worked then the "Verification" line will say "OK".  If its not
       working as expected then you might see output like this instead:

        SSL handshake has read 4584 bytes and written 403 bytes
        Verification error: unable to get local issuer certificate

       The "unable to get local issuer certificate" error means that OpenSSL has been unable to
       find a trusted CA for the chain of certificates provided by the server in its trusted
       certificate store. Check your trusted certificate store configuration again.

       Note that s_client is a testing tool and will still allow you to connect to the TLS server
       regardless of the verification error. Most applications should not do this and should
       abort the connection in the event of a verification error.

IMPORTANT OBJECTS FOR AN OPENSSL TLS APPLICATION

       A TLS connection is represented by the SSL object in an OpenSSL based application. Once a
       connection with a remote peer has been established an endpoint can "write" data to the SSL
       object to send data to the peer, or "read" data from it to receive data from the server.

       A new SSL object is created from an SSL_CTX object. Think of an SSL_CTX as a "factory" for
       creating SSL objects. You can create a single SSL_CTX object and then create multiple
       connections (i.e. SSL objects) from it.  Typically you can set up common configuration
       options on the SSL_CTX so that all the SSL object created from it inherit the same
       configuration options.

       Note that internally to OpenSSL various items that are shared between multiple SSL objects
       are cached in the SSL_CTX for performance reasons. Therefore it is considered best
       practice to create one SSL_CTX for use by multiple SSL objects instead of having one
       SSL_CTX for each SSL object that you create.

       Each SSL object is also associated with two BIO objects. A BIO object is used for sending
       or receiving data from the underlying transport layer. For example you might create a BIO
       to represent a TCP socket. The SSL object uses one BIO for reading data and one BIO for
       writing data. In most cases you would use the same BIO for each direction but there could
       be some circumstances where you want them to be different.

       It is up to the application programmer to create the BIO objects that are needed and
       supply them to the SSL object. See ossl-guide-tls-client-block(7) for further information.

       Finally, an endpoint can establish a "session" with its peer. The session holds various
       TLS parameters about the connection between the client and the server.  The session
       details can then be reused in a subsequent connection attempt to speed up the process of
       connecting. This is known as "resumption". Sessions are represented in OpenSSL by the
       SSL_SESSION object. In TLSv1.2 there is always exactly one session per connection. In
       TLSv1.3 there can be any number per connection including none.

PHASES OF A TLS CONNECTION

       A TLS connection starts with an initial "set up" phase. The endpoint creates the SSL_CTX
       (if one has not already been created) and configures it.

       A client then creates an SSL object to represent the new TLS connection. Any connection
       specific configuration parameters are then applied and the underlying socket is created
       and associated with the SSL via BIO objects.

       A server will create a socket for listening for incoming connection attempts from clients.
       Once a connection attempt is made the server will create an SSL object in the same way as
       for a client and associate it with a BIO for the newly created incoming socket.

       After set up is complete the TLS "handshake" phase begins. A TLS handshake consists of the
       client and server exchanging a series of TLS handshake messages to establish the
       connection. The client starts by sending a "ClientHello" handshake message and the server
       responds with a "ServerHello". The handshake is complete once an endpoint has sent its
       last message (known as the "Finished" message) and received a Finished message from its
       peer. Note that this might occur at slightly different times for each peer. For example in
       TLSv1.3 the server always sends its Finished message before the client. The client later
       responds with its Finished message. At this point the client has completed the handshake
       because it has both sent and received a Finished message. The server has sent its Finished
       message but the Finished message from the client may still be in-flight, so the server is
       still in the handshake phase. It is even possible that the server will fail to complete
       the handshake (if it considers there is some problem with the messages sent from the
       client), even though the client may have already progressed to sending application data.
       In TLSv1.2 this can happen the other way around, i.e. the server finishes first and the
       client finishes second.

       Once the handshake is complete the application data transfer phase begins.  Strictly
       speaking there are some situations where the client can start sending application data
       even earlier (using the TLSv1.3 "early data" capability) - but we're going to skip over
       that for this basic introduction.

       During application data transfer the client and server can read and write data to the
       connection freely. The details of this are typically left to some higher level application
       protocol (for example HTTP). Not all information exchanged during this phase is
       application data. Some protocol level messages may still be exchanged - so it is not
       necessarily the case that, just because the underlying socket is "readable", that
       application data will be available to read.

       When the connection is no longer required then it should be shutdown. A shutdown may be
       initiated by either the client or the server via a message known as a "close_notify"
       alert. The client or server that receives a close_notify may respond with one and then the
       connection is fully closed and application data can no longer be sent or received.

       Once shutdown is complete a TLS application must clean up by freeing the SSL object.

FURTHER READING

       See ossl-guide-tls-client-block(7) to see an example of applying these concepts in order
       to write a simple TLS client based on a blocking socket.  See
       ossl-guide-quic-introduction(7) for an introduction to QUIC in OpenSSL.

SEE ALSO

       ossl-guide-introduction(7), ossl-guide-libraries-introduction(7),
       ossl-guide-libssl-introduction(7), ossl-guide-tls-client-block(7),
       ossl-guide-quic-introduction(7)

COPYRIGHT

       Copyright 2023 The OpenSSL Project Authors. All Rights Reserved.

       Licensed under the Apache License 2.0 (the "License").  You may not use this file except
       in compliance with the License.  You can obtain a copy in the file LICENSE in the source
       distribution or at <https://www.openssl.org/source/license.html>.