Secure Socket Layer (SSL)

SSL provides communication security between two hosts. It provides integrity, authentication and confidentiality. It is used most commonly in web browsers, but can be used with any protocol that uses TCP as the transport layer.

History

SSL was originally a Netscape project realized in association with MasterCard, Bank of America, MDI & Silicon Graphics. The first version, SSLv1, wasn't released. SSLv2 was replaced by SSLv3 in 1999 because of security problems. At this time, SSL became a standard so IETF bought a patent and created TLS in 2001 (standard actually used a derivation of SSLv3).

Protocol dependencies

Some well known TCP ports for SSL traffic are

Example traffic

Below is some excerpt from the snakeoil2 capture:

Secure Socket Layer
    SSLv2 Record Layer: Client Hello
        Length: 103
        Handshake Message Type: Client Hello (1)
        Version: SSL 3.0 (0x0300)
        Cipher Spec Length: 78
        Session ID Length: 0
        Challenge Length: 16
        Cipher Specs (26 specs)
            Cipher Spec: SSL2_RC4_128_WITH_MD5 (0x010080)
            [ more Cipher Specs deleted ]
        Challenge

Secure Socket Layer
    SSLv3 Record Layer: Handshake Protocol: Server Hello
        Content Type: Handshake (22)
        Version: SSL 3.0 (0x0300)
        Length: 74
        Handshake Protocol: Server Hello
            Handshake Type: Server Hello (2)
            Length: 70
            Version: SSL 3.0 (0x0300)
            Random
                gmt_unix_time: Apr 24, 2006 11:04:15.000000000
                random_bytes: FE81ED93650288A3F8EB63860E2CF68DD00F2C2AD64FCD2D...
            Session ID Length: 32
            Session ID (32 bytes)
            Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
            Compression Method: null (0)
    SSLv3 Record Layer: Handshake Protocol: Certificate
        Content Type: Handshake (22)
        Version: SSL 3.0 (0x0300)
        Length: 836
        Handshake Protocol: Certificate
            Handshake Type: Certificate (11)
            Length: 832
            [ Certificate details deleted ]
    SSLv3 Record Layer: Handshake Protocol: Server Hello Done
        Content Type: Handshake (22)
        Version: SSL 3.0 (0x0300)
        Length: 4
        Handshake Protocol: Server Hello Done
            Handshake Type: Server Hello Done (14)
            Length: 0

Secure Socket Layer
    SSLv3 Record Layer: Handshake Protocol: Client Key Exchange
        Content Type: Handshake (22)
        Version: SSL 3.0 (0x0300)
        Length: 132
        Handshake Protocol: Client Key Exchange
            Handshake Type: Client Key Exchange (16)
            Length: 128
    SSLv3 Record Layer: Change Cipher Spec Protocol: Change Cipher Spec
        Content Type: Change Cipher Spec (20)
        Version: SSL 3.0 (0x0300)
        Length: 1
        Change Cipher Spec Message
    SSLv3 Record Layer: Handshake Protocol: Finished
        Content Type: Handshake (22)
        Version: SSL 3.0 (0x0300)
        Length: 64
        Handshake Protocol: Finished
            Handshake Type: Finished (20)
            Length: 36
            MD5 Hash
            SHA-1 Hash

Secure Socket Layer
    SSLv3 Record Layer: Change Cipher Spec Protocol: Change Cipher Spec
        Content Type: Change Cipher Spec (20)
        Version: SSL 3.0 (0x0300)
        Length: 1
        Change Cipher Spec Message
    SSLv3 Record Layer: Handshake Protocol: Finished
        Content Type: Handshake (22)
        Version: SSL 3.0 (0x0300)
        Length: 64
        Handshake Protocol: Finished
            Handshake Type: Finished (20)
            Length: 36
            MD5 Hash
            SHA-1 Hash

Secure Socket Layer
    SSLv3 Record Layer: Application Data Protocol: http
        Content Type: Application Data (23)
        Version: SSL 3.0 (0x0300)
        Length: 432
        Encrypted Application Data: 4AC33E9D7778012CB4BC4C9A84D7B9900C2110F0FA007C16...
Hypertext Transfer Protocol
    GET / HTTP/1.1\r\n
        Request Method: GET
        Request URI: /
        Request Version: HTTP/1.1
    Host: localhost\r\n
    User-Agent: Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2\r\n
    Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5\r\n
    Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3\r\n
    Accept-Encoding: gzip,deflate\r\n
    Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n
    Keep-Alive: 300\r\n
    Connection: keep-alive\r\n
    \r\n

Wireshark

The SSL dissector is fully functional and even supports advanced features such as decryption of SSL if the encryption key can be provided and Wireshark is compiled against GnuTLS (rather than OpenSSL or bsafe). This works for RSA private keys.

Preference Settings

If Wireshark is compiled with SSL decryption support, there will be a new option in the preferences for SSL. If the key entry option is absent - then verify if your Wireshark is linked against the required GnuTLS library. This can be done with  wireshark -v . The output should include GnuTLS and GCrypt. If you see without GnuTLS, without Gcrypt, then you will need reconfigure with --with-gnutls, recompile and reinstall.

To configure a RSA private key, go to the SSL dissector preference in the Protocols tree. Then press the RSA keys list button. A new dialog windows appears showing you the currently configured RSA private keys. Press the New button to configure a new RSA private key. In the new window you have to configure the following fields:

IP address

The IP address of the SSL server in IPv4 or IPv6 format, or the following special values: any, anyipv4, anyipv6, 0.0.0.0

Port

The TCP port number, or the special value start_tls or 0.

Protocol

A protocol name for the decrypted network data. Popular choices are http or data. If you enter an invalid protocol name an error message will show you the valid values.

Key File

path to the RSA private key.

The RSA key file can either be a PEM format private key or a PKCS#12 keystore. If the file is a PKCS#12 keystore (typically a file with a .pfx or .p12 extension), the password for the keystore must be specified in the Password field.

Key File format conversion

The fileformat needed is 'PEM'. Note that it is common practice on webservers to combine the public key (or certificate) and the private key in a single PEM file.

In that case - locate this PEM file and cut and paste the section headed by 'PRIVATE KEY' (including header and footer) into a new 'file.key' file.

On Windows keys are often stored in PKCS7/DER format (locally) or in NET format (from any directory server). Use the following to convert:

# for PKCS7/DER keys (as held on disk)
openssl pkcs8 -nocrypt -in derfile.key -inform DER -out key.pem -outform PEM
# for NET keys (from the directory server)
openssl pkcs8 -nocrypt -in file.ick -inform NET -out key.pem -outform PEM

On OS X and Solaris, around Oracle and various other systems the file format used is often PKCS#12. Convert with:

openssl pkcs12 -nodes -in file.p12 -out key.pem -nocerts

And check that the file contains a 'PRIVATE KEY' header. I.e. it should look like this:

-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAtIvaDmeOGleYuxT01GfAmgugHVlqCOFfGYqy3gxMWt/fxO/7
s7BJzqnhAFOWBjmBAdj7hHmPyCoJM7/MdCDJt1y7d20BJAGxD0ZQ4kxzGZDCjc5z
....... some 20-100 lines of base64 encoded data ...............
Jh2kZkKoVG3Qr+66IlBDuVllIbwQU0F1fYy2FTjZL4vbmdupwHUyTnPK57vP8RJ7
cpc1qwLZxfurxZfhI9gxXOO5eUg1WBupw029SSoSafYBqO4a9wg1OA==
-----END RSA PRIVATE KEY-----

If your key file looks like this:

-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,CB7BE7B5A318ACE6

ScuaEtGA1xy7iVvvntc4hZ9Kl0VOKmA9sOcfP1CnrUVpAuLoHPEXTsc10smlXwsl
[...]
yy7ANfGCZTWaWP89uOIwlXK0n8hHZjTjw5axBuWXvgWHNbvein7tsg==
-----END RSA PRIVATE KEY-----

Then your keyfile is protected by a passphrase (which is a good thing). Unfortunately Wireshark can't use passphrase protected keys so you will need to use openssl (or something else) to create a keyfile that is not protected by a passphrase.

To do this with openssl, run this command and enter the keyfile's passphrase when prompted:

openssl rsa -in <old-keyfile> -out <new-keyfile>

On linux you occasionally may encounter a wrongly packaged DER or NET file with a certain commercial product; in which case you can use:

openssl x509 -nocrypt -in foo.der -informat DER -out key.pem -outformat PEM
openssl x509 -nocrypt -in foo.net -informat NET -out key.pem -outformat PEM

and them can manually edit the file to just leave the 'PRIVATE KEY' section.

start_tls

SSL may be introduced underneath a protocol in the course of a conversation through the use of a "start_tls" command. For example, an LDAP conversation may be proceeding on port 389 until the LDAP client issues a "start_tls" command - see RFC2830 - at which point the subsequent LDAP operations are protected by SSL.

If the key list is specified as:

127.0.01,389,ldap,c:\path\to\snakeoil2.key

then all the traffic on port 389 will be treated as SSL, including the LDAP traffic prior to the "start_tls" command.

In order to dissect both clear LDAP traffic and the SSL protected LDAP traffic (on the same port), use the string "start_tls" rather than the port number. For example:

127.0.0.1,start_tls,ldap,c:\path\to\snakeoil2.key

Example capture file

SampleCaptures/snakeoil2_070531.tgz Set RSA keys list to 127.0.0.1,443,http,/path/to/rsasnakeoil2.key to decrypt.

dump.pcapng TLSv1.2 capture with 73 cipher suites, you need this premaster.txt file for decrypting the traffic. (linked from https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=9144)

Display Filter

A complete list of SSL display filter fields can be found in the display filter reference

Show only the SSL based traffic:

Capture Filter

You cannot directly filter SSL protocols while capturing. However, if you know the TCP port used (see above), you can filter on that one.

Complete walk through

Ensure you have a version of Wireshark with GnuTLS support:

$ wireshark --version
wireshark 1.0.0

Copyright 1998-2008 Gerald Combs <gerald@wireshark.org> and contributors.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Compiled with GTK+ 2.12.9, with GLib 2.16.3, with libpcap 0.9.8, with libz
1.2.3, without POSIX capabilities, with libpcre 7.4, with SMI 0.4.7, with ADNS,
without Lua, with GnuTLS 2.2.2, with Gcrypt 1.4.0, with Heimdal Kerberos,
without PortAudio, without AirPcap.

Running on FreeBSD 7.0-RELEASE, with libpcap version 0.9.8.

Built using gcc 4.2.1 20070719  [FreeBSD].

Specifically check for the with GnuTLS 2.2.2 in the output.

Next create an RSA key and certificate pair with:

openssl req -new -x509 -out server.crt -nodes -keyout server.pem -subj /CN=localhost

Now run a server using above:

openssl s_server -www -cipher AES256-SHA -key server.pem -cert server.crt

and test that the server works by going to https://localhost:4433/ (use the flag -accept 443 to bind above to the normal https port).

Now start Wireshark - add above privkey.pem in the SSL preference pane:

http://people.apache.org/~dirkx/settings.png

This should result in a config snipped in the file ~/.wireshark/preferences

ssl.desegment_ssl_records: TRUE
ssl.desegment_ssl_application_data: TRUE
ssl.keys_list: 127.0.0.1,4443,http,/home/dirkx/xx/privkey.pem
ssl.debug_file: /home/dirkx/.wireshark-log

and configure the capturing:

http://people.apache.org/~dirkx/config.png

and then do a test request; for example with the command

printf 'GET / HTTP/1.0\r\n\r\n' | openssl s_client -ign_eof

Then stop your capture. The screen should look like attached:

http://people.apache.org/~dirkx/dump.png

And the TCP connection like

http://people.apache.org/~dirkx/tcp.png

and analyze the SSL shows you:

http://people.apache.org/~dirkx/ssl.png

Or if you want to observe authentication with a client cert; try the following:

# Generate self signed cert
openssl req -new -x509 -nodes -out client.crt -keyout client.key -subj /CN=Moi/O=Foo/C=NL

# Start a server
openssl s_server -cipher AES256-SHA -accept 4443 -www -CAfile client.crt -verify 1 -key server.pem -cert server.cert

# And test
printf 'GET / HTTP/1.0\r\n\r\n' | openssl s_client -connect localhost:4443 -ssl3 -cert client.pem -key client.key -ign_eof

# tshark commands
tshark -o "ssl.desegment_ssl_records: TRUE" -o "ssl.desegment_ssl_application_data: TRUE" -o "ssl.keys_list: 127.0.0.1,4443,http,/home/dirkx/xx/privkey.pem" -o "ssl.debug_file: /home/dirkx/.wireshark-log" -i eth0 -R "tcp.port == 4443"

The log should look like http://people.apache.org/~dirkx/wireshark.log. Or, a more realistic example with Firefox is at http://people.apache.org/~dirkx/wireshark-firefox.log (from 10.11.0.200->10.11.0.111, port 4433).

Using the (Pre)-Master-Secret

Decoding an SSL connection requires either knowledge of the (asymmetric) secret server key and a handshake that does not use DH or the (base of) the symmetric keys used to run the actual encryption. Support was added to Wireshark with SVN revision 37401 to do this, so it became available with Wireshark 1.6. For instructions look at this question on ask.wireshark.org

Since SVN revision 36876, it is also possible to decrypt traffic when you do not possess the server key but have access to the pre-master secret. For more details, see this security.stackexchange.com answer. That answer also contains some suggestions on finding out why SSL/TLS sessions do not get decrypted. In short, it should be possible to log the pre-master secret to a file with a current version of Firefox by setting an environment variable (SSLKEYLOGFILE=</path/to/private/directory/with/logfile>). Current versions of QT (both 4 and 5) allow to export the pre-master secret as well, but to the fixed path /tmp/qt-ssl-keys and they require a compile time option: Blog with details

Testing SSL / adding new cipher suites

The previously mentioned example capture file containing 73 cipher suites was generated using OpenSSL 1.0.1e. For an automated way to set up a server with RSA, DSA and EC certificates and testing them, follow instructions from Bug 9144: Update TLS ciphers. Basically:

  1. Start the capture, you can use wireshark directly or (as I prefer for automated tests), dumpcap.
  2. Use the Bash script openssl-connect to generate certificates and start three servers (RSA, DSA, EC).

  3. Use the Bash script openssl-listen to connect to the servers and extract (in a dumb way) the required Random to Pre-Master key mappings.

  4. Check the resulting capture using tshark or wireshark, providing the keylog file (and different ports for recognition of SSL/TLS traffic), filtering on SSL.
  5. If a cipher suite is not supported (or if keys are missing), then you will likely see encrypted "Application Data" instead of, say, HTTP. By parsing the tshark output, it is easy to see which cipher suites needs attention. In order to debug, add the -o ssl.debug_file:debug.txt file to write SSL debug lines to debug.txt.

$ dumpcap -i lo '(host ::1 or host 127.0.0.1) and tcp portrange 4430-4433' -w dump.pcapng
$ ./openssl-listen /tmp/test-certs/
$ ./openssl-connect > premaster.txt

(It is also possible to test a specific set of ciphers:)
$ openssl ciphers -V | grep CAMELLIA | ./openssl-connect > premaster.txt

(now dumpcap and openssl-listen can be interrupted)

(for a GUI:)
$ wireshark -n -r dump.pcapng -o http.ssl.port:443,4430-4433 -o ssl.keylog_file:premaster.txt -Y ssl

(for automated testing, a CLI:)
$ tshark -n -r dump.pcapng -o http.ssl.port:443,4430-4433 -o ssl.keylog_file:premaster.txt -Y ssl > tshark.txt\
-o gui.column.format:'"No.", "%m", "Cipher Suite", "%Cus:ssl.handshake.ciphersuite:0:R", "Info", "%i"'

(RC2 is expected not to be supported since libgcrypt does not support it)
$ awk '/Server Hello/{frame=$1;cipher=$2}/Application Data/{if(cipher)print cipher, frame;cipher=""}' tshark.txt
TLS_RSA_EXPORT_WITH_RC2_CBC_40_MD5 1071

If it turns out that a cipher is missing, you will find the following message in your debug log (the number refers to the cipher suite):

dissect_ssl3_hnd_srv_hello can't find cipher suite 0x88

Use IANS's TLS Cipher Suite Registry as a guide for adding other ciphers, your single cipher suite likely comes with a bunch of other related cipher suites. Then you can use the generate-wireshark-cs tool to generate code that can be used in epan/dissectors/packet-ssl-utils.c.

Discussion


SSL (last edited 2014-01-08 21:21:03 by GuyHarris)