Differences between revisions 6 and 7
Revision 6 as of 2019-06-13 19:01:25
Size: 10612
Editor: PeterWu
Comment: Rewrite article, removing outdated information, making it more user-centric. Co-written with Graham Bloice
Revision 7 as of 2019-06-13 23:08:25
Size: 12019
Editor: PeterWu
Comment: mention DSB
Deletions are marked like this. Additions are marked like this.
Line 34: Line 34:
 * The TLS version is SSLv3, TLS 1.0, 1.1 or 1.2. It does ''not'' work with TLS 1.3.  * The protocol version is SSLv3, (D)TLS 1.0-1.2. It does ''not'' work with TLS 1.3.
Line 60: Line 60:
{{{  {{{
Line 104: Line 104:
{{{  {{{
Line 117: Line 117:
For a survey of supported TLS applications and libraries, see also [[https://lekensteyn.nl/files/wireshark-ssl-tls-decryption-secrets-sharkfest18eu.pdf#page=19|page 19 of Peter Wu's SSL/TLS Decryption SharkFest '18 presentation]]. For a survey of supported TLS applications and libraries, see also [[https://lekensteyn.nl/files/wireshark-ssl-tls-decryption-secrets-sharkfest18eu.pdf#page=19|page 19 of Peter Wu's SSL/TLS Decryption SharkFest'18 EU presentation]].

=== Embedding decryption secrets in a pcapng file ===
Since Wireshark 3.0 you can embed the TLS key log file in a '''pcapng''' file. This makes it much easier to distribute capture files with decryption secrets, and makes switching between capture files easier since the TLS protocol preference does not have to be updated. To add the contents of key log file `keys.txt` to capture file `in.pcap` and write the result to `out-dsb.pcapng`:

 {{{editcap --inject-secrets tls,keys.txt in.pcap out-dsb.pcapng}}}

The ''dsb'' suffix stands for Decryption Secrets Block (DSB) and is part of the [[https://github.com/pcapng/pcapng|pcapng specification]].

A key log file might contain keys that are not related to a capture file. To ensure that unnecessary keys are not leaked, you can use the `inject-tls-secrets.sh` script from https://gist.github.com/Lekensteyn/f64ba6d6d2c6229d6ec444647979ea24 to filter the key log file and add the required secrets to a capture file. This has been tested with Linux and macOS, but not Windows. Example:

 {{{
git clone https://gist.github.com/Lekensteyn/f64ba6d6d2c6229d6ec444647979ea24 ~/its
~/its/inject-tls-secrets.sh keys.txt some.pcap
}}}
Line 128: Line 142:
 * https://lekensteyn.nl/files/wireshark-tls-debugging-sharkfest19us.pdf SharkFest'19 US presentation by Peter Wu describing TLS decryption and use of embedded decryption secrets (https://youtu.be/Ha4SLHceF6w).

Transport Layer Security (TLS)

Transport Layer Security (TLS) provides security in the communication between two hosts. It provides integrity, authentication and confidentiality. It is used most commonly in web browsers, but can be used with any protocol that uses TCP as the transport layer.

Secure Sockets Layer (SSL) is the predecessor of the TLS protocol. These names are often used interchangeably which can lead to some confusion:

  • A configuration that uses the SSL protocol (SSLv2/SSLv3) is insecure. The TLS protocol should be used instead.

  • X.509 certificates for authentication are sometimes also called SSL Certificates.

  • Some applications (such as email) use a single port for both unencrypted and encrypted sessions. To change from unencrypted to encrypted, (START)TLS is used. When a single port directly uses the TLS protocol, it is often referred to as SSL.

  • For historical reasons, software (Wireshark included) refer to SSL or SSL/TLS while it actually means the TLS protocol since that is nowadays what everyone uses.

Protocol dependencies

  • TCP: Typically, TLS uses TCP as its transport protocol.

TLS dissection in Wireshark

The TLS dissector is fully functional and even supports advanced features such as decryption of TLS if appropriate secrets are provided (#TLS_Decryption).

Since Wireshark 3.0, the TLS dissector has been renamed from SSL to TLS. Use of the ssl display filter will emit a warning.

TLS Decryption

Wireshark supports TLS decryption when appropriate secrets are provided. The two available methods are:

A key log file is a universal mechanism that always enables decryption, even if a Diffie-Hellman (DH) key exchange is in use. The RSA private key only works in a limited number of cases.

The key log file is a text file generated by applications such as Firefox, Chrome and curl when the SSLKEYLOGFILE environment variable is set. To be precise, their underlying library (NSS, OpenSSL or boringssl) writes the required per-session secrets to a file. This file can subsequently be configured in Wireshark (#Using_the_.28Pre.29-Master-Secret).

The RSA private key file can only be used in the following circumstances:

  • The cipher suite selected by the server is not using (EC)DHE.
  • The protocol version is SSLv3, (D)TLS 1.0-1.2. It does not work with TLS 1.3.

  • The private key matches the server certificate. It does not work with the client certificate, nor the Certificate Authority (CA) certificate.

  • The session has not been resumed. The handshake must include the ClientKeyExchange handshake message.

The key log file is generally recommended since it works in all cases, but requires the continuous ability to export the secrets from either the client or server application. The only advantage of the RSA private key is that it needs to be configured only once in Wireshark to enable decryption, subject to the above limitations.

Preference Settings

Go to Edit -> Preferences. Open the Protocols tree and select TLS. Alternatively, select a TLS packet in the packet list, right-click on the TLS layer in the packet details view and open the Protocol preferences menu.

The notable TLS protocol preferences are:

  • (Pre)-Master-Secret log filename (tls.keylog_file): path to read the TLS key log file for decryption.
  • RSA keys list: opens a dialog to configure RSA private keys for decryption. Deprecated in favor of the Preferences -> RSA Keys dialog.

  • Pre-Shared-Key: used to configure the decryption key for PSK cipher suites. Not generally used.
  • TLS debug file (tls.debug_logfile): path to write internal details about the decryption process. Will contain the results of decryption and the keys that were used in this process. This can be used to diagnose why decryption fails.

The following TCP protocol preferences are also required to enable TLS decryption:

  • Allow subdissector to reassemble TCP streams. Enabled by default.
  • Reassemble out-of-order segments (since Wireshark 3.0, disabled by default).

Starting with Wireshark 3.0, a new RSA Keys dialog can be found at Edit -> Preferences -> RSA Keys. In this dialog, use the Add new keyfile... button to select a file. You will be prompted for a password if necessary. The Add new token... button can be used to add keys from a HSM which might require using Add new provider... to select select a DLL/.so file, and additional vendor-specific configuration.

The RSA key file can either be a PEM format private key or a PKCS#12 keystore (typically a file with a .pfx or .p12 extension). The PKCS#12 key is a binary file, but the PEM format is a text file which looks like this:

  • -----BEGIN PRIVATE KEY-----
    MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDReQzlKVeAK8b5
    TRcRBhSi9IYwHX8Nqc8K4HeDRvN7HiBQQP3bhUkVekdoXpRLYVuc7A8h1BLr93Qw
    ...
    KOi8FZl+jhG+p8vtpK5ZAIyp
    -----END PRIVATE KEY-----

The deprecated RSA keys list dialog may be removed at some point. To configure keys, use the RSA keys dialog instead. To change the protocol for decrypted network data, right-click on a TLS packet and use Decode As to change the Current protocol for the TLS port. The IP address and Port fields are unused.

Example capture file

See also SampleCaptures#SSL_with_decryption_keys.

Display Filter

A complete list of TLS display filter fields can be found in the display filter reference

Show only the TLS based traffic:

  • tls

Capture Filter

You cannot directly filter TLS protocols while capturing. However, if you know the TCP port used (see above), you can filter on that one, for example using tcp port 443.

Using the (Pre)-Master-Secret

The master secret enables TLS decryption in Wireshark and can be supplied via the Key Log File. The pre-master secret is the result from the key exchange and can be converted to a master secret by Wireshark. This pre-master secret can be obtained when a RSA private key is provided and a RSA key exchange is in use.

Step-by-step instructions to decrypt TLS traffic from Chrome or Firefox in Wireshark:

  1. Close the browser completely (check your task manager just to be sure).
  2. Set environment variable SSLKEYLOGFILE to the absolute path of a writable file.

  3. Start the browser.
  4. Verify that the location from step 2 is created.
  5. In Wireshark, go to Preferences -> Protocols -> TLS, and change the (Pre)-Master-Secret log filename preference to the path from step 2.

  6. Start the Wireshark capture.
  7. Open a website, for example https://www.wireshark.org/

  8. Check that the decrypted data is visible. For example, using the tls and (http or http2) filter.

For Windows, an environment variable can be set globally as described in this walkthrough, but this is not recommended since it is easy to forget about and may be a security issue since it allows decryption of all your TLS traffic. A better way to set the environment variable is via a batch file. Create a file start-fx.cmd with:

  • @echo off
    set SSLKEYLOGFILE=%USERPROFILE%\Desktop\keylogfile.txt
    open firefox

Change the SSLKEYLOGFILE path as needed, and replace firefox with chrome for Google Chrome. This mechanism currently (2019) does not work for Safari, Microsoft Edge, and others since their TLS libraries (Microsoft SChannel/Apple SecureTransport) do not support this mechanism. This mechanism works for applications other than web browsers as well, but it dependent on the TLS library used by the application.

Examples of other applications:

  • Applications using OpenSSL could use a GDB or a LD_PRELOAD trick to extract the secrets. This includes Python.

  • For Java programs, pre-master secrets can be extracted from the TLS debug log preference, or output directly in the format Wireshark requires via this agent.

  • Python scripts can be edited to dump keys as well.

For a survey of supported TLS applications and libraries, see also page 19 of Peter Wu's SSL/TLS Decryption SharkFest'18 EU presentation.

Embedding decryption secrets in a pcapng file

Since Wireshark 3.0 you can embed the TLS key log file in a pcapng file. This makes it much easier to distribute capture files with decryption secrets, and makes switching between capture files easier since the TLS protocol preference does not have to be updated. To add the contents of key log file keys.txt to capture file in.pcap and write the result to out-dsb.pcapng:

  • editcap --inject-secrets tls,keys.txt in.pcap out-dsb.pcapng

The dsb suffix stands for Decryption Secrets Block (DSB) and is part of the pcapng specification.

A key log file might contain keys that are not related to a capture file. To ensure that unnecessary keys are not leaked, you can use the inject-tls-secrets.sh script from https://gist.github.com/Lekensteyn/f64ba6d6d2c6229d6ec444647979ea24 to filter the key log file and add the required secrets to a capture file. This has been tested with Linux and macOS, but not Windows. Example:

  • git clone https://gist.github.com/Lekensteyn/f64ba6d6d2c6229d6ec444647979ea24 ~/its
    ~/its/inject-tls-secrets.sh keys.txt some.pcap

See also

Some other protocols are derived from TLS. This includes:


TLS (last edited 2019-06-13 23:08:25 by PeterWu)