Development/PcapNg

WARNING: This wiki page is woefully out of date.

PcapNg

The PCAP Next Generation Dump File Format (or pcapng for short) is an attempt to overcome the limitations of the currently widely used (but limited) libpcap format.

The pcapng file format specification is still work in progress, see:

Example pcapng Capture Files

Included below are some example pcapng files. While the block headers for these files appears to be formatted correctly, the payloads contained within these may NOT necessarily be well-formed (this is especially true of the "http.bigendian.ntar" file). Examples of pcapng files are currently very scarce. A few of these files were found elsewhere on the net (most likely the at the winpcap.org site referenced above). The "http.littlendian.ntar" file was found as an attachment(under a different name) in an email regarding pcapng. The http.bigendian.ntar is a version of this same file that was edited by hand to illustrate how a big-endian and little-endian machine might write the same data.

Some more recent example files:

Please note: The recommended file name extension for pcapng files is .pcapng

Some very old example files:

Test pcapng Files

There is a github repository of various test files to try a pcapng reader/parser on, in both little endian and big endian formats.

You can dump the contents of a pcapng file with TShark:

tshark -V -Xread_format:"MIME Files Format" -r /path/to/a.pcapng

Challenging pcapng Files

Malformed pcapng Files

ntartest - a simplistic standalone pcapng (ntar) file reader

Included below is the C source code to a very simplistic program to read and dump header information about a pcapng (a.k.a. ntar) file. This program has been successfully compiled using gcc and used on several different types of systems including Linux, cygwin and Solaris 9.

To compile this program use the command:

# gcc -o ntartest ntartest.c

To use the program provide a pcapng file as the first argument to the program:

# ntartest http.littleendian.pcapng

File

File 5.08+ should be able to identify pcapng files (ml thread).

NTAR

The Network Trace Archival and Retrieval library is able to read and write pcapng files. Caveats: i) frozen since 2007, ii) no plugins for EPB, ISB, and NRB blocks. Mailing list archives.

Libpcap

As of the 1.1.0 release, libpcap adds limited support for reading pcapng files, see CHANGES.

Wireshark

As of Wireshark 1.2.0, pcapng files can be read and written, and live captures can be done in pcapng format as well as pcap format. There were a number of bugs in 1.2.0's support that were fixed in 1.2.1.

The current limitations for pcapng format are:

tshark 1.6 should be able to read and write NRB blocks (command line options: -W n and -H hosts_file).

dumpcap 1.4/1.6/1.7 uses the libpcap_write_.*_block functions from pcapio.c to write the pcapng blocks. It currently writes SHB, IDB, EPB and ISB blocks.

Options working

These options are working:

These options will only be displayed on the console (missing API from wiretap to Wireshark):

Options status

Block Option dumpcap pcapio.{c,h} Wiretap API Wireshark GUI
All
(0) opt_endofopt OK OK -
(1) opt_comment SHB, IDB, ISB WIP: see bug #3096 (closed) see per block
SHB Section Header Block WIP: see bug #3096 (closed)
(1) opt_comment NULL (1.7.1 rev 41026) (1.7.1 rev 41032) Menu Statistics/Summary
(2) shb_hardware NULL (1.7.1 rev 41026) (1.7.1 rev 41032) Menu Statistics/Summary
(3) shb_os OK (1.7.1 rev 41026) (1.7.1 rev 41032) Menu Statistics/Summary
(4) shb_userappl OK (1.7.1 rev 41032) Menu Statistics/Summary
IDB Interface Description Block WIP: see bug #3096 (closed); rev 41082
(1) opt_comment NULL (1.7.1 rev 41108)
(2) if_name OK
(3) if_description NULL (1.7.1 rev 41108)
(4) if_IPv4addr
(5) if_IPv6addr
(6) if_MACaddr
(7) if_EUIaddr
(8) if_speed 0 (1.7.1 rev 41108)
(9) if_tsresol 6 or 9 (1.7.1 rev 41108, rev 41455)
(10) if_tzone
(11) if_filter OK
(12) if_os OK (1.7.1 rev 41108)
(13) if_fcslen
(14) if_tsoffset
EPB Enhanced Packet Block
(1) opt_comment NULL EditMenu ExpertInfo StatusBar
(2) epb_flags
(3) epb_hash
(4) epb_dropcount
NRB Name Resolution Block
(2) ns_dnsname
(3) ns_dnsIP4addr
(4) ns_dnsIP6addr
ISB Interface Statistics Block WIP (1.7.1 rev 41137)
(1) opt_comment OK (1.7.1 revs 41136 and 41160) OK
(2) isb_starttime OK (1.7.1 revs 41136 and 41160) OK
(3) isb_endtime OK (1.7.1 revs 41136 and 41160) OK
(4) isb_ifrecv OK OK
(5) isb_ifdrop OK OK Menu Statistics/Summary
(6) isb_filteraccept OK
(7) isb_osdrop OK
(8) isb_usrdeliv OK

Future

So after implementing the "skeleton" to read the pcapng file, there's a lot more to do:

Wishlist

List here things users would like Wireshark to support in pcapng (this was recorded at the Developer's Roundtable at Sharkfest '10).

Existing pcapng features:

Existing, but insufficiently specified, pcapng features:

Features not yet in pcapng:


FAQ

File format conversion

From .pcap to .pcapng

Using editcap or tshark:

From .pcapng to .pcap

Using editcap:

Note: you need to specify an encapsulation type (-T <encapsulation type>) (1)

Using tcpdump/libpcap (4.1.0+/1.1.0+) (1,2):

Merging pcapng files

Again, you need to manually specify an encapsulation type. For example:

Note: This operation can't still be done via Wireshark's File/Merge ... (as of rev 40759).

Create pcap files instead of pcapng files

You may use the dumpcap's -P command line option if you need to create capture files in the old libpcap format (instead of the new default pcapng):


Imported from https://wiki.wireshark.org/Development/PcapNg on 2020-08-11 23:12:58 UTC