VAST supports reading and writing PCAP traces via
On the read path, VAST can either acquire packets from a trace file or in live mode from a network interface.
While decapsulating packets, VAST extracts
802.1Q VLAN tags into the nested
vlan record, consisting of an
inner field for the respective
tags. The value of the VLAN tag corresponds to the 12-bit VLAN identifier (VID).
Special values include
0 (frame does not carry a VLAN ID) and
(reserved value; sometimes wildcard match).
In addition, VAST computes the Community
ID per packet to support
pivoting from other log data. The packet record contains a field
that represents the string representation of the Community ID, e.g.,
1:wCb3OG7yAFWelaUydu0D+125CLM=. If you prefer to not have the Community ID in
your data, add the option
--disable-community-id to the
To ingest a PCAP file
input.trace, pass it to the
pcap command on standard
vast import pcap < input.trace
You can also acquire packets by listening on an interface:
vast import pcap -i eth0
Real-World Traffic Replay
When reading PCAP data from a trace, VAST processes packets directly one after another. This differs from live packet capturing where there exists natural inter-packet arrival times, according to the network traffic pattern. To emulate "real-world" trace replay, VAST supports a pseudo-realtime mode, which works by introducing inter-packet delays according to the difference between subsquent packet timestamps.
-p takes a positive integer c to delay
packets by a factor of 1/c. For example, if the first packet arrives at time
t0 and the next packet at time t1, then VAST would sleep for time
(t1 - t0)/c before releasing the second packet. Intuitively, the larger c
gets, the faster the replay takes place.
For example, to replay packets as if they arrived in realtime, use
-p 1. To
replay packets twice as fast as they arrived on the NIC, use
The PCAP plugin has a few tuning knows for controlling storage of connection
data. Naive approaches, such as sampling or using a "snapshot" (
make transport-level analysis impractical due to an incomplete byte stream.
Inspired by the Time Machine, the PCAP plugin supports recording only the
first N bytes of a connection (the cutoff) and skipping the bulk of the flow
data. This allows for recording most connections in their entirety while
achieving a massive space reduction by forgoing the heavy tail of the traffic
To record only the first 1,024 bytes every connection, pass
-c 1024 as option.
Not that the cut-off is bi-directional, i.e., it applies to both the
originator and responder TCP streams and a flow gets evicted only after both
sides have reached their cutoff value.
In addition to cutoff configuration, the PCAP plugin has a few other tuning
parameters. VAST keeps a flow table with per-connection state. The
-m option specifies an upper bound on the flow table size in
number of connections. After a certain amount of inactivity of a flow,
the corresponding state expires. The option
-a controls this
timeout value. Finally, the frequency of when the flow table expires entries
can be controlled via
On the write path, VAST can write packets to a trace file.
VAST can only write PCAP traces for events of type
pcap.packet. To avoid
bogus trace file files, VAST automatically appends
#type == "pcap.packet" to
every query expression.
Below are some examples queries the generate PCAP traces. In principle, you can
also use other output formats aside from
pcap. These will render the binary
PCAP packet representation in the
Extract packets in a specific time range
VAST uses the timestamp from the PCAP header to determine the event time for a
given packet. To query all packets from the last 5 minutes, leverage the
vast export pcap 'pcap.packet.time > 5 mins ago' | tcpdump -r - -nl
Extract packets matching IPs and ports
To extract packets matching a combination of the connection 4-tuple, you can
dport fields. For example:
vast export pcap '184.108.40.206 && dport == 5158' | tcpdump -r - -nl
Extract packets matching VLAN IDs
VAST extracts outer and inner VLAN IDs from 802.1Q headers. You can query VLAN
vast export pcap 'vlan.outer > 0 || vlan.inner in [1, 2, 3]' | tcpdump -r - -nl
Special IDs include
0x000 (frame does not carry a VLAN ID) and
(reserved value; sometimes wildcard match). If you would like to check the
presence of a header, check whether it is null, e.g.,
vlan.outer != null.
Extract packet matching a Community ID
community_id field to query all packets belonging to a single flow
identified by a Community ID:
vast export pcap 'community_id == "1:wCb3OG7yAFWelaUydu0D+125CLM="' |
tcpdump -r - -nl