Streams
Description
Streams is developed by Tillmann Werner, Senior Virus Analyst by Kaspersky Lab. Streams is a tool for browsing, mining and processing TCP streams in pcap files and will be particularly suitable for large pcap files.
Streams' reassembly processor is based on following standards:
- A SYN segment starts a new stream
- A RST or FIN segment terminates a stream
- Any segment gets copied at the right offset according to its sequence number
External resources are available here:
Installation
Prerequisites
$ sudo apt-get install libreadline-dev libtool
Installation via git
This is the recommended way to install streams as the tar.bz2 source contains a bug in the time of this writing.
$ cd /data/src/ $ git clone git://git.carnivore.it/streams.git $ cd streams/ $ autoreconf -fv --install $ ./configure --with-libreadline-includes=/usr/include/readline/ $ make $ sudo make install
Installation via FTP
As there is a known bug in the tar.bz2 (not corrected yet in the time of this writing), it is recommended that you install streams via the git tree.
$ cd /data/src/ $ wget ftp://ftp.carnivore.it/projects/streams/streams-0.1.0.tar.bz2 $ bzip2 -cd streams-0.1.0.tar.bz2 | tar xf - $ cd streams-0.1.0/ $ ./configure --with-libreadline-includes=/usr/include/readline/ $ make $ sudo make install
Usage
Syntax
/opt/streams/bin$ ./streams
_
_____ _____ ___| |_ _ __ ___ __ _ _ __ ___ ___
_____ |_____|_____| / __| __| '__/ _ \/ _` | '_ ` _ \/ __|
|_____| _ |_____|_____| \__ \ |_| | | __/ (_| | | | | | \__ \ _ _ _
(_|_)____ (_) |___/\__|_| \___|\__,_|_| |_| |_|___/ (_) (_|_)
|_____|
version 0.1.0, Copyright (C) 2011 by Tillmann Werner
streams> help
Options
- analyze
- analyze trace file. See example
- bpf
- specify a berkeley packet filter expression
- count
- display number of streams
- dump
- dump selected stream to a file (see outfile). See example
- ext
- specify external program (+ arguments) to pipe streams through (see pipe). See example
- filter
- toggle stream filter status (include/exclude empty and incomplete streams)
- help
- show help (this output)
- list
- list streams. See example
- match
- specify a content pattern, use 'x [pattern]' for patterns in hexadecimal encoding. See example
- offset
- set datalink layer offset for packet trace file
- outfile
- specify an output file for stream dumps (see dump). See example
- pipe
- pipe selected stream through an external program (see ext). See example
- quit
- quit program
- status
- display program status
- timestamps
- toggle time display format (absolute/relative)
Examples
analyze
analyze is a prerequisite to all other commands. It loads the file in the engine.
streams> analyze evidence02.pcap file processed, 4 streams (4 non-empty and complete).
list
list enables to list streams inside a pcap. The example shows 4 streams: streams> list
0: 0.000000 1.562695 192.168.1.159:1036 > 64.12.102.142:587 (1495 bytes) 1: 0.109879 1.552142 64.12.102.142:587 > 192.168.1.159:1036 (507 bytes) 2: 160.087705 164.204003 192.168.1.159:1038 > 64.12.102.142:587 (285778 bytes) 3: 160.198821 164.198079 64.12.102.142:587 > 192.168.1.159:1038 (507 bytes)
ext
ext enables to define an external command to read the packets
streams> ext hd
pipe
Once you have defined an external tool to parse the content of the packets, you can use pipe and the number of the stream to analyze:
streams> pipe 0 00000000 45 48 4c 4f 20 61 6e 6e 6c 61 70 74 6f 70 0d 0a |EHLO annlaptop..| 00000010 41 55 54 48 20 4c 4f 47 49 4e 0d 0a 63 32 35 6c |AUTH LOGIN..c25l| 00000020 59 57 74 35 5a 7a 4d 7a 61 30 42 68 62 32 77 75 |YWt5ZzMza0Bhb2wu| 00000030 59 32 39 74 0d 0a 4e 54 55 34 63 6a 41 77 62 48 |Y29t..NTU4cjAwbH| 00000040 6f 3d 0d 0a 4d 41 49 4c 20 46 52 4f 4d 3a 20 3c |o=..MAIL FROM: <| 00000050 73 6e 65 61 6b 79 67 33 33 6b 40 61 6f 6c 2e 63 |[email protected]| 00000060 6f 6d 3e 0d 0a 52 43 50 54 20 54 4f 3a 20 3c 73 |om>..RCPT TO: <s| 00000070 65 63 35 35 38 40 67 6d 61 69 6c 2e 63 6f 6d 3e |[email protected]>| 00000080 0d 0a 44 41 54 41 0d 0a 4d 65 73 73 61 67 65 2d |..DATA..Message-| 00000090 49 44 3a 20 3c 30 30 30 39 30 31 63 61 34 39 61 |ID: <000901ca49a| 000000a0 65 24 38 39 64 36 39 38 63 30 24 39 66 30 31 61 |e$89d698c0$9f01a| 000000b0 38 63 30 40 61 6e 6e 6c 61 70 74 6f 70 3e 0d 0a |8c0@annlaptop>..| 000000c0 46 72 6f 6d 3a 20 22 41 6e 6e 20 44 65 72 63 6f |From: "Ann Derco| 000000d0 76 65 72 22 20 3c 73 6e 65 61 6b 79 67 33 33 6b |ver" <sneakyg33k| 000000e0 40 61 6f 6c 2e 63 6f 6d 3e 0d 0a 54 6f 3a 20 3c |@aol.com>..To: <| 000000f0 73 65 63 35 35 38 40 67 6d 61 69 6c 2e 63 6f 6d |[email protected]| 00000100 3e 0d 0a 53 75 62 6a 65 63 74 3a 20 6c 75 6e 63 |>..Subject: lunc| 00000110 68 20 6e 65 78 74 20 77 65 65 6b 0d 0a 44 61 74 |h next week..Dat| 00000120 65 3a 20 53 61 74 2c 20 31 30 20 4f 63 74 20 32 |e: Sat, 10 Oct 2| 00000130 30 30 39 20 30 37 3a 33 35 3a 33 30 20 2d 30 36 |009 07:35:30 -06| 00000140 30 30 0d 0a 4d 49 4d 45 2d 56 65 72 73 69 6f 6e |00..MIME-Version| [...TRUNCATED...]
outfile & dump
First select an output file with outfile:
streams> outfile test02
streams> dump
dump: need an argument
streams> list
0: 0.000000 1.562695 192.168.1.159:1036 > 64.12.102.142:587 (1495 bytes)
1: 0.109879 1.552142 64.12.102.142:587 > 192.168.1.159:1036 (507 bytes)
2: 160.087705 164.204003 192.168.1.159:1038 > 64.12.102.142:587 (285778 bytes)
3: 160.198821 164.198079 64.12.102.142:587 > 192.168.1.159:1038 (507 bytes)
You can then use dump with the number of the stream to dump the content of the stream in your external file:
streams> dump 2 285778 bytes written to test02
Filters: bpf, filter and match
You can use one of the following filters to minimize data: bpf, filter and match.
Here is an example. The unfiltered pcap is composed of 4 streams as follows:
streams> list
0: 0.000000 1.562695 192.168.1.159:1036 > 64.12.102.142:587 (1495 bytes)
1: 0.109879 1.552142 64.12.102.142:587 > 192.168.1.159:1036 (507 bytes)
2: 160.087705 164.204003 192.168.1.159:1038 > 64.12.102.142:587 (285778 bytes)
3: 160.198821 164.198079 64.12.102.142:587 > 192.168.1.159:1038 (507 bytes)
By filtering streams that contain the keyword "GET" (HTTP requests), we minimize to only one stream:
streams> match GET
applying new match expression...
streams> list
2: 160.087705 164.204003 192.168.1.159:1038 > 64.12.102.142:587 (285778 bytes)