EXAMPLES.md

Some helpful tips:

• When the intention is to match all packets (i.e. blank regex), it is technically faster to use an empty regex ('') than to use a wildcard (e.g. '.*', '*').

• When sniffing interfaces that are very busy or are seeing large amounts of packet traffic, make sure to craft a BPF filter to limit what PCAP has to deliver to ngrep. The ngrep parser takes a certain amount of time and while negligible on a slow interface, it can add up very quickly on a busy one.

• Hexadecimal expressions can be in straight numeric form, 'DEADBEEF', or in symbolic form, '0xDEADBEEF'. A byte is the smallest unit of measure you can match against.

• As of v1.28, ngrep doesn't require a match expression. However, there are cases where ngrep can be confused and think part of your bpf filter is the match expression, as in:

  % ngrep not port 80
  interface: eth0 (192.168.1.0/255.255.255.0)
  filter: ip and ( port 80 )
  match: not
  

  In cases like this, you will need to specify a blank match expression:

  % ngrep '' not port 80
  interface: eth0 (192.168.1.0/255.255.255.0)
  filter: ip and ( not port 80 )
  

Basic Packet Sniffing

Basic packet sniffing is easy with ngrep. It supports BPF filter logic, which means to say constraining what ngrep sees and displays is as easy as saying something like ngrep host foo.bar.com and port 25. Following are a few examples of common invocations of ngrep to do basic packet sniffing. Please note the usage of `any' as the specified ethernet adaptor to attach to; in most recent UNIX libpcap implementations this will instruct ngrep to attach to all interfaces at once, local (lo) and all external interfaces that may be active.

• ngrep -d any port 25

Monitor all activity crossing source or destination port 25 (SMTP).

• ngrep -d any 'error' port syslog

Monitor any network-based syslog traffic for the occurrence of the word "error". ngrep knows how to convert service port names (on UNIX, located in /etc/services) to port numbers.

• ngrep -wi -d any 'user|pass' port 21

Monitor any traffic crossing source or destination port 21 (FTP), looking case-insensitively for the words "user" or "pass", matched as word-expressions (the match term(s) must have non-alphanumeric, delimiting characters surrounding them).

Debugging HTTP interactions

In certain scenarios it is desirous to see how web browsers communicate with web servers, and to inspect the HTTP headers and possibly cookie values that they are exchanging.

In this example, we run an ngrep on a webserver. Since it only has one interface, eth0, we omit specifying the interface manually on the command line and allow ngrep to choose the default interface for us, for convenience.

# ngrep port 80
interface: eth0 (64.90.164.72/255.255.255.252)
filter: ip and ( port 80 )
####
T 67.169.59.38:42167 -> 64.90.164.74:80 [AP]
  GET / HTTP/1.1..User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i
  686) Opera 7.21  [en]..Host: www.darkridge.com..Accept: text/html, applicat
  ion/xml;q=0.9, application/xhtml+xml;q=0.9, image/png, image/jpeg, image/gi
  f, image/x-xbitmap, */*;q=0.1..Accept-Charset: iso-8859-1, utf-8, utf-16, *
  ;q=0.1..Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0..Cookie: SQ
  MSESSID=5272f9ae21c07eca4dfd75f9a3cda22e..Cookie2: $Version=1..Connection:
  Keep-Alive, TE..TE: deflate, gzip, chunked, identity, trailers....
##
T 64.90.164.74:80 -> 67.169.59.38:42167 [AP]
  HTTP/1.1 200 OK..Date: Mon, 29 Mar 2004 00:44:40 GMT..Server: Apache/2.0.49
  (Unix)..Last-Modified: Tue, 04 Nov 2003 12:09:41 GMT..ETag: "210e23-326-f8
  200b40"..Accept-Ranges: bytes..Vary: Accept-Encoding,User-Agent..Content-En
  coding: gzip..Content-Length: 476..Keep-Alive: timeout=15, max=100..Connect
  ion: Keep-Alive..Content-Type: text/html; charset=ISO-8859-1..Content-Langu
  age: en..............}S]..0.|...........H...8........@..\....(.....Dw.%.,..
  ;.k.....Y>q<........d ...........3.i..kdm.u@d{.Q..\....@..B1.0.2YI^..R.....
  ....X......X..y...\.....,..(........1...g.......*...j..a.`._@.W....0.....?.
  .R.K.j..Y.....>...;kw*U.j.<...\0Tn.l.:......>Fs....'....h.'...u.H4..'.6.vID
  I.......N.r.O...}...I.w. ...mX...L.s..{.L.R..-...e....~nu..t.3...H..#..J...
  .u.?..]....^..2.....e8v/gP.....].48...qD!..........#y...m}..>/?..#........I
  ..I..4.P......2:...n8l.......!.Yr&...
##

As you can see, all headers and aspects of the HTTP transmission are exposed in their gory detail. It's a little hard to parse though, so let's see what happens when -W byline mode is used:

# ngrep -W byline port 80
interface: eth0 (64.90.164.72/255.255.255.252)
filter: ip and ( port 80 )
####
T 67.169.59.38:42177 -> 64.90.164.74:80 [AP]
GET / HTTP/1.1.
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i686) Opera ...
Host: www.darkridge.com.
Accept: text/html, application/xml;q=0.9, application/xhtml+xml;q=0.9 ...
Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1.
Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0.
Cookie: SQMSESSID=5272f9ae21c07eca4dfd75f9a3cda22e.
Cookie2: $Version=1.
Cache-Control: no-cache.
Connection: Keep-Alive, TE.
TE: deflate, gzip, chunked, identity, trailers.
.

##
T 64.90.164.74:80 -> 67.169.59.38:42177 [AP]
HTTP/1.1 200 OK.
Date: Mon, 29 Mar 2004 00:47:25 GMT.
Server: Apache/2.0.49 (Unix).
Last-Modified: Tue, 04 Nov 2003 12:09:41 GMT.
ETag: "210e23-326-f8200b40".
Accept-Ranges: bytes.
Vary: Accept-Encoding,User-Agent.
Content-Encoding: gzip.
Content-Length: 476.
Keep-Alive: timeout=15, max=100.
Connection: Keep-Alive.
Content-Type: text/html; charset=ISO-8859-1.
Content-Language: en.
.
..........}S]..0.|...........H...8........@..\....(.....Dw.%.,..;.k.. ...
.;kw*U.j.<...\0Tn.l.:......>Fs....'....h.'...u.H4..'.6.vIDI.......N.r ...
..H..#..J....u.?..]....^..2.....e8v/gP.....].48...qD!..........#y...m ...
####

(Content visually truncated for display purposes.)

-W byline mode tells ngrep to respect embedded line feeds when they occur. You'll note from the output above that there is still a trailing dot (".") on each line, which is the carriage-return portion of the CRLF pair. Using this mode, now the output has become much easier to visually parse.

Processing PCAP dump files, looking for patterns

I had a friend who worked at Network Solutions and among the things he did was analyze huge 500M+ PCAP dump files of DNS traffic, looking for patterns and anomalies. ngrep was an invaluable tool for this purpose; it allowed him to take one instance of a network dump and search it quickly and repeatedly for patterns in the data packets.

To save a PCAP dump file from ngrep is very easy; simply run ngrep as you normally would but add one more command line option: -O some.file.dump (the name of the file is largely irrelevant). To illustrate another feature of ngrep, we will use the -T option (print time differential information).

# ngrep -O /tmp/dns.dump -d any -T port domain
interface: any
filter: ip and ( port domain )
output: /tmp/dns.dump
#
U +0.000000 203.115.225.24:53 -> 64.90.164.74:53
  .............m.razor2.cloudmark.com.......)........
#
U +0.000281 64.90.164.74:53 -> 203.115.225.24:53
  .............m.razor2.cloudmark.com................'.ns1...hostmaster..ws..
  ..p.... ..:.......)........
#
U +0.078184 195.113.155.7:2949 -> 64.90.164.74:53
  .............a.razor2.cloudmark.com.....
#
U +0.000351 64.90.164.74:53 -> 195.113.155.7:2949
  .............a.razor2.cloudmark.com..................agony...4..........B..
  ..............ns1...............ns2...............ns3...X..........@Z.J.j..
  ........@Z...|..........B..;
^Cexit
6 received, 0 dropped

Note the output: indicator and timestamp information. Now we have a PCAP dump file, and so let's search it for some patterns:

# ngrep -w 'm'  -I /tmp/dns.dump
input: /tmp/dns.dump
match: ((^m\W)|(\Wm$)|(\Wm\W))
#
U 203.115.225.24:53 -> 64.90.164.74:53
  .............m.razor2.cloudmark.com.......)........
#
U 64.90.164.74:53 -> 203.115.225.24:53
  .............m.razor2.cloudmark.com................'.ns1...hostmaster..ws..
  ..p.... ..:.......)........
##exit

Above we searched for the letter "m", matched as a word (-w). This yields two packets.

# ngrep -tD ns3  -I /tmp/dns.dump
input: /tmp/dns.dump
match: ns3
####
U 2004/03/28 20:32:37.088525 64.90.164.74:53 -> 195.113.155.7:2949
  .............a.razor2.cloudmark.com..................agony...4..........B..
  ..............ns1...............ns2...............ns3...X..........@Z.J.j..
  ........@Z...|..........B..;
exit

Here we've added -t which means print the absolute timestamp on the packet, and -D which means replay the packets by the time interval at which they were recorded. The latter is a neat little feature for observing the traffic at the rates/times they originally seen, though in this example it's not terribly effective as there is only one packet being matched.

# ngrep -I /tmp/dns.dump port 80
input: /tmp/dns.dump
filter: ip and ( port 80 )
exit

There's no port 80 traffic in the dump, so of course the BPF filter yields us no results.

Observing binary being transferred across the wire

One interesting feature of ngrep is its ability to take a hexademical (binary) expression and search for that in lieu of a regular expression. ngrep can also display the packets it observes in a hexadecimal format, which is more effective for inspecting binary content patterns.

In this example, we will simply look for a binary pattern in a web stream, but the more obvious usage is to look for a DDoS Zombie's unique binary signature (say, from a command packet), or even a Worm/Virus being transferred across the wire as it propogates itself.

For this test, let's assume we have a GIF on a web server that has the data pattern "0xc5d5e5f55666768696a6b6c6d6e6" (hexademical) in it. Once -X is specified, the expression will be interpreted as a hexademical pattern instead of a regular expression, and the "0x" prefix is optional.

To see a packet like this cross the wire:

# ngrep -xX '0xc5d5e5f55666768696a6b6c6d6e6' port 80
interface: eth0 (64.90.164.72/255.255.255.252)
filter: ip and ( port 80 )
match: 0xc5d5e5f55666768696a6b6c6d6e6
###
T 64.90.164.74:80 -> 67.169.59.38:42306 [A]
  ff d8 ff e0 00 10 4a 46    49 46 00 01 02 01 00 48    ......JFIF.....H
  00 48 00 00 ff ed 13 ba    50 68 6f 74 6f 73 68 6f    .H......Photosho
  70 20 33 2e 30 00 38 42    49 4d 03 ed 00 00 00 00    p 3.0.8BIM......
  00 10 00 48 00 00 00 01    00 01 00 48 00 00 00 01    ...H.......H....
  00 01 38 42 49 4d 04 0d    00 00 00 00 00 04 00 00    ..8BIM..........
  00 78 38 42 49 4d 03 f3    00 00 00 00 00 08 00 00    .x8BIM..........
  00 00 00 00 00 00 38 42    49 4d 04 0a 00 00 00 00    ......8BIM......
  00 01 00 00 38 42 49 4d    27 10 00 00 00 00 00 0a    ....8BIM'.......
  00 01 00 00 00 00 00 00    00 02 38 42 49 4d 03 f5    ..........8BIM..
  00 00 00 00 00 48 00 2f    66 66 00 01 00 6c 66 66    .....H./ff...lff
  00 06 00 00 00 00 00 01    00 2f 66 66 00 01 00 a1    ........./ff....
  99 9a 00 06 00 00 00 00    00 01 00 32 00 00 00 01    ...........2....
  00 5a 00 00 00 06 00 00    00 00 00 01 00 35 00 00    .Z...........5..
  00 01 00 2d 00 00 00 06    00 00 00 00 00 01 38 42    ...-..........8B
  49 4d 03 f8 00 00 00 00    00 70 00 00 ff ff ff ff    IM.......p......
  ff ff ff ff ff ff ff ff    ff ff ff ff ff ff ff ff    ................
  ff ff 03 e8 00 00 00 00    ff ff ff ff ff ff ff ff    ................
  ff ff ff ff ff ff ff ff    ff ff ff ff ff ff 03 e8    ................
  00 00 00 00 ff ff ff ff    ff ff ff ff ff ff ff ff    ................
  ff ff ff ff ff ff ff ff    ff ff 03 e8 00 00 00 00    ................
  ff ff ff ff ff ff ff ff    ff ff ff ff ff ff ff ff    ................
  ff ff ff ff ff ff 03 e8    00 00 38 42 49 4d 04 08    ..........8BIM..
  00 00 00 00 00 10 00 00    00 01 00 00 02 40 00 00    .............@..
  02 40 00 00 00 00 38 42    49 4d 04 14 00 00 00 00    .@....8BIM......
  00 04 00 00 00 06 38 42    49 4d 04 0c 00 00 00 00    ......8BIM......
  12 2a 00 00 00 01 00 00    00 70 00 00 00 57 00 00    .*.......p...W..
  01 50 00 00 72 30 00 00    12 0e 00 18 00 01 ff d8    .P..r0..........
  ff e0 00 10 4a 46 49 46    00 01 02 01 00 48 00 48    ....JFIF.....H.H
  00 00 ff fe 00 26 46 69    6c 65 20 77 72 69 74 74    .....&File writt
  65 6e 20 62 79 20 41 64    6f 62 65 20 50 68 6f 74    en by Adobe Phot
  6f 73 68 6f 70 a8 20 35    2e 30 ff ee 00 0e 41 64    oshop. 5.0....Ad
  6f 62 65 00 64 80 00 00    00 01 ff db 00 84 00 0c    obe.d...........
  08 08 08 09 08 0c 09 09    0c 11 0b 0a 0b 11 15 0f    ................
  0c 0c 0f 15 18 13 13 15    13 13 18 11 0c 0c 0c 0c    ................
  0c 0c 11 0c 0c 0c 0c 0c    0c 0c 0c 0c 0c 0c 0c 0c    ................
  0c 0c 0c 0c 0c 0c 0c 0c    0c 0c 0c 0c 0c 0c 0c 01    ................
  0d 0b 0b 0d 0e 0d 10 0e    0e 10 14 0e 0e 0e 14 14    ................
  0e 0e 0e 0e 14 11 0c 0c    0c 0c 0c 11 11 0c 0c 0c    ................
  0c 0c 0c 11 0c 0c 0c 0c    0c 0c 0c 0c 0c 0c 0c 0c    ................
  0c 0c 0c 0c 0c 0c 0c 0c    0c 0c 0c 0c 0c 0c 0c 0c    ................
  ff c0 00 11 08 00 57 00    70 03 01 22 00 02 11 01    ......W.p.."....
  03 11 01 ff dd 00 04 00    07 ff c4 01 3f 00 00 01    ............?...
  05 01 01 01 01 01 01 00    00 00 00 00 00 00 03 00    ................
  01 02 04 05 06 07 08 09    0a 0b 01 00 01 05 01 01    ................
  01 01 01 01 00 00 00 00    00 00 00 01 00 02 03 04    ................
  05 06 07 08 09 0a 0b 10    00 01 04 01 03 02 04 02    ................
  05 07 06 08 05 03 0c 33    01 00 02 11 03 04 21 12    .......3......!.
  31 05 41 51 61 13 22 71    81 32 06 14 91 a1 b1 42    1.AQa."q.2.....B
  23 24 15 52 c1 62 33 34    72 82 d1 43 07 25 92 53    #$.R.b34r..C.%.S
  f0 e1 f1 63 73 35 16 a2    b2 83 26 44 93 54 64 45    ...cs5....&D.TdE
  c2 a3 74 36 17 d2 55 e2    65 f2 b3 84 c3 d3 75 e3    ..t6..U.e.....u.
  f3 46 27 94 a4 85 b4 95    c4 d4 e4 f4 a5 b5 c5 d5    .F'.............
  e5 f5 56 66 76 86 96 a6    b6 c6 d6 e6 f6 37 47 57    ..Vfv........7GW
  67 77 87 97 a7 b7 c7 d7    e7 f7 11 00 02 02 01 02    gw..............
  04 04 03 04 05 06 07 07    06 05 35 01 00 02 11 03    ..........5.....
  21 31 12 04 41 51 61 71    22 13 05 32 81 91 14 a1    !1..AQaq"..2....
  b1 42 23 c1 52 d1 f0 33    24 62 e1 72 82 92 43 53    .B#.R..3$b.r..CS
  15 63 73 34 f1 25 06 16    a2 b2 83 07 26 35 c2 d2    .cs4.%......&5..
  44 93 54 a3 17 64 45 55    36 74 65 e2 f2 b3 84 c3    D.T..dEU6te.....
  d3 75 e3 f3 46 94 a4 85    b4 95 c4 d4 e4 f4 a5 b5    .u..F...........
  c5 d5 e5 f5 56 66 76 86    96 a6 b6 c6 d6 e6 f6 27    ....Vfv........'
  37 47 57 67 77 87 97 a7    b7 c7 ff da 00 0c 03 01    7GWgw...........
  00 02 11 03 11 00 3f 00    f2 a5 3a ad 35 ba 40 0e    ......?...:.5.@.
  04 16 90 78 20 a8 25 07    94 aa d3 19 18 90 41 a2    ...x .%.......A.
  13 9a 4b 9b b9 a0 91 c8    3d c8 ef a7 f2 14 46 35    ..K.....=.....F5
  af fe 6c 6f f8 73 e3 3b    7e 92 6a ad 2c 30 75 64    ..lo.s.;~.j.,0ud
  82 47 fd f9 a7 f3 5c 8a    ec d7 b5 e4 d2 4b 79 0d    .G....\......Ky.
  73 a0 ba 3f f2 49 87 8b    61 4d 88 fd de 40 4a 66    s..?.I..aM...@Jf
  51 fd e8 c7 e6 ff 00 03    f4 5a ee 63 d8 76 bd a5    Q........Z.c.v..
  a4 76 22 13 29 d9 75 b6    99 b1 ee 7c 71 b8 ca 82    .v".).u....|q...
  78 be ad 79 70 f1 1e 1b    e1 e9 c5 f3 29 24 92 49    x..yp.......)$.I
  0a 49 24 92 52 92 45 c7    c4 bf 25 c5 b4 b7 76 d1    .I$.R.E...%...v.
  2e 3c 00 3f 94 ef a2 d5    6f 33 a3 64 e1 63 7a f9    .<.?....o3.d.cz.
  0f a9 85 c5 bb 29 f5 18    eb 1c 1c 1d b9 e2 ba 9c    .....)..........
  ff 00 63 36 fe 7a 69 c9    00 44 4c 87 11 da 3d 57    ..c6.zi..DL...=W
  8c 59 0c 4c c4 4f 08 fd    2e 8d 3a da e7 1d 8d 11    .Y.L.O....:.....
  22 75 47 ca fb 35 78 d5    d2 c2 1f 7c 87 58 f6 ea    "uG..5x....|.X..
  06 91 e9 ef fc e4 1b 5f    4c 33 d1 05 a7 68 0f 27    ......._L3...h.'
  b9 fc e8 42 4a ac 83 a8    ae 8c 9e e0 84 65 00 23    ...BJ........e.#
  23 21 5c 7f 37 0c 7e 6f    47 f5 9f ff d0 f2 ae ca    #!\.7.~oG.......
  62 36 c1 3a 1f c0 84 cd    69 71 81 c9 47 a6 f6 e3    b6.:....iq..G...
  3f 75 41 af 78 e1 ef 12    27 fe 0d bf f9 24 f1 3e    ?uA.x...'....$.>
  0d 40 e2 24 55 7f 15 f0    80 91 1c 52 e0 85 eb 2a    .@.$U......R...*
  e2 ff 00 16 3f a4 c2 fc    5c 8a 1a c7 da c2 c6 d9    ....?...\.......
  f4 67 c9 05 5f ca ea 37    3c fa 77 1a b2 1b e2 01    .g.._..7<.w.....
  81 3d 83 bd 8a 2e a8 67    39 b5 e0 63 90 e6 34 9b    .=.....g9..c..4.
  00 20 ff 00 68 f1 ec 67    ef a8 63 29 50 e3 00 5f    . ..h..g..c)P.._
  51 f2 b3 f3 18 79 70 66    70 e5 26 30 e1 a8 65 1f    Q....ypfp.&0..e.
  ad 9c bf 4e 8e 3e 2c 5f    f3 d8 62 f4 dc ac a6 ef    ...N.>,_..b.....
  a8 02 c0 40 73 8b 86 93    fb df 9c b6 3a 66 36 0d    ...@s.......:f6.
  6c 73 18 45 b7 6a 2c de                               ls.E.j,.
#########

Above we specified -X to tell ngrep to treat the match expression as hexadecimal, and -x to tell ngrep to print out the patterns it matches in hexadecimal form.

As it turns out, several other packets also matched this pattern, but this should give you a good idea of how to use hexadecimal patterns and the hex output mode.