Since the setfilter is set after pcap_activate，other packets is written to the libpcap buffer

[lostf199]

When there is a large amount of traffic in the network, if you activate it first and then set the filter, it will cause other data to be written to the buffer.

clear libpcap buffer in the setfilter method

[guyharris]

Do you mean that packets that don't match the new filter show up as libpcap input?

If so, on what operating system is this?

On systems using the BPF capture mechanism, the BIOCSETF used by libpcap should flush any unread packets, so that packets that don't match the new filter, and that haven't already been read, should be discarded and never shown to whatever code is using libpcap.

On Windows, it appears that Npcap, at least, works the same way.

On Linux, libpcap should note internally that some number of blocks have not yet been read and have possibly been added to the ring buffer before the new kernel filter was installed and should itself run all the the packets in all those blocks through the new filter, so that they're discarded if they don't match the new filter.

So packets that don't match the new filter shouldn't be provided to the user; if they are provided, that's either a libpcap bug that should be fixed in libpcap, or an OS kernel bug (UN*X) or an Npcap driver bug (Windows) that should be fixed there, not something that's a pypcap bug, as pypcap is just exporting libpcap behavior here.

[guyharris]

(And there' no "clear libpcap buffer" operation in libpcap - as noted, that's what libpcap itself is supposed to do internally - so there's no routine for pypcap to call in order to clear the libpcap buffer.)

[lostf199]

Python on Linux
The libpcap method used is as follows:
pcap_create() is used to create a packet capture handle to look at packets on the network.
pcap_activate() is used to activate a packet capture handle to look at packets on the network, with the options that were set on the handle being in effect.
pcap_compile() is used to compile the string str into a filter program
pcap_dispatch get packets in buffer

When python defines pcap.pcap()，network packets have been capture into the buffer because pypcap completes pcap_create() and pcap_activate()。After pcap.setfilter() calls pcap_compile() to complete the filter settings，the previously captured packets will occupy the buffer. The packets cannot be get useing readpkts().
The default buffer size of libpcap is 2M, which can capture about 30 packets. If the Python runs for a few milliseconds with heavy network traffic, it will be full. Packets with new filtering rules will not be captured.

pypcap use pcap_create() --> pcap_compile() --> pcap_activate() --> pcap_dispatch()

Creating with pcap_create() then setting the rules with pcap_compile() and finally pcap_activate() won't have this problem

Or run pcap.readpkts() first to clear the buffer.

pc = pcap.pcap(name = interface,)
pc.setfilter("host %s" % multicast_addr)
_ = pc.readpkts()
sleep(time_span)
packets = pc.readpkts()

[guyharris]

The libpcap method used is as follows:
pcap_create() is used to create a packet capture handle to look at packets on the network.
pcap_activate() is used to activate a packet capture handle to look at packets on the network, with the options that were set on the handle being in effect.
pcap_compile() is used to compile the string str into a filter program
pcap_dispatch get packets in buffer

You forgot pcap_setfilter(). If you just compile the filter, all you get is a (classic) BPF program; it's not set as a filter for the pcap_t *.

Creating with pcap_create() then setting the rules with pcap_compile() and finally pcap_activate() won't have this problem

It will, indeed, not have that particular problem.

Instead, it will have the problem that pcap_compile() fails, reporting an error of "not-yet-activated pcap_t passed to pcap_compile", and will not compile whatever filter you pass to it. You will thus not have any filter to pass to pcap_setfilter(), and will thus have the problem that either 1) your program will give up before it even tries to activate the pcap_t * and capture traffic or 2) if it just doesn't bother setting the filter, it will capture without filtering out the packets you don't want.

Try the following program (the source code is called "testit.c.txt" rather than "testit.c" because GitHub is unaware that ".c" means "C source file", i.e. that it's a text file):

testit.c.txt

The first argument is an interface name and the second argument is a filter string.

[lostf199]

I forgot pcap_setfilter(), but that's not what I meant.

testit.c.txt

It is right. but pcap.pyx is not like this. It activate the pcap_create() and the pcap_activate() that define the pcap() at line 198. This causes the pcap_compile() and pcap_setfilter() to be set after the pcap_activate()

[guyharris]

It is right.

No, it's wrong - and deliberately so! I wrote it to demonstrate that your suggested sequence of pcap calls Will. Not. Work., by trying that sequence.

In older versions of libpcap, if you call pcap_compile() before calling pcap_activate(), it will generate code for DLT_NULL, as:

• the linktype member of the pcap_t is set in pcap_activate(), so it hasn't been set yet;
• pcap_create() initializes the pcap_t to 0, which means linktype is, in effect, initialized to DLT_NULL.

But that doesn't matter, because if you call pcap_setfilter() before calling pcap_activate(), pcap_setfilter() will fail with PCAP_ERROR_NOT_ACTIVATED, so, if you call pcap_activate() anyway, there will be no filter, and you will get all packets, not just the ones you want.

In newer versions of libpcap, if you call pcap_compile() before calling pcap_activate(), pcap_compile() will fail and return -1, without generating any filter. If you then call pcap_setfilter(), it will, fortunately, not look at the uninitialized bpf_program structure - it will, just as in older versions of libpcap, fail with PCAP_ERROR_NOT_ACTIVATED, so, if you call pcap_activate() anyway, there will be no filter, and you will get all packets, not just the ones you want.

but pcap.pyx is not like this.

Yes, because "this" Does. Not. Work. Instead, it does what does work.

[lostf199]

Thanks.
This is like a libpcap issus.