How packetbeat to work with pf_ring

shell.b2t · October 16, 2016, 5:21am

I have set as follows :

interfaces:
device: any
type: pf_ring

but error as follows:

2016/10/16 05:16:37.012239 packetbeat.go:215: CRIT Initializing sniffer failed: Error creating sniffer: Pfring sniffing is not compiled in

monica · October 16, 2016, 8:46pm

In order to be able to use pf_ring, you need to load a kernel module. For more details please have a look at ntop documentation.

andrewkroh · October 17, 2016, 1:22pm

@shell.b2t See Starting packetbeat: 2016/01/08 12:13:43.009060 packetbeat.go:195: CRIT Initializing sniffer failed: Error creating sniffer: Pfring sniffing is not compiled in

shell.b2t · October 17, 2016, 3:59pm

thanks

I have make with "go build --tags havepfring"
when i run "./packetbeat -c packetbeat.yml -e",some error as follows:
CRIT Exiting: Initializing sniffer failed: Error creating sniffer: SetBPFFilter failed: Unable to set BPF filter, got error code -1
xiting: Initializing sniffer failed: Error creating sniffer: SetBPFFilter failed: Unable to set BPF filter, got error code -1

my packetbeat.yml:
============================== Network device ============================

# Select the network interface to sniff the data. On Linux, you can use the
# "any" keyword to sniff on all connected interfaces.
packetbeat.interfaces.device: ens4f1
packetbeat.interfaces.type: pf_ring
================================== Flows =====================================

all message as follows:

2016/10/17 15:51:37.581632 beat.go:264: INFO Home path: [/root/src/github.com/elastic/beats/packetbeat] Config path: [/root/src/github.com/elastic/beats/packetbeat] Data path: [/root/src/github.com/elastic/beats/packetbeat/data] Logs path: [/root/src/github.com/elastic/beats/packetbeat/logs]
        2016/10/17 15:51:37.581702 beat.go:174: INFO Setup Beat: packetbeat; Version: 6.0.0-alpha1
        2016/10/17 15:51:37.581821 logp.go:219: INFO Metrics logging every 30s
        2016/10/17 15:51:37.581924 output.go:167: INFO Loading template enabled. Reading template file: /root/src/github.com/elastic/beats/packetbeat/packetbeat.template.json
        2016/10/17 15:51:37.584692 output.go:178: INFO Loading template enabled for Elasticsearch 2.x. Reading template file: /root/src/github.com/elastic/beats/packetbeat/packetbeat.template-es2x.json
        2016/10/17 15:51:37.587822 client.go:107: INFO Elasticsearch url: http://localhost:9200
        2016/10/17 15:51:37.587901 outputs.go:106: INFO Activated elasticsearch as output plugin.
        2016/10/17 15:51:37.588063 publish.go:291: INFO Publisher name: localhost.localdomain
        2016/10/17 15:51:37.590137 async.go:63: INFO Flush Interval set to: 1s
        2016/10/17 15:51:37.590188 async.go:64: INFO Max Bulk Size set to: 50
        2016/10/17 15:51:37.590348 procs.go:91: INFO Process matching disabled
        2016/10/17 15:51:37.591090 protos.go:89: INFO registered protocol plugin: cassandra
        2016/10/17 15:51:37.591131 protos.go:89: INFO registered protocol plugin: mysql
        2016/10/17 15:51:37.591152 protos.go:89: INFO registered protocol plugin: nfs
        2016/10/17 15:51:37.591174 protos.go:89: INFO registered protocol plugin: pgsql
        2016/10/17 15:51:37.591193 protos.go:89: INFO registered protocol plugin: thrift
        2016/10/17 15:51:37.591211 protos.go:89: INFO registered protocol plugin: amqp
        2016/10/17 15:51:37.591230 protos.go:89: INFO registered protocol plugin: dns
        2016/10/17 15:51:37.591252 protos.go:89: INFO registered protocol plugin: http
        2016/10/17 15:51:37.591274 protos.go:89: INFO registered protocol plugin: memcache
        2016/10/17 15:51:37.591295 protos.go:89: INFO registered protocol plugin: mongodb
        2016/10/17 15:51:37.591317 protos.go:89: INFO registered protocol plugin: redis
        2016/10/17 15:51:37.697358 beat.go:285: CRIT Exiting: Initializing sniffer failed: Error creating sniffer: SetBPFFilter failed: Unable to set BPF filter, got error code -1
        Exiting: Initializing sniffer failed: Error creating sniffer: SetBPFFilter failed: Unable to set BPF filter, got error code -1

andrewkroh · October 17, 2016, 5:22pm

When running Packetbeat add -d "*" so that more debugging information is logged, then post that output.

shell.b2t · October 18, 2016, 12:27am

as following:

[root@localhost packetbeat]# ./packetbeat -c packetbeat.yml -e -d "*"
2016/10/18 00:26:32.873613 beat.go:264: INFO Home path: [/root/src/github.com/elastic/beats/packetbeat] Config path: [/root/src/github.com/elastic/beats/packetbeat] Data path: [/root/src/github.com/elastic/beats/packetbeat/data] Logs path: [/root/src/github.com/elastic/beats/packetbeat/logs]
2016/10/18 00:26:32.873675 beat.go:174: INFO Setup Beat: packetbeat; Version: 6.0.0-alpha1
2016/10/18 00:26:32.873699 processor.go:43: DBG  Processors: 
2016/10/18 00:26:32.873720 beat.go:180: DBG  Initializing output plugins
2016/10/18 00:26:32.873729 logp.go:219: INFO Metrics logging every 30s
2016/10/18 00:26:32.873893 output.go:167: INFO Loading template enabled. Reading template file: /root/src/github.com/elastic/beats/packetbeat/packetbeat.template.json
2016/10/18 00:26:32.875812 output.go:178: INFO Loading template enabled for Elasticsearch 2.x. Reading template file: /root/src/github.com/elastic/beats/packetbeat/packetbeat.template-es2x.json
2016/10/18 00:26:32.877960 client.go:107: INFO Elasticsearch url: http://localhost:9200
2016/10/18 00:26:32.878028 outputs.go:106: INFO Activated elasticsearch as output plugin.
2016/10/18 00:26:32.878052 publish.go:234: DBG  Create output worker
2016/10/18 00:26:32.878137 publish.go:276: DBG  No output is defined to store the topology. The server fields might not be filled.
2016/10/18 00:26:32.878186 publish.go:291: INFO Publisher name: localhost.localdomain
2016/10/18 00:26:32.879847 async.go:63: INFO Flush Interval set to: 1s
2016/10/18 00:26:32.879881 async.go:64: INFO Max Bulk Size set to: 50
2016/10/18 00:26:32.879898 async.go:72: DBG  create bulk processing worker (interval=1s, bulk size=50)
2016/10/18 00:26:32.880030 procs.go:91: INFO Process matching disabled
2016/10/18 00:26:32.880653 packetbeat.go:109: DBG  Initializing protocol plugins
2016/10/18 00:26:32.880683 protos.go:89: INFO registered protocol plugin: nfs
2016/10/18 00:26:32.880700 protos.go:89: INFO registered protocol plugin: redis
2016/10/18 00:26:32.880717 protos.go:89: INFO registered protocol plugin: http
2016/10/18 00:26:32.880732 protos.go:89: INFO registered protocol plugin: memcache
2016/10/18 00:26:32.880745 protos.go:89: INFO registered protocol plugin: mongodb
2016/10/18 00:26:32.880760 protos.go:89: INFO registered protocol plugin: mysql
2016/10/18 00:26:32.880774 protos.go:89: INFO registered protocol plugin: pgsql
2016/10/18 00:26:32.880789 protos.go:89: INFO registered protocol plugin: amqp
2016/10/18 00:26:32.880803 protos.go:89: INFO registered protocol plugin: cassandra
2016/10/18 00:26:32.880817 protos.go:89: INFO registered protocol plugin: dns
2016/10/18 00:26:32.880832 protos.go:89: INFO registered protocol plugin: thrift
2016/10/18 00:26:32.880920 packetbeat.go:115: DBG  Initializing sniffer
2016/10/18 00:26:32.880947 sniffer.go:269: DBG  BPF filter: ''
2016/10/18 00:26:32.880968 sniffer.go:155: DBG  Sniffer type: pf_ring device: ens4f1
2016/10/18 00:26:32.970878 beat.go:285: CRIT Exiting: Initializing sniffer failed: Error creating sniffer: SetBPFFilter failed: Unable to set BPF filter, got error code -1
Exiting: Initializing sniffer failed: Error creating sniffer: SetBPFFilter failed: Unable to set BPF filter, got error code -1
[root@localhost packetbeat]#

andrewkroh · October 18, 2016, 1:30am

Can you try disabling flows (either comment out of add enabled: false). This should cause the log line containing DBG BPF filter: '' to change. It could be that the empty BPF is causing the problem for pf_ring which would be a bug.

packetbeat.flows:
  enabled: false

shell.b2t · October 18, 2016, 5:43am

as following:

   [root@localhost packetbeat]# ./packetbeat -c packetbeat.yml -e -d "*"
    2016/10/18 05:39:06.894818 beat.go:264: INFO Home path: [/root/src/github.com/elastic/beats/packetbeat] Config path: [/root/src/github.com/elastic/beats/packetbeat] Data path: [/root/src/github.com/elastic/beats/packetbeat/data] Logs path: [/root/src/github.com/elastic/beats/packetbeat/logs]
    2016/10/18 05:39:06.894877 beat.go:174: INFO Setup Beat: packetbeat; Version: 6.0.0-alpha1
    2016/10/18 05:39:06.894901 processor.go:43: DBG  Processors: 
    2016/10/18 05:39:06.894921 beat.go:180: DBG  Initializing output plugins
    2016/10/18 05:39:06.894929 logp.go:219: INFO Metrics logging every 30s
    2016/10/18 05:39:06.895084 output.go:167: INFO Loading template enabled. Reading template file: /root/src/github.com/elastic/beats/packetbeat/packetbeat.template.json
    2016/10/18 05:39:06.907008 output.go:178: INFO Loading template enabled for Elasticsearch 2.x. Reading template file: /root/src/github.com/elastic/beats/packetbeat/packetbeat.template-es2x.json
    2016/10/18 05:39:06.910049 client.go:107: INFO Elasticsearch url: http://localhost:9200
    2016/10/18 05:39:06.910113 outputs.go:106: INFO Activated elasticsearch as output plugin.
    2016/10/18 05:39:06.910137 publish.go:234: DBG  Create output worker
    2016/10/18 05:39:06.910230 publish.go:276: DBG  No output is defined to store the topology. The server fields might not be filled.
    2016/10/18 05:39:06.910286 publish.go:291: INFO Publisher name: localhost.localdomain
    2016/10/18 05:39:06.911768 async.go:63: INFO Flush Interval set to: 1s
    2016/10/18 05:39:06.911799 async.go:64: INFO Max Bulk Size set to: 50
    2016/10/18 05:39:06.911819 async.go:72: DBG  create bulk processing worker (interval=1s, bulk size=50)
    2016/10/18 05:39:06.911946 procs.go:91: INFO Process matching disabled
    2016/10/18 05:39:06.912521 packetbeat.go:109: DBG  Initializing protocol plugins
    2016/10/18 05:39:06.912551 protos.go:89: INFO registered protocol plugin: http
    2016/10/18 05:39:06.912568 protos.go:89: INFO registered protocol plugin: memcache
    2016/10/18 05:39:06.912580 protos.go:89: INFO registered protocol plugin: mysql
    2016/10/18 05:39:06.912801 packetbeat.go:115: DBG  Initializing sniffer
    2016/10/18 05:39:06.912840 sniffer.go:269: DBG  BPF filter: 'tcp port 80 or tcp port 8080 or tcp port 8000 or tcp port 5000 or tcp port 8002'
    2016/10/18 05:39:06.912862 sniffer.go:155: DBG  Sniffer type: pf_ring device: ens4f1
    2016/10/18 05:39:07.013588 tcp.go:307: DBG  tcp%!(EXTRA string=Port map: %v, map[uint16]protos.Protocol=map[80:http 8080:http 8000:http 5000:http 8002:http])
    2016/10/18 05:39:07.013649 udp.go:94: DBG  Port map: map[]
    2016/10/18 05:39:07.013703 decoder.go:98: DBG  Layer type: Ethernet
    2016/10/18 05:39:07.013823 beat.go:204: INFO packetbeat start running.
    2016/10/18 05:39:07.013860 packetbeat.go:161: DBG  Waiting for the sniffer to finish
    panic: runtime error: cgo argument has Go pointer to Go pointer

    goroutine 14 [running]:
    panic(0xa6e440, 0xc4211d90c0)
    	/usr/local/go/src/runtime/panic.go:500 +0x1a1
    github.com/elastic/beats/vendor/github.com/tsg/gopacket/pfring.(*Ring).ReadPacketDataTo(0xc4200fc2d0, 0xc4212e6000, 0xffff, 0xffff, 0x0, 0x0, 0x0, 0x0, 0x0, 0x28, ...)
    	/root/src/github.com/elastic/beats/vendor/github.com/tsg/gopacket/pfring/pfring.go:116 +0x1aa
    github.com/elastic/beats/vendor/github.com/tsg/gopacket/pfring.(*Ring).ReadPacketData(0xc4200fc2d0, 0x0, 0x0, 0xc420060800, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    	/root/src/github.com/elastic/beats/vendor/github.com/tsg/gopacket/pfring/pfring.go:135 +0xba
    github.com/elastic/beats/packetbeat/sniffer.(*PfringHandle).ReadPacketData(0xc4200800d8, 0xc42121ce70, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    	/root/src/github.com/elastic/beats/packetbeat/sniffer/pfring.go:37 +0x7f
    github.com/elastic/beats/packetbeat/sniffer.(*SnifferSetup).Run(0xc420072fc0, 0xbab990, 0xc4211d9060)
    	/root/src/github.com/elastic/beats/packetbeat/sniffer/sniffer.go:309 +0xe1
    github.com/elastic/beats/packetbeat/beater.(*Packetbeat).Run.func2(0xc4211d9060, 0xc4200ac8c0, 0xc420073380)
    	/root/src/github.com/elastic/beats/packetbeat/beater/packetbeat.go:155 +0x6a
    created by github.com/elastic/beats/packetbeat/beater.(*Packetbeat).Run
    	/root/src/github.com/elastic/beats/packetbeat/beater/packetbeat.go:159 +0x1b2

tudor · October 18, 2016, 8:50am

Hmm, might be that it got broken with newer Golang versions (Cgo backwards compatibility is not guaranteed). What Go version are you using?

shell.b2t · October 18, 2016, 9:13am

go version go1.7.1 linux/amd64

tudor · October 18, 2016, 9:18am

Thanks, I think it's a bug. Can you open a ticket on the Github repo, please?

Also, I'm wondering if af_packet wouldn't be enough in your case? Do you really need pfring?

andrewkroh · October 18, 2016, 1:16pm

That looks similar the trace recently posted here GO Panic with v1.3.1 pf_ring support. Please open a new Github issue.

system · November 6, 2016, 5:21am

This topic was automatically closed after 21 days. New replies are no longer allowed.