How to listen only specific http ports in 6.5.1

Elastic Stack Beats
1

I'm trying to use packetbeats 6.5.1 on my mac

I'd like to listen only local elastic HTTP port (9200) and my spring boot application which queries elastic (port 8078). How do I listen only to these two? What do i miss? Any help is much appreciated.

My packetbeat.yml as below

packetbeat.interfaces.device: en0

packetbeat.flows.timeout: 30s
packetbeat.flows.period: 10s
# packetbeat.flows.enabled: false

packetbeat.protocols:
- type: http
  ports: [80, 9200, 8078, 8080, 8000, 5601, 8002]
  send_request: true
  send_response: true
  send_all_headers : true
  include_body_for: ["application/json", "x-www-form-urlencoded"]


packetbeat.ignore_outgoing: true

name: qa-elastic-http

fields:
 env: qa

output.logstash:
  hosts: ["localhost:5044"]

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
2

Is the traffic on 8078 HTTP? You can disable flows and then set the http protocol to only capture 8078 and 9200.

packetbeat.flows.enabled: false
packetbeat.protocols:
- type: http
  ports: [9200, 8078]

Or you could set a custom BPF filter if you need to be more selective.

packetbeat.interfaces.device: en0
packetbeat.interfaces.bpf_filter: "tcp port 9200 or tcp port 8078"

Both of these will make Packetbeat only capture 9200 and 8078 (tcp).

3

Thanks, @andrewkroh. I'm afraid that it didn't work. I tried both approaches (disabling the flows and custom BPF filters). if I disable the flows, nothing is captured.

Is it something related to Mac Majove?

Here are my console messages. Can you think anything else would help?

2018-12-04T09:55:35.519-0600	INFO	instance/beat.go:592	Home path: [/Users/Arun/tools/packetbeat-6.5.1-darwin-x86_64] Config path: [/Users/Arun/tools/packetbeat-6.5.1-darwin-x86_64] Data path: [/Users/Arun/tools/packetbeat-6.5.1-darwin-x86_64/data] Logs path: [/Users/Arun/tools/packetbeat-6.5.1-darwin-x86_64/logs]
2018-12-04T09:55:35.519-0600	INFO	instance/beat.go:599	Beat UUID: 8364ad26-889a-4641-81ac-4f0ad64ac3b8
2018-12-04T09:55:35.519-0600	INFO	[beat]	instance/beat.go:825	Beat info	{"system_info": {"beat": {"path": {"config": "/Users/Arun/tools/packetbeat-6.5.1-darwin-x86_64", "data": "/Users/Arun/tools/packetbeat-6.5.1-darwin-x86_64/data", "home": "/Users/Arun/tools/packetbeat-6.5.1-darwin-x86_64", "logs": "/Users/Arun/tools/packetbeat-6.5.1-darwin-x86_64/logs"}, "type": "packetbeat", "uuid": "8364ad26-889a-4641-81ac-4f0ad64ac3b8"}}}
2018-12-04T09:55:35.519-0600	INFO	[beat]	instance/beat.go:834	Build info	{"system_info": {"build": {"commit": "b1c6ac83cc037bd75395334eed23b64bd63c87ef", "libbeat": "6.5.1", "time": "2018-11-16T01:42:23.000Z", "version": "6.5.1"}}}
2018-12-04T09:55:35.519-0600	INFO	[beat]	instance/beat.go:837	Go runtime info	{"system_info": {"go": {"os":"darwin","arch":"amd64","max_procs":8,"version":"go1.10.3"}}}
2018-12-04T09:55:35.521-0600	INFO	[beat]	instance/beat.go:841	Host info	{"system_info": {"host": {"architecture":"x86_64","boot_time":"2018-11-27T17:22:12.714729-06:00","name":"admins-mbp.na1.ad.group","ip":["127.0.0.1/8","::1/128","fe80::1/64","fe80::cbc:60c9:ecc:6649/64","10.12.13.73/24","fe80::948e:26ff:fef4:8c41/64","fe80::903b:ed5f:5b67:e94f/64","fe80::aede:48ff:fe00:1122/64"],"kernel_version":"18.2.0","mac":["8c:85:90:ca:71:18","0e:85:90:ca:71:18","96:8e:26:f4:8c:41","22:00:45:21:2c:01","22:00:45:21:2c:00","22:00:45:21:2c:05","22:00:45:21:2c:04","22:00:45:21:2c:01","ac:de:48:00:11:22"],"os":{"family":"darwin","platform":"darwin","name":"Mac OS X","version":"10.14.1","major":10,"minor":14,"patch":1,"build":"18B75"},"timezone":"CST","timezone_offset_sec":-21600}}}
2018-12-04T09:55:35.522-0600	INFO	[beat]	instance/beat.go:870	Process info	{"system_info": {"process": {"cwd": "/Users/Arun/tools/packetbeat-6.5.1-darwin-x86_64", "exe": "./packetbeat", "name": "packetbeat", "pid": 58638, "ppid": 58637, "start_time": "2018-12-04T09:55:35.424-0600"}}}
2018-12-04T09:55:35.522-0600	INFO	instance/beat.go:278	Setup Beat: packetbeat; Version: 6.5.1
2018-12-04T09:55:38.526-0600	INFO	add_cloud_metadata/add_cloud_metadata.go:319	add_cloud_metadata: hosting provider type not detected.
2018-12-04T09:55:38.527-0600	DEBUG	[publish]	pipeline/consumer.go:137	start pipeline event consumer
2018-12-04T09:55:38.527-0600	INFO	[publisher]	pipeline/module.go:110	Beat name: qa-elastic-http
2018-12-04T09:55:38.527-0600	INFO	procs/procs.go:91	Process watcher disabled
2018-12-04T09:55:38.528-0600	INFO	[monitoring]	log/log.go:117	Starting metrics logging every 30s
2018-12-04T09:55:38.528-0600	INFO	instance/beat.go:400	packetbeat start running.
2018-12-04T09:56:08.531-0600	INFO	[monitoring]	log/log.go:144	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":49,"time":{"ms":49}},"total":{"ticks":191,"time":{"ms":191},"value":191},"user":{"ticks":142,"time":{"ms":142}}},"info":{"ephemeral_id":"6e87e85b-b79b-4470-84f5-b463f0e8f50f","uptime":{"ms":33019}},"memstats":{"gc_next":36522784,"memory_alloc":18565744,"memory_total":22605648,"rss":45441024}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"logstash"},"pipeline":{"clients":1,"events":{"active":0}}},"system":{"cpu":{"cores":8},"load":{"1":3.3447,"15":3.3066,"5":3.2202,"norm":{"1":0.4181,"15":0.4133,"5":0.4025}}}}}}
2018-12-04T09:56:38.534-0600	INFO	[monitoring]	log/log.go:144	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":70,"time":{"ms":21}},"total":{"ticks":221,"time":{"ms":30},"value":221},"user":{"ticks":151,"time":{"ms":9}}},"info":{"ephemeral_id":"6e87e85b-b79b-4470-84f5-b463f0e8f50f","uptime":{"ms":63022}},"memstats":{"gc_next":36522784,"memory_alloc":19675792,"memory_total":23715696,"rss":126976}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"system":{"load":{"1":2.6772,"15":3.248,"5":3.0698,"norm":{"1":0.3347,"15":0.406,"5":0.3837}}}}}}
4

If no events were generated when you disabled flows then I suspect that either

  • the traffic on ports 9200, 8078 is not HTTP
  • or Packetbeat is not listening on the interface over which this traffic flows.

Is this traffic on the loopback interface?

5

They are HTTP but I'll check the interfaces. I use the default one "en0". Let me try switching the interfaces. thanks!

6

You are right @andrewkroh. The localhost traffic in my mac on interface lo0. Thanks for helping me in debugging this issue.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.