# Debug Unmatched Responses and tcp dropped because of gaps **URL:** \ **Category:** Beats\ **Tags:** packetbeat\ **Created:** [October 9, 2018, 11:07pm UTC](https://discuss.elastic.co/t/debug-unmatched-responses-and-tcp-dropped-because-of-gaps/151722 "2018-10-09T23:07:18Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![manavkapoor](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@manavkapoor](https://discuss.elastic.co/u/manavkapoor)\ **Post date:** [October 9, 2018, 11:07pm UTC](https://discuss.elastic.co/t/debug-unmatched-responses-and-tcp-dropped-because-of-gaps/151722/1 "2018-10-09T23:07:18Z")
Hello. I have configured Packetbeat and currently I have packetbeat flows going into a remote monitoring cluster. However, I am having trouble getting http logging sent to this same remote monitoring cluster and really appreciate some assistance. These are my settings in the config file (I only included the parts that are important/relate to this issue directly. ```auto packetbeat.interfaces.device: any packetbeat.interfaces.bpf_filter: tcp port 9200 #========================== Transaction protocols ============================= packetbeat.protocols: - type: http ports: [9200] ``` And I am receiving these logs: `{"level":"info","timestamp":"2018-10-09T23:05:27.615Z","logger":"monitoring","caller":"log/log.go:124","message":"Non-zero metrics in the last 30s","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":830,"time":830},"total":{"ticks":13890,"time":13890,"value":13890},"user":{"ticks":13060,"time":13060}},"info":{"ephemeral_id":"f168c886-7246-4eaa-9e5a-e627b12c18c8","uptime":{"ms":180011}},"memstats":{"gc_next":51770192,"memory_alloc":32270640,"memory_total":4809348680}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":283,"batches":7,"total":283},"read":{"bytes":63697},"write":{"bytes":185764}},"pipeline":{"clients":2,"events":{"active":0,"published":283,"total":283},"queue":{"acked":283}}},"system":{"load":{"1":0.08,"15":0.13,"5":0.11,"norm":{"1":0.005,"15":0.0081,"5":0.0069}}},"tcp":{"dropped_because_of_gaps":10},"xpack":{"monitoring":{"pipeline":{"events":{"published":3,"total":3},"queue":{"acked":3}}}}}}} {"level":"info","timestamp":"2018-10-09T23:05:57.616Z","logger":"monitoring","caller":"log/log.go:124","message":"Non-zero metrics in the last 30s","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":990,"time":990},"total":{"ticks":16460,"time":16469,"value":16460},"user":{"ticks":15470,"time":15479}},"info":{"ephemeral_id":"f168c886-7246-4eaa-9e5a-e627b12c18c8","uptime":{"ms":210011}},"memstats":{"gc_next":50617088,"memory_alloc":35928832,"memory_total":5735327272,"rss":1036288}},"http":{"unmatched_responses":6},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":276,"batches":7,"total":276},"read":{"bytes":62141},"write":{"bytes":181266}},"pipeline":{"clients":2,"events":{"active":0,"published":276,"total":276},"queue":{"acked":276}}},"system":{"load":{"1":0.21,"15":0.14,"5":0.13,"norm":{"1":0.0131,"15":0.0088,"5":0.0081}}},"tcp":{"dropped_because_of_gaps":2},"xpack":{"monitoring":{"pipeline":{"events":{"published":3,"total":3},"queue":{"acked":3}}}}}}}` Can I get some tips for debugging these? Thanks. ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [October 10, 2018, 9:08am UTC](https://discuss.elastic.co/t/debug-unmatched-responses-and-tcp-dropped-because-of-gaps/151722/2 "2018-10-10T09:08:09Z")
If you are on Linux, then I'd try using [af\_packet](https://www.elastic.co/guide/en/beats/packetbeat/current/configuration-interfaces.html#_sniffing_configuration_options). It's usually a little more efficient. ```auto packetbeat.interfaces.device: eth0 packetbeat.interfaces.snaplen: 1514 packetbeat.interfaces.type: af_packet packetbeat.interfaces.buffer_size_mb: 100 ``` If you are just wanting http monitoring then I would disable the flows monitoring too. This will remove the need to use the custom bpf filter since the generated one then be the same as you custom one (tcp port 9200). ```auto packetbeat.flows.enabled: false ``` ---
**Author:** ![manavkapoor](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@manavkapoor](https://discuss.elastic.co/u/manavkapoor)\ **Post date:** [October 10, 2018, 5:44pm UTC](https://discuss.elastic.co/t/debug-unmatched-responses-and-tcp-dropped-because-of-gaps/151722/3 "2018-10-10T17:44:25Z")
I have used your suggestions and now tcp.dropped\_because\_of\_gaps is a very large number. Http.unmatched\_responses also is high. ``` {"level":"info","timestamp":"2018-10-10T17:42:36.202Z","logger":"monitoring","caller":"log/log.go:124","message":"Non-zero metrics in the last 30s","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":160,"time":165},"total":{"ticks":2110,"time":2120,"value":2110},"user":{"ticks":1950,"time":1955}},"info":{"ephemeral_id":"e95515d3-b8af-49b4-ad76-51ba9186dc4d","uptime":{"ms":60009}},"memstats":{"gc_next":41192944,"memory_alloc":27027264,"memory_total":531861152,"rss":528384}},"http":{"unmatched_responses":63},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"system":{"load":{"1":0.12,"15":0.12,"5":0.1,"norm":{"1":0.0075,"15":0.0075,"5":0.0063}}},"tcp":{"dropped_because_of_gaps":9198},"xpack":{"monitoring":{"pipeline":{"events":{"published":3,"total":3},"queue":{"acked":3}}}}}}} {"level":"info","timestamp":"2018-10-10T17:43:06.202Z","logger":"monitoring","caller":"log/log.go:124","message":"Non-zero metrics in the last 30s","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":210,"time":217},"total":{"ticks":3350,"time":3358,"value":3350},"user":{"ticks":3140,"time":3141}},"info":{"ephemeral_id":"e95515d3-b8af-49b4-ad76-51ba9186dc4d","uptime":{"ms":90009}},"memstats":{"gc_next":40469584,"memory_alloc":30402272,"memory_total":887802872}},"http":{"unmatched_responses":2},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"system":{"load":{"1":0.18,"15":0.13,"5":0.12,"norm":{"1":0.0113,"15":0.0081,"5":0.0075}}},"tcp":{"dropped_because_of_gaps":11916},"xpack":{"monitoring":{"pipeline":{"events":{"published":3,"total":3},"queue":{"acked":3}}}}}}} {"level":"info","timestamp":"2018-10-10T17:43:36.202Z","logger":"monitoring","caller":"log/log.go:124","message":"Non-zero metrics in the last 30s","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":270,"time":272},"total":{"ticks":4330,"time":4336,"value":4330},"user":{"ticks":4060,"time":4064}},"info":{"ephemeral_id":"e95515d3-b8af-49b4-ad76-51ba9186dc4d","uptime":{"ms":120009}},"memstats":{"gc_next":41075264,"memory_alloc":20936872,"memory_total":1153167968,"rss":720896}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"system":{"load":{"1":0.25,"15":0.14,"5":0.14,"norm":{"1":0.0156,"15":0.0088,"5":0.0088}}},"tcp":{"dropped_because_of_gaps":8478},"xpack":{"monitoring":{"pipeline":{"events":{"published":3,"total":3},"queue":{"acked":3}}}}}}} ``` What can be the cause of this? ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [October 13, 2018, 12:15am UTC](https://discuss.elastic.co/t/debug-unmatched-responses-and-tcp-dropped-because-of-gaps/151722/4 "2018-10-13T00:15:36Z")
Can you try adding back your custom BPF filter to the config. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 10, 2018, 12:15am UTC](https://discuss.elastic.co/t/debug-unmatched-responses-and-tcp-dropped-because-of-gaps/151722/5 "2018-11-10T00:15:37Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.