Http.response 404


(Ivan) #1

Hi,

Could you help me please to figure out what is going on?
Packetbeat returns http.response 404 for existing files.
But in the same time when I check it manually with CURL it returns 200

packetbeat version 5.6.4
elasticsearch version 5.5.2

Thanks.


(ruflin) #2

Can you please share your packetbeat config file and logs? Please also share a bit more details on what you are trying to do?


(Ivan) #3

Hi,

packetbeat.yml

#============================== Network device ================================

packetbeat.interfaces.device: any

#================================== Flows =====================================

packetbeat.flows:

timeout: 30s

period: 10s

#========================== Transaction protocols =============================

packetbeat.protocols.http:

ports: [80, 443]

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:

hosts: ["sm-prod-elastic-04.va2:9200", "sm-prod-elastic-05.va2:9200", "sm-prod-elastic-06.va2:9200"]


(ruflin) #4

Please also add the logs and a bit more details on what you are trying to do. Please also try to format the above config and logs properly to make them readable.


(Ivan) #5

the log file is full of >>>>>
2018-05-11T08:34:12Z INFO Non-zero metrics in the last 30s: http.unmatched_responses=54 libbeat.es.call_count.PublishEvents=105 libbeat.es.publish.read_bytes=51029 libbeat.es.publish.write_byte
s=3167012 libbeat.es.published_and_acked_events=4537 libbeat.publisher.messages_in_worker_queues=2073 libbeat.publisher.published_events=4520 tcp.dropped_because_of_gaps=231
2018-05-11T08:34:42Z INFO Non-zero metrics in the last 30s: http.unmatched_responses=40 libbeat.es.call_count.PublishEvents=98 libbeat.es.publish.read_bytes=46918 libbeat.es.publish.write_bytes
=2839994 libbeat.es.published_and_acked_events=4097 libbeat.publisher.messages_in_worker_queues=1719 libbeat.publisher.published_events=4119 tcp.dropped_because_of_gaps=185
2018-05-11T08:35:12Z INFO Non-zero metrics in the last 30s: http.unmatched_responses=32 libbeat.es.call_count.PublishEvents=89 libbeat.es.publish.read_bytes=42447 libbeat.es.publish.write_bytes
=2717090 libbeat.es.published_and_acked_events=3784 libbeat.publisher.messages_in_worker_queues=1884 libbeat.publisher.published_events=3782 tcp.dropped_because_of_gaps=77
2018-05-11T08:35:42Z INFO Non-zero metrics in the last 30s: http.unmatched_responses=39 libbeat.es.call_count.PublishEvents=86 libbeat.es.publish.read_bytes=41395 libbeat.es.publish.write_bytes
=2506780 libbeat.es.published_and_acked_events=3576 libbeat.publisher.messages_in_worker_queues=1758 libbeat.publisher.published_events=3540 tcp.dropped_because_of_gaps=105
2018-05-11T08:36:12Z INFO Non-zero metrics in the last 30s: http.unmatched_responses=55 libbeat.es.call_count.PublishEvents=91 libbeat.es.publish.read_bytes=43041 libbeat.es.publish.write_bytes
=2661650 libbeat.es.published_and_acked_events=3666 libbeat.publisher.messages_in_worker_queues=2055 libbeat.publisher.published_events=3691 tcp.dropped_because_of_gaps=126
2018-05-11T08:36:30Z INFO packet decode failed with: Invalid (too small) IP header length (0 < 5)
2018-05-11T08:36:42Z INFO Non-zero metrics in the last 30s: http.unmatched_responses=67 libbeat.es.call_count.PublishEvents=102 libbeat.es.publish.read_bytes=48986 libbeat.es.publish.write_byte
s=3118782 libbeat.es.published_and_acked_events=4319 libbeat.publisher.messages_in_worker_queues=2121 libbeat.publisher.published_events=4295 tcp.dropped_because_of_gaps=180
2018-05-11T08:37:12Z INFO Non-zero metrics in the last 30s: http.unmatched_responses=31 libbeat.es.call_count.PublishEvents=99 libbeat.es.publish.read_bytes=47092 libbeat.es.publish.write_bytes
=2803159 libbeat.es.published_and_acked_events=4035 libbeat.publisher.messages_in_worker_queues=1799 libbeat.publisher.published_events=4054 tcp.dropped_because_of_gaps=130
2018-05-11T08:37:42Z INFO Non-zero metrics in the last 30s: http.unmatched_responses=35 libbeat.es.call_count.PublishEvents=90 libbeat.es.publish.read_bytes=43349 libbeat.es.publish.write_bytes
=2776744 libbeat.es.published_and_acked_events=3842 libbeat.publisher.messages_in_worker_queues=1986 libbeat.publisher.published_events=3852 tcp.dropped_because_of_gaps=131
2018-05-11T08:38:12Z INFO Non-zero metrics in the last 30s: http.unmatched_responses=30 libbeat.es.call_count.PublishEvents=91 libbeat.es.publish.read_bytes=43638 libbeat.es.publish.write_bytes
=2757774 libbeat.es.published_and_acked_events=3844 libbeat.publisher.messages_in_worker_queues=2043 libbeat.publisher.published_events=3830 tcp.dropped_because_of_gaps=59
2018-05-11T08:38:42Z INFO Non-zero metrics in the last 30s: http.unmatched_responses=44 libbeat.es.call_count.PublishEvents=84 libbeat.es.publish.read_bytes=40174 libbeat.es.publish.write_bytes
=2533535 libbeat.es.published_and_acked_events=3507 libbeat.publisher.messages_in_worker_queues=1869 libbeat.publisher.published_events=3528 tcp.dropped_because_of_gaps=119
2018-05-11T08:39:12Z INFO Non-zero metrics in the last 30s: http.unmatched_responses=27 libbeat.es.call_count.PublishEvents=87 libbeat.es.publish.read_bytes=40890 libbeat.es.publish.write_bytes=2509261 libbeat.es.published_and_acked_events=3418 libbeat.publisher.messages_in_worker_queues=2013 libbeat.publisher.published_events=3378 tcp.dropped_because_of_gaps=23
2018-05-11T08:39:42Z INFO Non-zero metrics in the last 30s: http.unmatched_responses=26 libbeat.es.call_count.PublishEvents=76 libbeat.es.publish.read_bytes=35931 libbeat.es.publish.write_bytes=2211417 libbeat.es.published_and_acked_events=3032 libbeat.publisher.messages_in_worker_queues=1720 libbeat.publisher.published_events=3041 tcp.dropped_because_of_gaps=81
2018-05-11T08:40:12Z INFO Non-zero metrics in the last 30s: http.unmatched_responses=26 libbeat.es.call_count.PublishEvents=87 libbeat.es.publish.read_bytes=41027 libbeat.es.publish.write_bytes=2541132 libbeat.es.published_and_acked_events=3444 libbeat.publisher.messages_in_worker_queues=2138 libbeat.publisher.published_events=3436 tcp.dropped_because_of_gaps=37
2018-05-11T08:40:42Z INFO Non-zero metrics in the last 30s: http.unmatched_responses=32 libbeat.es.call_count.PublishEvents=85 libbeat.es.publish.read_bytes=40269 libbeat.es.publish.write_bytes=2533687 libbeat.es.published_and_acked_events=3419 libbeat.publisher.messages_in_worker_queues=2100 libbeat.publisher.published_events=3417 tcp.dropped_because_of_gaps=83
2018-05-11T08:41:12Z INFO Non-zero metrics in the last 30s: http.unmatched_responses=29 libbeat.es.call_count.PublishEvents=87 libbeat.es.publish.read_bytes=41639 libbeat.es.publish.write_bytes=2673085 libbeat.es.published_and_acked_events=3631 libbeat.publisher.messages_in_worker_queues=2229 libbeat.publisher.published_events=3666 tcp.dropped_because_of_gaps=60
2018-05-11T08:41:24Z INFO packet decode failed with: Invalid (too small) IP header length (0 < 5)


(ruflin) #6

One interesting bit in the above log:

What we are still missing here is some more details on what you exactly try to do?


(Ivan) #7

The main case: Checking the response codes for static files.
I use Packetbeat for sending the responses to ELK.


(ruflin) #8

Thanks for the details. So if I understand you correctly, you make a http request through curl and get a 200 response but the packet captured by packetbeat says it's a 400? Could you share the full json content of this document to see if there any other interesting bits inside that could give use more details?


(Ivan) #9

I make a http request through curl and get:

$ curl -I https://host/vs_fb_en/assets_haxe/cid_1525940250739/_hx_assets/remote/root-package-html5.json
HTTP/2 200
server: VSP
content-type: application/json
last-modified: Fri, 25 May 2018 11:49:24 GMT
etag: "5b07f844-4ae8c"
access-control-allow-origin: *
content-length: 306828
accept-ranges: bytes
x-varnish: 1296387168
host: host
cache-control: public, max-age=31535855
expires: Sat, 25 May 2019 13:24:26 GMT
date: Fri, 25 May 2018 13:26:51 GMT

at the same time in kibana for packetbeat index:

http.request.headers.content-length |0|
http.request.headers.content-type |text/plain; charset=UTF-8|
http.response.code |404|
http.response.headers.content-length |178|
http.response.headers.content-type |text/html|
http.response.phrase |Found|
ip |10.144.4.37|
method |GET|
path |host/vs_fb_en/assets_haxe/cid_1527253507926/_hx_assets/remote/root-package-html5.json| port |80|
proc ||
query |GET /host/vs_fb_en/assets_haxe/cid_1527253507926/_hx_assets/remote/root-package-html5.json|


(Ivan) #10

$ curl -I http://host/vs_fb_en/assets_haxe/cid_1527253507926/_hx_assets/remote/root-package-html5.json
HTTP/1.1 200 OK
Server: VSP
Content-Type: application/json
Last-Modified: Fri, 25 May 2018 11:49:24 GMT
ETag: "5b07f844-4ae8c"
Access-Control-Allow-Origin: *
Content-Length: 306828
Accept-Ranges: bytes
X-Varnish: 1296517258
Host: host
Cache-Control: public, max-age=31535955
Expires: Sat, 25 May 2019 13:32:21 GMT
Date: Fri, 25 May 2018 13:33:06 GMT
Connection: keep-alive
Set-Cookie: f5avrbbbbbbbbbbbbbbbb=JCBABHEGICPEKLCMPLJLFKNEFKLFLNKEPCMAGFEHKHDGOMAIHGPBINOMAMHGIONFFOHLINPAEPFDKHGJGBNLOKAEGMDAKDFCHACMLMCNEGPBCMAGEBDHPNKNKGGFFENK; HttpOnly


(Adrian Serrano) #11

Hi,

Your original request used HTTP/2, which is not supported by Packetbeat, so it was not captured.

The 404 error that you observe in Kibana is for a different path, so it was caused by another request. Is it possible that this request failed with a 404?

Please repeat the scenario by passing the --http1.1 argument to curl, so it doesn't use http2.


(Ivan) #12

I have got the same result for --http1.1


(Ivan) #13

I'm just wondered if I only one with this issue)


(Adrian Serrano) #14

can you show an example of a request with --http1.1 that doesn't fail and its indexed as a 404?


(Ivan) #15

32%20PM


(system) #16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.