What's the best way to track calls to a specific URL?

Elastic Stack Beats
1

I have Packetbeat installed on an application server. I'd like a way to determine how many calls to a given URL (say, www.xyz.com) are made per time frame I'm searching on in Kibana. What would be the best way to accomplish this, via Packetbeat? There are the flows, and there are also the HTTP and DNS protocols that I imagine would be useful.

What's difficult with the flows and the HTTP protocol is the log information appears to be happening after the domain name has been translated to an IP address. I don't know the IP addresses of this URL ahead of time, and I don't think I can rely on it staying the same.

DNS protocol logs seem like they might be useful here. I believe the dns.question.name maps to the URL that is being called. But it doesn't look like those logs map 1 to 1 with calls to those URLs. (Once the domain name has been resolved to an IP address, I imagined it's cached somewhere?)

Thanks!

2

If you configure packetbeat to record the Host header you will know the domain name.

If you are looking to track incoming requests to the application server then the DNS traffic won't be helpful because the DNS requests will occur on the client side before making the HTTP request to your server.

3

Hey Andrew,

What field am I looking for with the once I add send_headers: ["Host"]?

4

The request headers are written to http.request.headers. So http.request.headers.Host.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.