Network data with Logstash and Packetbeat

Elastic Stack Logstash
1

I have configured packetbeat to send data to logstash and I have setup a conf file in logstash; however, seems nothing is generated, can anyone advise me for how to collect network data with packetbeat and logstash?

input{
	beats{
	port => 5044
	}
}
output{
	stdout{codec=>rubydebug}
	elasticsearch{
		hosts => "localhost"
	}
}

Great Thanks in Advance.

2

How do you know Packetbeat is collecting anything? Are there any indications in the Packetbeat logs that it's having problems sending to Logstash?

3

Oh yea, I have solved the issue as I relook at the packetbeat.yml
Thanks for your advice.
But when I start read the data from logstash, it seems that it can only read data from udp
How about if I also want to read data from http?

And as I seen from the logs, it stated dns answer as follows

"answers": [
{
"class": "IN",
"data": "googlecode.l.googleusercontent.com",
"name": "google-code-prettify.googlecode.com",
"ttl": 74280,
"type": "CNAME"
},
{
"class": "IN",
"data": "173.194.72.82",
"name": "googlecode.l.googleusercontent.com",
"ttl": 300,
"type": "A"
}
]

How can I extract the data "173.194.72.82" from it? Thanks

4

How about if I also want to read data from http?

What do you mean by this, exactly? Logstash has an http_poller input if you want Logstash to actively poll an HTTP URL and an http input where Logstash listens and receives events via inbound HTTP requests.

How can I extract the data "173.194.72.82" from it?

The answers field is an array. How do you know which element you're interested in, i.e. why do you want 173.194.72.82 rather than googlecode.l.googleusercontent.com?

5

1. I have put the http request in the configuration file as follow but it didnt seem to collect HTTP data as all the log received are transported from udp

input {
       http{}
       beats {
           port => 5044
       }
}

2. I want to map the ip with tile map visualization in kibana so I want to extract the answers field.

6

I have put the http request in the configuration file as follow but it didnt seem to collect HTTP data as all the log received are transported from udp

The http input doesn't actively collect anything. Its wait for other parties to post data to it. Do you have anything that's supposed to do that?

  1. I want to map the ip with tile map visualization in kibana so I want to extract the answers field.

Yes, but the answers field is an array with multiple values. How do you know which of the values that should be extracted?

7

I saw in kibana that the data are split like there will be an analyzed field called dns.answers.data in the hided field label. So couldn't I grab this data?

8

Sorry, I don't understand.

9

Capture.PNG1259×616 74.1 KB

I saw this in kibana which said you can still visulize the data but as it is only a string but not IP, I can't map it with tiles map.