Filebeat store under different index per host?

Elastic Stack Beats
1

Hi,

I am having different instances of filebeat running sending different logs and i want to output to elasticsearch and index per host. Is this a nested if statement and which field should I cross reference? Is it possible for someone to provide me a short example?

Thanks

2

Would you be able to use beat.hostname?

https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-beat.html#_beat_hostname

3

Thanks for your help. Still this doesnt work

output {
elasticsearch {
if beat.hostname == "testinghost1" {
hosts => "192.168.0.1:9200"
manage_template => true
index => "testing-%{+YYYY.MM.dd}"
}
}
}

Error says:
ERROR logstash.agent - Cannot create pipeline {:reason=>"Expected one of #, => at line 28, column 8 (byte 478) after output {\n elasticsearch {\n if "}

i went over this here ==> Question: Using the beat.hostname variable in Logstash config · Issue #869 · elastic/beats · GitHub

where it says

"I also had to switch from the field syntax beat.hostname to [beat][hostname] to get it to work with the file output properly" Stiill it doesnt work

4

please add some more details about your environment. It was so not clear you're trying to filter events in Logstash.

Your syntax for the field accessor is incorrect. I think in Logstash you have to use [beat][hostname].

5

I tried that and still it doesnt work. Line 28 is where the if statement is...

output {
elasticsearch {
if [beat][hostname] == "testinghost" {
hosts => "192.168.0.1:9200"
manage_template => true
index => "testing-%{+YYYY.MM.dd}"
document_type => "apache"
}
}
}

Basically there is filebeat running in two different web servers and I want to index their data separately. My filebeat part is looking alright since I can index everything together under a single index however I need to create two different.

Thanks

6

what exactly do you mean by "doesn't work"? Any error message?

This doc shows how to use conditionals in Logstash.

Have you verified the events matching structure and content as you expect?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.