Wrong ElasticSearch template being used


(Corie White) #1

I am not sure if this is the correct forum for this but I am having issues with one of my logstash configurations when it is sent to elasticsearch. I am brand new to ELK and still learning how everything works.

So far, I have 2 logstash configurations setup, one is for NetFlow data and the other is for local logs (I am also using packetbeat but I don't think that is related). So, the NetFlow template works great for the NetFlow data but my issue is that when I send the local logs to ElasticSearch, it is trying to use the NetFlow template for the local logs aswell.

I am testing this with CentOS on /var/log/messages so it is typical syslog messages. I tried setting up a Grok filter but I don't even know if that is necessary (also it didn't help). Below is my configurations:

Local logs (syslog)

NetFlow

My templates wouldn't fit into the body: http://coriewhite.me/elasticsearchtemplates.html

Please let me know if you guys need any further information and thank you in advanced for the help!


(Mark Walkom) #2

How are you defining the Netflow template?
Also are you merging these configs?


(Corie White) #3

Well, the configs are in separate files:

Not sure what you mean by defining NetFlow template but it's under the _templates like so:

curl -XGET localhost:9200/_template/logstash_netflow5?pretty


(Mark Walkom) #4

Even though these are different files, they get merged by LS at run time.
So you need to add a conditional to the output to make sure each message goes to it's correct config.


(Corie White) #5

Oo thank you! Sounds like you know what's going on. Er, do you have an example on how to do this though or a link related to this?

This is my Kibana results: http://coriewhite.me/locallog.png


(Corie White) #6

I got it working! Thanks!


(system) #7