Manually checking if confs work properly

Elastic Stack Logstash
1

Is there a possible way to check if some new grok rule or filter-mutate works without waiting index rotation?
We added some new stuff and we wanna check out if the new index will work properly as expected.

2

Check what, exactly? You may find Logstash Filter Verifier useful.

3

You don't need to wait for an index to rotate to know if filters work, they are independant.

4

Thanks a lot for your replies.I am sorry for my delay to answer but i got stack with personal problems.

@magnusbaeck - What i mean is this.If i make a change on my logstash filters and then add on my output file a new index like this:

//filter:

  if "%" in [message]{
 	if [syslog_program] == "charon"{
	mutate{add_tag => ["IPSEC2"]}
}else{
	mutate{add_tag => ["Cisco"]}
}
  }

//output :

if "IPSEC2" in [tags] {
    elasticsearch {
       hosts => ["localhost:9200"]
       index => "ipsec2-%{+YYYY.MM.dd}"
    }
}

if "Cisco" in [tags] {
    elasticsearch {
        hosts => ["localhost:9200"]
        index => "cisco-%{+YYYY.MM.dd}"
    }
}

if i had these 2 inside a single "Cisco-xxxxxx" index that was already working for the day.Will the changes take place and a new index will be created for ipsec2 and split the outputs between the 2 the moment i restart logstash and elastic?Or do i need to wait for the new indeces to be created?

And by checking i mean is there a way to give a single log for input and capture the json that is going to be forwarded to elastic and see the tags,fields etc that are going to be added to it?

PS. I added the filter i commented above and expected the new index to be created and split the outputs.But i still get them on the same index :confused:

5

Will the changes take place and a new index will be created for ipsec2 and split the outputs between the 2 the moment i restart logstash and elastic?

Yes, but you don't have to restart ES.

Or do i need to wait for the new indeces to be created?

Indexes are created when they receive data.

And by checking i mean is there a way to give a single log for input and capture the json that is going to be forwarded to elastic and see the tags,fields etc that are going to be added to it?

You can use a stdout { codec => rubydebug } output to verify that events look as you expect them to. Testing the conditionals surrounding outputs is tricky.

1 Like