Selectively filtering messages log

Elastic Stack Logstash
1

I have come across with some issues in logging some data.

This log file contains various data.

2016-07-07 13:30:02 [Main] *** Program start ***
2016-07-07 13:30:02 [UnzipFile] Before file collection
2016-07-07 13:30:02 [GetZipCol] Start get sorted zip file collection
2016-07-07 13:30:02 [GetZipCol] End get sorted zip file collection
2016-07-07 13:30:02 [Main] [ERROR] No unzip file
2016-07-07 13:30:03 [Main] *** Program end ***

This is my output part in conf

if [Message] == "*** Program start ***" {
	elasticsearch { 
	hosts => ["localhost:9200"] 
	index => "log-%{+YYYY.MM.dd}"
	template => "C:/logstash/log.json"
	template_overwrite => true
	}	
}
if [Message] == "*** Program end ***" {
	elasticsearch { 
	hosts => ["localhost:9200"] 
	index => "log-%{+YYYY.MM.dd}"
	template => "C:/logstash/log.json"
	template_overwrite => true
	}	
}  
if [Level] =~ /.+/ {
	elasticsearch { 
	hosts => ["localhost:9200"] 
	index => "log-%{+YYYY.MM.dd}"
	template => "C:/logstash/log.json"
	template_overwrite => true
	}	
}

If I only want to grasp the event when the Program starts and ends and also the events with errors while the other events can be dropped. However, according to what I have written. I can only grasp the data with [Error]. How should I also grasp the other data? And will there be a simpler way of doing that instead of typing 3 if conditional statements?
Thanks.

2

You can do it in this way. In filter part of conf, use add tag to group the records you need

if [Progress] in ["*** Program start ***", "*** Program end ***"] or [Level] =~ /.+/ {
    	mutate {
    		add_tag => "send_to_es"
    	}
    }

Then in the output part, add this.

	if "send_to_es" in [tags] {
  		elasticsearch { 
		hosts => ["localhost:9200"] 
		index => "log-%{+YYYY.MM.dd}"
		template => "C:/logstash/log.json"
		template_overwrite => true
		}	
	}

This will only add those with the tags into ES. One thing yo should not miss is that you should add \r at the end of your matching message in your grok part.