Make extracted field visible to all events

Elastic Stack Logstash
1

Hello, I've the following filter configurations

filter{
    if "Dumping Scrapy stats" in [message] {

    		grok{
    				match => [ "message", "'item_scraped_count': %{NUMBER:scraped:int}" ]
    		}
    		grok{
    				match => [ "message", "'invalid_items_count': %{NUMBER:invalid:int}" ]
    		}
    	
    	}

    	 else {
    		grok{
    				match => [ "message", "Crawled iteration for merchant %{WORD:merchant_name} started" ]
    			}
    		}
}
output {
    	if "_grokparsefailure" not in [tags]{
    		stdout {
    			codec => rubydebug
    		}

    		elasticsearch {
    			hosts => "elasticsearch:9200"
    			index => "logstash-%{merchant_name}"
    		}
    	}
    }

The log line that triggers the grok in the else part, comes before the one that triggeres the if part in the logfile, so after parsing the logfile, I get 2 elasticsearch indices logstash-adidas & logstash-%{merchant_name}, I want the merchant_name field to be (global?) once it is extracted.

How can I solve this problem?

2

Take a look at this and see if that helps/solves your problem

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.