Regex in conditionals

[timmy8ken (Tim Aitken)]

Hi All,

I have log files that I need to parse that have multiple and varying lines, I have managed to write a number of grok queries that are successfully matching all lines in the log, all similar this:

	    if !("parsed" in [tags]) {
			grok {
				match => { "message" => "#%{BASE10NUM:EventID}\|%{TIMESTAMP_ISO8601:eventTimestamp}\|\|\((ProductID:)%{BASE10NUM:ProductID}\)%{SPACE}%{GREEDYDATA:tempMessage}"}
				add_field => {
					"tags" => "parsed"
				}
				tag_on_failure => []
			}
		}

However I would like the tag field to contain which grok filter is catching the log entry, so I have updated to something like this:

	    if [tags] !~ /parsed[A-Za-z0-9_]+/ {
			grok {
				match => { "message" => "#%{BASE10NUM:EventID}\|%{TIMESTAMP_ISO8601:eventTimestamp}\|\|\((ProductID:)%{BASE10NUM:ProductID}\)%{SPACE}%{GREEDYDATA:tempMessage}"}
				add_field => {
					"tags" => "parsed_wp_log_1"
				}
				tag_on_failure => []
			}
		}

unfortunately I can't get the regex in the conditional to work and multiple grok filters are applying which is ultimately breaking the results. As we can see by all the tags applied to one of the parsed log entries:

tags beats_input_codec_plain_applied, parsed_log, parsed_wp_log_5, _dateparsefailure

Is there something I have missed here, and also, am I approaching this problem in the right way?

Thanks,

Tim

[magnusbaeck (Magnus Bäck)]

I don't think you can do regexp matches against tags. Why not just tag with parsed and parsed_wp_log_5?

[timmy8ken (Tim Aitken)]

That worked perfectly, thanks @Magnusbaeck

For anyone looking, this is how I made it work:

		if !("parsed" in [tags]) {
			grok {
				match => { "message" => "#%{BASE10NUM:EventID}\|%{TIMESTAMP_ISO8601:eventTimestamp}\|\|(?<PostID>\w*\s#\d*(?= ))%{SPACE}%{NOTSPACE}%{SPACE}\(\w*:%{BASE10NUM:PID}\)%{SPACE}%{GREEDYDATA:tempMessage}"}
				add_field => {
					"tags" => ["parsed","parsed_wp_log_4"]
				}
				tag_on_failure => []
				
			}
		}