Help to understand GROK fields in expression

[DHN71 (Hardy)]

Sample Data:

2018-05-22 15:34:58.068 UTC INFO name=StationRecord.unAssociatedGauge, value=32

The GROK expression needs to write everything after "name=" to a new field called "record". If I only use the expression:

(?(?<=name=).*unAssociatedGauge,)

Then the output is:

{
"record": "StationRecord.unAssociatedGauge,"
}

However, I want to also capture the timestamp, timezone, log level and value. I have tried using the following expression but an error is returned:

%{TIMESTAMP_ISO8601:timestamp} %{DATA:timezone} %{LOGLEVEL:level} (?(?<=name=).*unAssociatedGauge,) [value=]+%{NUMBER:value:int}

The expressions for timestamp, timezone and log level by themselves work. If I do not try and capture the data before the field, then the expression works. For example with the GROK expression:

(?(?<=name=).*unAssociatedGauge,) [value=]+%{NUMBER:value:int}

The returned data is:

{
"value": 32,
"record": "StationRecord.unAssociatedGauge,"
}

My questions are:

1. How can I insert the field expression in a GROK expression that would allow me to capture the data before the field and after it.
2. Why does the working example correctly process [value=]+%{NUMBER:value:int} when it comes AFTER the field?

[Badger]

@DHN71

%{TIMESTAMP_ISO8601:timestamp} %{DATA:timezone} %{LOGLEVEL:level} (?(?<=name=).*unAssociatedGauge,) [value=]+%{NUMBER:value:int}

Does this work for you? Using '[value=]+' looks really weird to me. I would just have used 'value='

grok { match => [ "message", "%{TIMESTAMP_ISO8601:timestamp} %{DATA:timezone} %{LOGLEVEL:level} name=(?<name>.*unAssociatedGauge), [value=]+%{NUMBER:value:int}" ] }

Did you consider grabbing everything after the LOGLEVEL with a GREEDYDATA and then using a kv filter?

[DHN71 (Hardy)]

Yes, the expression works for me. Without [value=]+ it doesn't work. My question is really why I can't use %{TIMESTAMP_ISO8601:timestamp} %{DATA:timezone} %{LOGLEVEL:level} followed by (?(?<=name=).*unAssociatedGauge,)

If I use only:

(?(?<=name=).*unAssociatedGauge,) [value=]+%{NUMBER:value:int}

Then the output is OK andI get the value and record correctly formatted.

[Badger]

It tells you right there in the log file if you try to use

%{TIMESTAMP_ISO8601:timestamp} %{DATA:timezone} %{LOGLEVEL:level} (?(?<=name=).*unAssociatedGauge,) [value=]+%{NUMBER:value:int}

RegexpError: unmatched close parenthesis: /(?<TIMESTAMP_ISO8601:timestamp>(?:(?>\d\d){1,2})-(?:(?:0?[1-9]|1[0-2]))-(?:(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))[T ](?:(?:2[0123]|[01]?[0-9])):?(?:(?:[0-5][0-9]))(?::?(?:(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?)))?(?:(?:Z|[+-](?:(?:2[0123]|[01]?[0-9]))(?::?(?:(?:[0-5][0-9])))))?) (?<DATA:timezone>.*?) (?<LOGLEVEL:level>([Aa]lert|ALERT|[Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nfo|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rr?(?:or)?|ERR?(?:OR)?|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?)) (?(?<=name=).*unAssociatedGauge,) [value=]+(?<NUMBER:value:int>(?:(?:(?<![0-9.+-])(?>[+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+))))))/m>,