Variable fieldname to be used as GREEDYDATA field

Elastic Stack Logstash
1

Hi,

I have the following logs:
2016-04-19 12:48:41,628 [DefaultPool] INFO DataXXXLog - some log data
2016-04-19 12:48:41,628 [DefaultPool] INFO DataYYYLog - some log data
2016-04-19 12:48:41,628 [DefaultPool] INFO WorkZZZLog - some log data

I'm trying to accomplish the below but to make it dynamic, instead of having to create match rules for the "Log Name" (4th field) I was hoping there was a simpler case that I could use that as a variable and pass that to GREEDYLOG?

This is what I have today, but if the 4th field name changes the line is not matched.

match => { "message" => "%{DATESTAMP:logdate} [%{WORD:thread}] %{LOGLEVEL:level} {1,2}DataXXXLog -\s+%{GREEDYDATA:DataXXXLog}" }

Thanks

2

Perhaps the NOTSPACE pattern would be useful? It matches any non-whitespace character, and the logger name probably doesn't have any spaces in it. If that isn't a safe assumption DATA would be better.

3

Thanks, I'm actually looking for that fieldname to be a dynamic name (the name of the Logger). Then reuse that dynamic name as the fieldname for the actual log info.
ie.

DataXXXLog = some log data
DataXXXLog = other loga data from same log fieldname
DataYYYLog = some other log data
WorkZZZLog = other log data

4

Why would you want to do that? It sounds like a really bad idea. The logger names are basically arbitrary strings which could lead to an explosion in mappings in your indexes. But sure, you could capture the logger name in one field and the message field in a generically named field and use a mutate filter to rename the message field. This probably works:

mutate {
  rename => ["message", "%{logger_name}"]
}
5

I was thinking it would be simpler to group certain types of logs together under the same field name. I could probably just alter my filter string to accomplish the same thing.

6

Exactly, just add a logger_name:DataXXXLog condition (or whatever) to any query.