I am looking for a solution to do the following: I have a log file where a single log entry can contain multiple key/value pairs that I want to extract. The problem is, I want to extract all occurences in a single entry, not just the first one. (kinda like the "g" option in sed and vim)
For example, a log entry could look like this:
2018-03-28 14:23:56 something something user=foo something something more user=bar
It's trivial to write the grok filter that matches the first occurrence:
grok {
match => [ "logText", "[^A-Z^a-z^0-9]user=(?<user>[A-Za-z0-9]*)" ]
}
Interesting, but I'm not looking to extract all possible kv pairs. For instance, a log entry often contains a lot of "uninterestingvariable=something" entries that I do not want to store.
Also, I would like to find something more generic, and not just applicable to key/value pairs. For example, there could be something like IDs in the logfile that have a specific, and easily matched alphanumeric format.
Interesting, but I'm not looking to extract all possible kv pairs. For instance, a log entry often contains a lot of "uninterestingvariable=something" entries that I do not want to store.
The kv filter's include_keys option can deal with that.
Also, I would like to find something more generic, and not just applicable to key/value pairs. For example, there could be something like IDs in the logfile that have a specific, and easily matched alphanumeric format.
Short of a ruby filter I don't think there's a way of doing that with the standard filters.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.