I have got a question concerning the KV filter.
Is it possible to use an regexp to get only the key matching the regex ? ?
For example, if the message is :
t_1=qsdfgh t_2=ploki p_1=nbvcx t_3=azerty
I would like to use the KV filter only to retrieve t_1=qsdfgh, t_2=ploki and t_3=azerty.
So my idea is to use a regexp. something like this for example [t.*]. That way, the p_1 isn't collected.
I saw the option default_keys and include_keys, but i'm not sure there are usefull.
Do you have a wide variety of numbers? For example would you have T_1 all the way up to T_900? Or is it just 1-3?
Include_keys is useful if you know exactly which keys you want. For your example above you would just include T_1, T_2, and T_3. Anything other than those three would be ignored. However this only works if you have a small number of fields and you know exactly what they are named.
Hi, Thank for your answer.
It was just an example. In fact there is a lot of fields, all begin with tk_ .
But I can't know which one will be in the log. That why I would like to use a regexp to match all field begin with tk_.
It seems that the prune plugin can make the job.
Does anybody already use this filter ?
I try to use the whitelist_names option with this regex :
whitelist_names => ["tk_\w*"]
but nothing goes into Elasticsearch. And of course nothing appears in the logstash log file.
Hoping someone could help me