Logstash kv and regex

Elastic Stack Logstash
1

Hi,

I have got a question concerning the KV filter.
Is it possible to use an regexp to get only the key matching the regex ? ?

For example, if the message is :

t_1=qsdfgh t_2=ploki p_1=nbvcx t_3=azerty

I would like to use the KV filter only to retrieve t_1=qsdfgh, t_2=ploki and t_3=azerty.
So my idea is to use a regexp. something like this for example [t.*]. That way, the p_1 isn't collected.

I saw the option default_keys and include_keys, but i'm not sure there are usefull.

Regards

2

Do you have a wide variety of numbers? For example would you have T_1 all the way up to T_900? Or is it just 1-3?

Include_keys is useful if you know exactly which keys you want. For your example above you would just include T_1, T_2, and T_3. Anything other than those three would be ignored. However this only works if you have a small number of fields and you know exactly what they are named.

3

Hi, Thank for your answer.

It was just an example. In fact there is a lot of fields, all begin with tk_ .
But I can't know which one will be in the log. That why I would like to use a regexp to match all field begin with tk_.

:stuck_out_tongue:

4

It seems that the prune plugin can make the job.

5

Does anybody already use this filter ?

I try to use the whitelist_names option with this regex :

prune{
whitelist_names => ["tk_\w*"]
}

but nothing goes into Elasticsearch. And of course nothing appears in the logstash log file.

Hoping someone could help me :stuck_out_tongue: