KV filter is unable to handle = within a key value

Elastic Stack Logstash
1

When I have the KV filter run against this string

last="3367" post="" ltime="3"

Instead of ending up with

last: 3367
post: " ltime="3

I end up with

last: 3367
post: \
ltime: \"3"

I am expecting the KV filter to accept \ as an escape characters so that the value is encapsulated by the non escaped quotation marks but it seems that the kv seperator process kicks in before the kv value is read completely.

2

It looks like a few characters were trimmed from your log example. Format it as preformatted text to avoid this.

3 
last="3367" post="\" ltime=\"3"

There, this is the original string

4

This is normal behaviour like in Java (String.split(" ")) or PHP (explode(" ", String)).

So you split your string by " " and after that in this parts you split it again on "=" to determine the key and the value part.
It just trimmed your " away because you have include_brackets enabled.
KV Man - Include brackets

This is how I understand this, the developer of KV can have other views on this.

5

Aha

So Logstash keeps the brackets and Elasticsearch takes them away so I never see them. This also explains that when there is nothing in the field Logstash sends it as key="" and in this case Elasticsearch does not strip them away and stores the value as ""

Sometimes the Logstash documentation really needs more examples.

6

Logstash aka KV filter takes the brackets away.
But this option is confusing. Because if include_brackets it's enabled it
will trim the brackets away.

I was wrong with the part of trimming the " away.
Because include_brackets only trim away brackets ()[]{}...

I'm wondering if with brackets option enabled you could split by " " and preserve the string as one value.
I've wrote my own splitter in ruby, so I won't test this.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.