Which Filter to use

Elastic Stack Logstash
1

My log is something like below

{@timestamp} some text here key1=value1 key2=value2 key3=value3

I have tried json filter as below and got parsing errors. Which filter should I choose here ? I want to push all key value pairs and timestamp to variables in ElasticSearch.

json {
		source => "message"
	}
2

As that log line is not JSON it is not surprising that the son filter does not work. I would recommend using a dissect or grok filter to parse the components of the string so that you end up with the final key-value list in a separate field. You can see how to go about doing this in this introductory blog post.

You can then apply a kv filter to this field to parse the key-value pairs out.

3

@Christian_Dahlqvist
I am using kv filter now as below. I have numbers as values in few of kv params, but in ElasticSearch they are being indexed as String's. How to index them as numbers !

      kv
		{ 
			source => "message" 
		}

I have response time in log as below .
ResponseTimeMillis=454
But when it's indexed to ES, it's indexed as below.

"ResponseTimeMillis": "454"

I want it to be "ResponseTimeMillis": 454. Is it possible ?

4

Yes, add a mutate filter to convert to a number after you have parsed the data.

5

Awesome. I'm now able to use mutate filter to convert string to a Integer.

Thank you

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.