Unable to see the Json data as fields in kibana

(Chandana Priya Sathavalli) #1

I have a sample log file with json content

{ LoanCode:99356, LoanDate:3/2/2018, LoanStatus:OPN}

Here is my logstash conf file

		path => "D:/elkstack/*.log"
		start_position => beginning

		match => { "message" => "%{GREEDYDATA:kvpairs}"  }		
			source => "kvpairs"

		hosts => "localhost:9200"
		index => "logstash-%{+YYY.MM.dd}"

I need to create visualization over the values in the log eg. how many loans are OPN .
I am unable to see these key value pairs in the kibana as fields.

Here is the screen shot of Kibana

(Christian Dahlqvist) #2

This doesn't make any sense as it just copies over the full content of the field and the content isn't valid JSON. Instead try something like this:

  match => { "message" => "{ %{GREEDYDATA:kvpairs}}"  }		

kv {
  source => "kvpairs"
  value_split => ":"
  field_split_pattern => ", "

(Chandana Priya Sathavalli) #3

Many thank @Christian_Dahlqvist !!

I tried with your suggestion, it almost worked but I see '}' in the field names.
Can you help me how to avoid them? Do I have to make any changes to the log file pattern?

(Christian Dahlqvist) #4

I modified the grok expression to remove those from the kvpairs string. It looks like you are still using your old pattern.

(Chandana Priya Sathavalli) #5

My Bad :frowning:
I was using my old pattern.

When I changed with your code, it worked as expected.

Thank you @Christian_Dahlqvist. You really made my day.

(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.