Logstash conf not working for json format

Elastic Stack Logstash
1

Hi Marcus,

I have tried the json plugin but it is not working plz help me

{"_id":{"$oid":"592fdb630b2262682c85351d"},"Date":"20170510","url":"Loading of Page /start","Device":"Mobile","Browser":"UC Browser mobile","BrowserVersion":"UC Browser mobile 11.3","OS":"Android","OSVesrion":"Android 6.0 Marshmallow","ErrorCount":1.0,"SuccessCount":27.0,"TotalHit":28.0,"ErrorPerc":3.5714285714285716,"SuccessPerc":96.42857142857143}

i have provided the sample of my json.
and below is my conf file

input {
file {
type => "json"
path => "D:\QK\Jalpesh\conf\ganesh.json"
codec=> "json"
}
}
filter{
json{
source => "message"
}
}

output {

elasticsearch {
hosts => "localhost"
index => "index99"
}
}

It is loading default template

2

Use a filter like this:

filter {    	
	json {
		source => "message"
		target => "__json"
	}	    	
}

I'm not sure that the json codec is needed in the input ( did not use it in my test ) and you should get something like:-

{
       "message" => "{\"_id\":{\"$oid\":\"592fdb630b2262682c85351d\"},\"Date\":\"20170510\",\"url\":\"Loading of Page /start\",\"Device\":\"Mobile\",\"Browser\":\"UC Browser mobile\",\"BrowserVersion\":\"UC Browser mobile 11.3\",\"OS\":\"Android\",\"OSVesrion\":\"Android 6.0 Marshmallow\",\"ErrorCount\":1.0,\"SuccessCount\":27.0,\"TotalHit\":28.0,\"ErrorPerc\":3.5714285714285716,\"SuccessPerc\":96.42857142857143}",
      "@version" => "1",
    "@timestamp" => "2017-06-20T14:23:32.300Z",
          "path" => "/Users/kryten/Documents/Utilities/logstash-2.3.1/source2.log",
          "host" => "kryten-mbp",
        "__json" => {
                   "_id" => {
            "$oid" => "592fdb630b2262682c85351d"
        },
                  "Date" => "20170510",
                   "url" => "Loading of Page /start",
                "Device" => "Mobile",
               "Browser" => "UC Browser mobile",
        "BrowserVersion" => "UC Browser mobile 11.3",
                    "OS" => "Android",
             "OSVesrion" => "Android 6.0 Marshmallow",
            "ErrorCount" => 1.0,
          "SuccessCount" => 27.0,
              "TotalHit" => 28.0,
             "ErrorPerc" => 3.5714285714285716,
           "SuccessPerc" => 96.42857142857143
    }
}

With all your json fields under "__json" ready for use.

3

Use JSON Editor for Edit, validate and Format JSON data.

4

I have tried the json plugin but it is not working plz help me

If you tell us in what way it doesn't work it'll be easier to help out.

5

Thank u for the reply .The problem is it is not loading the data from the file ,it is loading the default template

6

The file format i am having is .json while loading the data through logstash ,it doesnt throw error but loading default template and the index is not seen in the kibana.

7

Logstash is probably tailing the input file. Please check out the documentation and previous posts to see what's said about the sincedb_path and start_position configuration options.

8

Hi Marcus,

I have deleted the Sincedb files and also set the sincedb option to "NUL". But still it is loading the default template.

9

I have deleted the Sincedb files and also set the sincedb option to "NUL".

What about start_position?

But still it is loading the default template.

Don't get distracted by that log message.

Comment out the elasticsearch output and use a stdout { codec => rubydebug } output while debugging.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.