I've been looking around and experimented a bit, but haven't gotten it to work or so to say. I have a tcp input with the json codec, along with an elasticsearch output with a template that adds some custom fields. I used to just use a grok-pattern to match the logger message to separate it to diffe…

Sure, you can combine a json/json_lines codec with a json filter if you have JSON-encoded data inside JSON-encoded data.

Ok nice! so if I had this for example in the JSON-encoded data from the codec: { "_index": "logstash-2018.02", "_type": "application", "_id": "AWG-1QHz2w4sPkeuw0Bb", "_version": 1, "_score": null, "_source": { "level": "INFO", "message": "{"userName":"test", "clientName":"test"}", "type":…

The message field doesn't contain a string with JSON in it, it contains an object. This is what a string containing JSON data would've looked like: "message": "{\"userName\":\"test\", \"clientName\":\"test\"}",

Thanks for the insight, seems to be filtering into an object now after with the json filter. Just need to separate it into fields for my purpose now. Thanks!

Hey Magnus, thanks for the pointers before. I am having trouble separating the json filter target into it's own fields now. I have used this setting: json { source => "message" target => "message" } It seems to parse fine when looking at the JSON "_source": { "level": "INFO", "message": …

It seems to parse fine when looking at the JSON Your example shows a message field containing an unparsed JSON string so it seems the json filter isn't doing what it's supposed to be doing. If you want the fields from the JSON string at the top level of the event just omit the target option. The…

Ok, I think you are probably onto something. I tried omitted the "target" option before and got a _jsonparsefailure tag before and didn't think too much about it, but it's possibly my logger message is still not correctly configured. I'll try some other things. Thanks for the tips.

Is it possible to use the json codec along with a json filter to further parse things into other fields?

Elastic Stack Logstash

Tieis February 26, 2018, 9:03pm 9

holy it worked. I was double json encoding my logger message because i thought there was a json codec and filter, but doesn't seem like i have to.

Thanks for the help!

Topic		Replies	Views
Json filter question Logstash	3	342	July 6, 2017
Difference between json codec in input {} vs filter {}? Logstash	8	13944	April 4, 2017
Json codec vs json filter in file input Logstash	4	1321	June 7, 2017
Filter individual json fields using json filters Logstash	17	1572	December 28, 2020
Parse json "message" into separate fields in Kibana Logstash	3	8572	February 15, 2019

Is it possible to use the json codec along with a json filter to further parse things into other fields?

Related topics