Issue with message field while parsing JSON


I am parsing jboss fuse container logs ,which are JSON messags,following a sample JSON message.

{"@timestamp":"2019-06-25T08:34:36.517Z","source_host":"sitfabr02.local","level":"INFO","thread_name":"qtp1821745282-22143","@version":1,"logger_name":"","message": "{"released":true}","mdc":{"camel.messageId":"ID--local-36712-1561427486681-16-239094","portTypeName":"RestClient","headers":"{X-Client-Transaction-Id=1098.69747568.0, X-Client-Id=PS, Accept=application/json, breadcrumbId=topic_com.AL.ChangedV1__com..InventoryChangedV1_null, X-POS-Id=DigitalWeb, Content-Type=application/json}","exchangeId":"349f6de5-58f1-46ce-ba7f-b2269c5b5ca7","x-client-transaction-id":"1098.69747568.0","fps.service-id":"SVC_FPS_00022","messageId":"be67a43d-cc5c-40ab-ba25-35eca640f920","content-type":"application/json","portName":"SearchBaseFaresV1RestClient","":899,"fps.client-ip":"","bundle.version":"3.1.5.redhat-630371"}}

Filebeat and Logstash configs are standard.

json.keys_under_root: true
json.overwrite_keys: true
json.add_error_key: true

input {
beats {
port => 6001
codec => json
client_inactivity_timeout => 2400
I didn't had any filter section and standard output to Elastic.

All versions used are 6.2(beats,logstash and elastic)

The issue i am facing is these is no message field coming in the event even though we had it in JSON instead the message field getting tread as JSON itself and adding a field "released" with value "true".

How can we keep the message field intact and keep as is.

As these are fuse framework logs,my development team don't want to change their framework.

any help will be appreciated,thanks.

You have told filebeat to parse the json and add fields. That will result in it sending a [message] field that contains the string '{"released":true}'.

Then you have told the beats input to parse the [message] it receives as JSON, which results in it creating a [released] field. Remove the codec.

1 Like

Thanks Badger.Worked perfect.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.