I have an application deployed on IBM K8. It logs on the console. Logs range from simple json to nested json.
I use the custom template with fields.yml for the fields in the ES-Kibana.
Here is my filebeat prospector:
- type: log
Here is my filebeat.yaml config
The default fields.yml that comes with filebeat, has a field "message" of type text.
It is this field which has my stringified json log.
Now, if i disable the decode json part in the processor, i get my logs under the message field in the stringified way.
But, if I apply the decode json for the field "message", it decodes it properly, but it creates all the dynamics fields from the decoded json.
How can i contain the decoded json as part of "message" field only, and not allow it to be split and hv the fields created for them.
Also, when i do the json decode on "message" i get mapper issues, saying that the message if of type "text" but upon decode, it gets an object.
Can i change the type of the "message" field as "object" instead of "text". Will this resolve my mapper issue?
Can there be a way to change the type of the field in the fields.yml from "text" to "object" through filebeats.yml config?