How to avoid creating the dynamic fields in Kibana upon json decode of a specific field in the filebeat processor?

[rahul_mohan]

Hello,

I have an application deployed on IBM K8. It logs on the console. Logs range from simple json to nested json.

I use the custom template with fields.yml for the fields in the ES-Kibana.

Here is my filebeat prospector:

.....

• type: log
  paths:
  - /var/lib/docker/containers//.log
  - /var/data/kubeletlogs///*.log#
  json.message_key: log
  #json.keys_under_root: true
  logging.files.keepfiles: 7
  #fields_under_root: true
  ....

Here is my filebeat.yaml config

.....
processors:
- add_kubernetes_metadata:
in_cluster: true
- decode_json_fields:
fields: ["message"]
process_array: true
max_depth: 1

output.elasticsearch:
enabled: true
hosts: ["host"]
protocol: "https"
username: "username"
password: "password"
#ssl.enabled: true
ssl.certificate_authorities: ["/usr/share/filebeat/certificate.pem"]
#ssl.certificate: "/usr/share/filebeat/certificate.pem"
index: "my-log-%{+YYYY.MM.dd}"
setup.template:
name: "my-log"
pattern: "my-log-*"
overwrite: false
enabled: false
.....

The default fields.yml that comes with filebeat, has a field "message" of type text.
It is this field which has my stringified json log.

Now, if i disable the decode json part in the processor, i get my logs under the message field in the stringified way.

But, if I apply the decode json for the field "message", it decodes it properly, but it creates all the dynamics fields from the decoded json.

How can i contain the decoded json as part of "message" field only, and not allow it to be split and hv the fields created for them.

Also, when i do the json decode on "message" i get mapper issues, saying that the message if of type "text" but upon decode, it gets an object.

Can i change the type of the "message" field as "object" instead of "text". Will this resolve my mapper issue?
Can there be a way to change the type of the field in the fields.yml from "text" to "object" through filebeats.yml config?