Hello,
I have successfully set up an Elastic cluster by referring to the documentation provided by Elastic. I have configured the Elastic Stack version 8.7 in a self-managed manner. The following resources were particularly helpful:
Docker documentation: Install Elasticsearch with Docker | Elasticsearch Guide [8.7] | Elastic
Secure connections in Fleet: Configure SSL/TLS for self-managed Fleet Servers | Fleet and Elastic Agent Guide [master] | Elastic
In addition, I have set up a Fleet server. As part of the configuration, I have enabled the "Kubernetes integration" feature available in the Kibana dashboard. This process generates a manifest file that needs to be applied to the Kubernetes (K8s) cluster.
Everything is working smoothly. I have configured JSON logging in our application, and the Kubernetes integration policy includes the following parse configuration:
- ndjson:
target: ""
add_error_key: true
message_key: message
expand_keys: true
The Elastic agent is successfully pushing JSON logs to the Elastic servers. However, there is an issue when a value for a specific key exceeds 1024 characters, as it is currently being discarded. In our logging, we often include a stack trace where the key "exception" may exceed 1024 characters.
I would like to know how to configure the Elastic agent policy or Fleet policy to ensure that stack traces or long key values are not discarded and can be effectively stored and indexed in Elastic.