Modify field name based on the some condition like field"s value

Elastic Stack Beats
1

Hi,

This might be very simple or very tricky. Because I am not sure.

  • Spring boot with logstash logback encoder.
  • Filebeat to ES (Without Logstash )

Generates logs in JSON for our application which we are writing in JSON and third part lib and also logs in JSON but there message body is simple string whereas our application logs message body is also json.

So we can have "message" : "Simple stirng" // for third party logs
and for our case we will write "message" : our message as JSON structure

{
"timestamp": 2012.....
"message" : // Can be Json or simple string i.e Simple string but lib writes and json when we will write explicitly
}

Now before sending the logs to elastic search, I want to check if my message value is simple string or a json object (nested json object)

I am trying these two tricks to implement to fix it but unable to do it either of them.

  1. I want to change the name of the message to stacktrace. So that in elastic search there should not no collosion for message value for nested JSON or simple string.

  2. Or I am not that good in filebeat but if possible convert the value of message from string into JSON object like
    "message" : "messagevalue" // if value is String then only convert otherwise don't do anything
    "message" : {"stacktrace" : "message value"}

I tried regexp to match the processors in filebeat but somehow its not working. I don't why :frowning:
if possible please provide little code snippet of filebeat along with answer so that I can understand it better and don't trouble you guys.

Thanks for helping us : :+1:

2

If you want to modify events and do tricks like the above, Logstash is probably the tool you are looking for to put in between Filebeat and Elasticsearch.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.