Update a field if it does not contain any value

Elastic Stack Logstash
1

I have log files generated from some applications. Some of applications produces messages in following format. So there is "logContent" in message field.

{"instant":{"epochSecond":1628692763,"nanoOfSecond":792000000},"thread":"AWT-EventQueue-0","level":"INFO","loggerName":"com.client.logon.form.Logon","message":"errortype: SECURITY logContent:User Log on user 1","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":23,"threadPriority":6,"@timestamp":"2021-08-11T17:39:01.025+0300"}

some of applications produces logs in following format. There is no logContent text in message field

{"@timestamp":"2021-08-16T18:16:23.925+03:00","sequence":3042,"loggerClassName":"org.jboss.logmanager.Logger","loggerName":"stdout","level":"INFO","message":"\t\tCLIENT_UNIQUE_ID:e9d58a5e-67b8-4a11-88b8-71067cea4111","threadName":"EJB default - 1","threadId":357,"mdc":{},"ndc":"","hostName":"evt06001nb","processName":"jboss-modules.jar","processId":18888,"@version":"1"}

I am using Kibana for viewing logs. I want to show a single field in Dashboard. Otherwise it will be complicated for users to look at "logContent" and "message" fields at the same time. So I want to add a check to logstash that if logContent does not have any value , update its value to reflect message field. In discover, I will use "logContent" field.
I implemented following if condition . But it did not work

filter{
    json{
        source => "message"

    }
	grok {
    match => {
        "message" => [
            "errortype:%{GREEDYDATA:errorType} logContent:%{GREEDYDATA:logContent}",
            "logContent:%{GREEDYDATA:logContent}"
        ]
    }
	 
  }
  if ![logContent] {
		mutate {
			update => { "logContent" => "%{[message][message]}" }
    } 
	}

}```
2

What do you expect that to do?

3

for the following json log, I expect to assign "\t\tCLIENT_UNIQUE_ID:e9d58a5e-67b8-4a11-88b8-71067cea4111" for logContent. It is stored in "message" attribute of json

{"@timestamp":"2021-08-16T18:16:23.925+03:00","sequence":3042,"loggerClassName":"org.jboss.logmanager.Logger","loggerName":"stdout","level":"INFO","message":"\t\tCLIENT_UNIQUE_ID:e9d58a5e-67b8-4a11-88b8-71067cea4111","threadName":"EJB default - 1","threadId":357,"mdc":{},"ndc":"","hostName":"evt06001nb","processName":"jboss-modules.jar","processId":18888,"@version":"1"}
4

Yes, and if you parse that JSON using a json filter then the [message] field will get overwritten

            "ndc" => "",
        "message" => "\t\tCLIENT_UNIQUE_ID:e9d58a5e-67b8-4a11-88b8-71067cea4111",
          "level" => "INFO",
     "threadName" => "EJB default - 1"

etc.

5

So you mean following should work. But it does not. Do you think if condition is problem? Does it mean that if logContent is empty?

if ![logContent] {
		mutate {
			update => { "logContent" => "message" }
    } 
	}
6

mutate+update does nothing if a field does not exist. If you want to set logContent to the value of [message] when logContent is empty then use

if ! [logContent] { mutate { add_field => { "logContent" => "%{message}" } } }
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.