Logstash conditional check if filed exist then replace timestamp value with another fileds timestamp

Hello All,

I'm stuck in how to implement conditional check in logstash and how would it be implemented correctly.
Usecase:I have data coming in my index with multiple fields value,I'd like to send data to elastic
where the timesatmp field value should be replaced with last_execution timestamp value,this is achieved ,and should only be done if jobStatus.jobId this field exist.

How to implement entirely sudo code:
if(jobid field exist)
then
replace timestamp filed value with last_execution timestamp.

Ruby partial code is done but not sure entirely.

ruby {
code => "event.set('@timestamp', event.get('last_execution_time'));"
}

config:

input {
   exec {
      command => '. ../scripts/ordermonitor/run_ordermonitor.sh'
      schedule => "0 0 * * * *"
   } 
}

filter {   
  
     ruby {
       code => "event.set('@timestamp', event.get('last_execution_time'));"
    }
  
  }

output {
   elasticsearch {
      hosts => "http://abc.com:9200"
	  ilm_pattern => "{now/d}-000001"
      ilm_rollover_alias => "tix-monitor-order"
	  ilm_policy => "tix-monitor-order-policy"
	  doc_as_upsert => true
	  document_id => "%{[order][recordUniqueId]}"
   } 
}

Assuming this would be part of ruby code,if yes then should it be part of filter plugin?
Below is partial implementation of config file,would like to achieve mentioned above.

Thanx

I do not think you need ruby to do that. Assuming that [last_execution_time] is already a LogStash::Timestamp object (if not, add a date filter to parse it):

if [jobStatus][jobId] {
    mutate { replace => { "@timestamp" => "%{last_execution_time}" } }
}

Hello @Badger ,

Thanx for your time to look into this,the below worked fine.

filter {   
   if [message] =~ "^\{.*\}[\s\S]*$" {
      json {
         source => "message"
         target => "parsed_json"
         remove_field => "message"
      }

      split {
         field => "[parsed_json][OrderMonitorReponse]"
         target => "order"
         remove_field => [ "parsed_json" ]
      }

        if [order][Job][lastStartTime] {
             mutate {
                convert => { "[order][Job][lastStartTime]" => "string" }
            }
            date {
              match => ["[order][Job][lastStartTime]", "yyyy-MM-dd'T'HH:mm:ssZ"]
              timezone => "UTC"
              target => "@timestamp"
            }
        }
    }

 

   else {
     drop { }
   }

Data from logs:
"lastStartTime":"2022-12-13T07:47:46Z"

Index template field:
"order.job.lastStartTime": {
"format": "yyyy-MM-dd'T'HH:mm:ssX",
"type": "date"
}

Many Thanx

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.