Get today's date that change automatically everyday

Elastic Stack Logstash
1

I am trying to add a new field, "Today's Date" to get the Today's/Current date from logstash into elasticsearch.

Here is my configuration in logstash

input {
  file {
    path => "C:/Users/Admin/Documents/dummy_test_date.csv"
    start_position => "beginning"
    sincedb_path => "NUL"
  }
}
filter {
  	csv {
    	separator => ","
    	columns => ["ID","Challenge Statements","Test Cases","Start Date","End Date]
    	
    	add_field => {
    		"Today's Date" => "%{+dd/MM/yyyy}"	 
    	}
  	}

  	date {
        match => ["Today's Date", "dd/MM/yyyy"]
        target => "Today's Date"
        timezone => "Asia/Kuala_Lumpur"
      }

	date {
    	match => ["Start Date", "dd/MM/yyyy"]
    	target => "Start Date"
    	timezone => "Asia/Kuala_Lumpur"
 	}

 	date {
    	match => ["End Date", "dd/MM/yyyy"]
    	target => "End Date"
    	timezone => "Asia/Kuala_Lumpur"
  	}

}


output {
   elasticsearch {
     hosts => "http://127.0.0.1:9200"
     index => ["dummydata-1"]
     document_id => "%{[ID]}"
     user => "elastic"
     password => "password"
  }
stdout {}
}

However with this configuration, the date in "Today's Date" field did not change automatically to the next day date in elasticsearch & kibana.

Would appreciate anyone could tell me what I am doing wrong in this config? Thank you!

2

Anyone?

3

if you're passing today's date, why would you expect next day's date?

also elastic shows @timestamp in UTC. since you're on UTC+8 date, is it possible that you're looking at UTC timestamp?

4

Thanks for the reply @ptamba! I am tracking a project progress based on the start date, end date and the current date. Hence, I want the current date to change automatically to Today's/Current date in Elasticsearch.

Something like date.now is JavaScript or =Today() in Microsoft excel

5

this code here add a new field called "Today's Date" into your document before sending to elasticsearch. I'm not sure what else is it that you expect.

6

Yes, you are right about this. However, this code only takes today's date.

For example, if I import my CSV file from logstash to elasticsearch today, in elasticsearch it will show today's date. But if I open up elasticsearch tomorrow, this code does not change to tomorrow's date, it will still give today's date instead.

Thus, I want the date to change to the current date as the day goes if this makes sense?

Apologies for the confusion.

7

so you want to modify the value of existing document stored in elasticsearch? elasticsearch topic would be a better forum to ask about it.

you could do it in logstash with elasticsearch filter. basically you search for documents that has “Today’s date” field then update the “Today’s date” value using current date value. you can run it with a daily cron at the start of day.

8

Sure, I will ask this in Elasticsearch topic instead.

Do you think I could create a ruby filter in logstash to achieve this? Nonetheless, I will definitely try the given options. Thank you!

9

doing it with ruby filter will achieve the same result.
from what you're saying, you're trying to modify existing documents that are already stored within elasticsearch.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.