# Adding 1 day to the date

**URL:** <https://discuss.elastic.co/t/adding-1-day-to-the-date/129168>\
**Category:** Logstash\
**Created:** [April 23, 2018, 7:24pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168 "2018-04-23T19:24:53Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![HRG](https://avatars.discourse-cdn.com/v4/letter/h/a4c791/32.png) [@HRG](https://discuss.elastic.co/u/HRG)\
**Post date:** [April 25, 2018, 2:40pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/21 "2018-04-25T14:40:18Z")

</div>

#Date comparision  
date  
{  
if "LogEventTimeStamp \>= "BusinesDateTimeConverted" {  
mutate {  
add\_field =\> {  
"LateFileStatus" =\> "LateArrival"  
}   
}   
} else {  
mutate {  
add\_field =\> {  
"LateFileStatus" =\> "On-Time"  
}   
}   
}   
}  
#End of Date Comparision

---

<div class="post-metadata">

**Author:** ![HRG](https://avatars.discourse-cdn.com/v4/letter/h/a4c791/32.png) [@HRG](https://discuss.elastic.co/u/HRG)\
**Post date:** [April 25, 2018, 2:41pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/22 "2018-04-25T14:41:12Z")

</div>

here is the complete code

filter {

```
#Match patterns based on source log origin

if [type] == "xxx" {		
	grok {
	match => ["message", "%{TIMESTAMP_ISO8601:LogEventTimeStamp}%{SPACE}%{LOGLEVEL:EventStatus}%{SPACE}%{INT:EventId}%{SPACE}%{NOTSPACE:dotchars}%{SPACE}%{NOTSPACE:SourceLogModule}%{SPACE}%{NOTSPACE:ProcessName}%{SPACE}%{NOTSPACE}%{SPACE}Get response data to file%{SPACE}%{GREEDYDATA:FilePath}/%{GREEDYDATA:file}"]
	overwrite => ["message"]
	}      

    if "_grokparsefailure" in [tags] {		
		mutate {				
			remove_tag => ["_grokparsefailure"]						
		}				
	}
}			
# End of xxx

#Extract Business date from file

grok {
	match => ["file", "%{WORD}.%{WORD:BusinessDate}.%{GREEDYDATA:FileLastPart}"]		
}      

if "_grokparsefailure" in [tags] {		
	mutate {				
			remove_tag => ["_grokparsefailure"]		
	}				
}

#Parse Business date
	
grok {
	match => ["BusinessDate", "%{YEAR:year}%{MONTHNUM:month}%{MONTHDAY:day}"]
}      

if "_grokparsefailure" in [tags] {		
	mutate {				
			remove_tag => ["_grokparsefailure"]					
	}				
}

# Set Timestamp

mutate {
	add_field => {
		ExpectedTime => "03:00:00.000"
	}
}

mutate {
	add_field => {
		BusinesDateTime => "%{year}-%{month}-%{day} %{ExpectedTime}"
	}
}
#Not required ExptectedTime in output
mutate {
	remove_field => [ExpectedTime]
}

#Convert to Date
date
{
	match => ["BusinesDateTime", "YYYY-MM-DD HH:mm:ss.SSS"]		
	target => "BusinesDateTime"
}

#Adding one day to BusinessDateTime

ruby {
	code => 'event.set("BusinesDateTimeConverted", LogStash::Timestamp.new(Time.at(event.get("BusinesDateTime").to_f+86400)))'				
}
	
#Remove Milliseconds	
mutate {
	gsub => ["LogEventTimeStamp", "\.\d{3}$", ""]
}

#Setting up Log timestamp to LogEventTimeStamp for the given formats.
date
{
	match => ["BusinesDateTimeConverted", "yyyy-MM-dd HH:mm:ss", "MMM dd, yyyy HH:mm:ss", "yyyy-MM-dd HH:mm:ss,SSS", "yyyy-MM-dd HH:mm:ss.SSS"]		
	target => "BusinesDateTimeConverted"		
}

#Setting up Log timestamp to LogEventTimeStamp for the given formats.
date
{
	match => ["LogEventTimeStamp", "yyyy-MM-dd HH:mm:ss", "MMM dd, yyyy HH:mm:ss", "yyyy-MM-dd HH:mm:ss,SSS", "yyyy-MM-dd HH:mm:ss.SSS"]		
	target => "LogEventTimeStamp"		
}

```

#Date comparision  
date  
{  
if "LogEventTimeStamp \>= "BusinesDateTimeConverted" {  
mutate {  
add\_field =\> {  
"LateFileStatus" =\> "LateArrival"  
}   
}   
} else {  
mutate {  
add\_field =\> {  
"LateFileStatus" =\> "On-Time"  
}   
}   
}   
}  
#End of Date Comparision

}

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [April 25, 2018, 3:02pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/23 "2018-04-25T15:02:33Z")

</div>

Aaaaaah. I should have seen it earlier. You have an if condition in a filter. That is not possible. Besides the date filter doesn't make any sense there.

---

<div class="post-metadata">

**Author:** ![HRG](https://avatars.discourse-cdn.com/v4/letter/h/a4c791/32.png) [@HRG](https://discuss.elastic.co/u/HRG)\
**Post date:** [April 25, 2018, 3:23pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/24 "2018-04-25T15:23:20Z")

</div>

Where should i keep the "if" condition". Should i write outside of " date filter" plugin?

---

<div class="post-metadata">

**Author:** ![HRG](https://avatars.discourse-cdn.com/v4/letter/h/a4c791/32.png) [@HRG](https://discuss.elastic.co/u/HRG)\
**Post date:** [April 25, 2018, 3:25pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/25 "2018-04-25T15:25:46Z")

</div>

If i remove below Date filter, the code is running as expected. Can't we have "if" condition inside date filter?

#Date comparision  
date  
{  
if "LogEventTimeStamp \>= "BusinesDateTimeConverted" {  
mutate {  
add\_field =\> {  
"LateFileStatus" =\> "LateArrival"  
}   
}   
} else {  
mutate {  
add\_field =\> {  
"LateFileStatus" =\> "On-Time"  
}   
}   
}   
}  
#End of Date Comparision

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [April 25, 2018, 3:42pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/26 "2018-04-25T15:42:49Z")

</div>

As I said, conditions must be outside of filters.

---

<div class="post-metadata">

**Author:** ![HRG](https://avatars.discourse-cdn.com/v4/letter/h/a4c791/32.png) [@HRG](https://discuss.elastic.co/u/HRG)\
**Post date:** [April 25, 2018, 3:51pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/27 "2018-04-25T15:51:16Z")

</div>

umm....somehow outside it does not like look like. It expecting some filter

[2018-04-25T11:48:27,347][ERROR][logstash.agent] Failed to execute action {:action=\>LogS  
or", :message=\>"Expected one of #, input, filter, output at line 116, column 5 (byte 2652) after ",

**Code**

filter {

```
#Match patterns based on source log origin

if [type] == "xxx" {		
	grok {
	match => ["message", "%{TIMESTAMP_ISO8601:LogEventTimeStamp}%{SPACE}%{LOGLEVEL:EventStatus}%{SPACE}%{INT:EventId}%{SPACE}%{NOTSPACE:dotchars}%{SPACE}%{NOTSPACE:SourceLogModule}%{SPACE}%{NOTSPACE:ProcessName}%{SPACE}%{NOTSPACE}%{SPACE}Get response data to file%{SPACE}%{GREEDYDATA:FilePath}/%{GREEDYDATA:file}"]
	overwrite => ["message"]
	}      

    if "_grokparsefailure" in [tags] {		
		mutate {				
			remove_tag => ["_grokparsefailure"]						
		}				
	}
}			
# End of xxx	

#Extract Business date from file

grok {
	match => ["file", "%{WORD}.%{WORD:BusinessDate}.%{GREEDYDATA:FileLastPart}"]		
}      

if "_grokparsefailure" in [tags] {		
	mutate {				
			remove_tag => ["_grokparsefailure"]		
	}				
}

#Parse Business date
	
grok {
	match => ["BusinessDate", "%{YEAR:year}%{MONTHNUM:month}%{MONTHDAY:day}"]
}      

if "_grokparsefailure" in [tags] {		
	mutate {				
			remove_tag => ["_grokparsefailure"]					
	}				
}

# Set Timestamp

mutate {
	add_field => {
		ExpectedTime => "03:00:00.000"
	}
}

mutate {
	add_field => {
		BusinesDateTime => "%{year}-%{month}-%{day} %{ExpectedTime}"
	}
}
#Not required ExptectedTime in output
mutate {
	remove_field => [ExpectedTime]
}

#Convert to Date
date
{
	match => ["BusinesDateTime", "YYYY-MM-DD HH:mm:ss.SSS"]		
	target => "BusinesDateTime"
}

#Adding one day to BusinessDateTime

ruby {
	code => 'event.set("BusinesDateTimeConverted", LogStash::Timestamp.new(Time.at(event.get("BusinesDateTime").to_f+86400)))'				
}
	
#Remove Milliseconds	
mutate {
	gsub => ["LogEventTimeStamp", "\.\d{3}$", ""]
}

#Setting up Log timestamp to LogEventTimeStamp for the given formats.
date
{
	match => ["BusinesDateTimeConverted", "yyyy-MM-dd HH:mm:ss", "MMM dd, yyyy HH:mm:ss", "yyyy-MM-dd HH:mm:ss,SSS", "yyyy-MM-dd HH:mm:ss.SSS"]		
	target => "BusinesDateTimeConverted"		
}

#Setting up Log timestamp to LogEventTimeStamp for the given formats.
date
{
	match => ["LogEventTimeStamp", "yyyy-MM-dd HH:mm:ss", "MMM dd, yyyy HH:mm:ss", "yyyy-MM-dd HH:mm:ss,SSS", "yyyy-MM-dd HH:mm:ss.SSS"]		
	target => "LogEventTimeStamp"		
}

```

} #End of Main Filter

#Date Comparision

```
if "LogEventTimeStamp" >= "BusinesDateTimeConverted" {
	mutate {
		add_field => {
			"LateFileStatus" => "LateArrival"
	    }	
	}	
} else {
	mutate {
		add_field => {
			"LateFileStatus" => "On-Time"
	    }	
	} 	
}	

```

#End of Date Comparision

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [April 25, 2018, 3:54pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/28 "2018-04-25T15:54:32Z")

</div>

Outside of filter objects, not the complete filter{} block

---

<div class="post-metadata">

**Author:** ![HRG](https://avatars.discourse-cdn.com/v4/letter/h/a4c791/32.png) [@HRG](https://discuss.elastic.co/u/HRG)\
**Post date:** [April 25, 2018, 3:55pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/29 "2018-04-25T15:55:36Z")

</div>

Would you please mind correct the code i sent?. It looks i am missing something.

---

<div class="post-metadata">

**Author:** ![HRG](https://avatars.discourse-cdn.com/v4/letter/h/a4c791/32.png) [@HRG](https://discuss.elastic.co/u/HRG)\
**Post date:** [April 25, 2018, 3:58pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/30 "2018-04-25T15:58:07Z")

</div>

Got it. Worked.

---

<div class="post-metadata">

**Author:** ![HRG](https://avatars.discourse-cdn.com/v4/letter/h/a4c791/32.png) [@HRG](https://discuss.elastic.co/u/HRG)\
**Post date:** [April 25, 2018, 4:01pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/31 "2018-04-25T16:01:25Z")

</div>

The "if" condition worked. But the date comparision is not giving correct result. It looks the dates are not in the same format.

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [April 25, 2018, 4:03pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/32 "2018-04-25T16:03:32Z")

</div>

The structure is always  
--- input/filter/output  
------ conditions  
---------inputs/filters/outputs

You can nest multiple conditions. But you can't put anything inside of mutate{}, date{}, etc, but their own options (add\_field =\>..., match =\>..., etc)

Did you make the changes to the condition I had suggested?

---

<div class="post-metadata">

**Author:** ![HRG](https://avatars.discourse-cdn.com/v4/letter/h/a4c791/32.png) [@HRG](https://discuss.elastic.co/u/HRG)\
**Post date:** [April 25, 2018, 4:07pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/33 "2018-04-25T16:07:57Z")

</div>

Yeah..i made the change. I moved "if" condition inside "filter" plug-in. It worked. Thank you for the solution.

Somehow , the date comparision result is giving wrong. It my case "LogEventTimeStamp" is smaller than "BusinesDateTimeConverted". The expected outcome is "on-time" but it is giving "LateArrival"

In the Kibana, the dates are showing as below

BusinesDateTimeConverted: 2018-01-20T08:00:00.000Z (This is logstash date format)

LogEventTimeStamp : April 19th 2018, 13:15:09.000

I am comparing both of the above two dates. The result should be "on-Time" but it showing "LateArrival"

Anything format issue?

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [April 25, 2018, 4:15pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/34 "2018-04-25T16:15:56Z")

</div>

I'm a bit irritated that Kibana doesn't display both of them in the same way. Do they have the same type in ES?

For the condition I had suggested square brackets. Did you make that change?

---

<div class="post-metadata">

**Author:** ![HRG](https://avatars.discourse-cdn.com/v4/letter/h/a4c791/32.png) [@HRG](https://discuss.elastic.co/u/HRG)\
**Post date:** [April 25, 2018, 4:22pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/35 "2018-04-25T16:22:59Z")

</div>

Somehow square brackers are not working. Its throwing error

In the logstash, both are same type(timestamp).

**Code**

if [LogEventTimeStamp] \>= [BusinesDateTimeConverted] {  
mutate {  
add\_field =\> {  
"LateFileStatus" =\> "LateArrival"  
}   
}   
} else {  
mutate {  
add\_field =\> {  
"LateFileStatus" =\> "On-Time"  
}   
}   
}

**Error**

[2018-04-25T12:17:09,633][ERROR][logstash.pipeline] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart L  
ogstash. {:pipeline\_id=\>"main", "exception"=\>"undefined method `>=' for nil:NilClass", "backtrace"=>["(eval):977:in`block in initialize'", "org/jruby/RubyArray.java:1734:in `each'", "( eval):975:in`block in initialize'", "(eval):558:in `block in filter_func'", "C:/Projects/logstash-6.2.3/logstash-6.2.3/logstash-core/lib/logstash/pipeline.rb:447:in`filter\_batch'", "C  
:/Projects/logstash-6.2.3/logstash-6.2.3/logstash-core/lib/logstash/pipeline.rb:426:in `worker_loop'", "C:/Projects/logstash-6.2.3/logstash-6.2.3/logstash-core/lib/logstash/pipeline.rb: 385:in`block in start\_workers'"], :thread=\>"#\<Thread:0x1d15c1c sleep\>"}

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [April 25, 2018, 4:45pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/36 "2018-04-25T16:45:41Z")

</div>

I'm not sure what's wrong. Could you comment the error causing part out, run the configuration with a rubydebug output and post it here?

---

<div class="post-metadata">

**Author:** ![HRG](https://avatars.discourse-cdn.com/v4/letter/h/a4c791/32.png) [@HRG](https://discuss.elastic.co/u/HRG)\
**Post date:** [April 25, 2018, 4:54pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/37 "2018-04-25T16:54:51Z")

</div>

{  
"tags"←[0;37m =\> ←[0m[  
←[1;37m[0] ←[0m←[0;33m"\_dateparsefailure"←[0m  
],  
"BusinesDateTime"←[0;37m =\> ←[0m←[0;33m"%{year}-%{month}-%{day} 03:00:00.000"←[0m,  
"BusinesDateTimeConverted"←[0;37m =\> ←[0m1970-01-02T00:00:00.000Z,  
"sourceSystem"←[0;37m =\> ←[0m←[0;33m"xxx"←[0m,  
"LateFileStatus"←[0;37m =\> ←[0m←[0;33m"LateArrival"←[0m,  
"message"←[0;37m =\> ←[0m←[0;33m"2018-04-19 00:18:42.394 INFO 4506994501 --- [http-nio-8080-exec-2] c.c.d.m.f.RequestLoggingFilter : [start-request]:  
client-ip=10.90.16.15, client-host=testserver, request-method=GET, url=http://10.90.106.48:8080/health, payload="←[0m,  
"@version"←[0;37m =\> ←[0m←[0;33m"1"←[0m,  
"host"←[0;37m =\> ←[0m←[0;33m"dt-x"←[0m,  
"type"←[0;37m =\> ←[0m←[0;33m"xxx"←[0m,  
"path"←[0;37m =\> ←[0m←[0;33m"C:/Projects/data/xxx/xxx.log"←[0m,  
"@timestamp"←[0;37m =\> ←[0m2018-04-25T16:52:34.207Z

In the kibana, it is showing 1 day added to the "BusinesDateTimeConverted" but when i did rubudebug, its not showing.

It looks this is the reason its not giving me "on-time" value when i compared dates.

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [April 25, 2018, 5:10pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/38 "2018-04-25T17:10:17Z")

</div>

What's this part of the configuration for? It should already be a Timestamp at this point, shouldn't it?

```
#Setting up Log timestamp to LogEventTimeStamp for the given formats. date { match => ["BusinesDateTimeConverted", "yyyy-MM-dd HH:mm:ss", "MMM dd, yyyy HH:mm:ss", "yyyy-MM-dd HH:mm:ss,SSS", "yyyy-MM-dd HH:mm:ss.SSS"] target => "BusinesDateTimeConverted" }
```

---

<div class="post-metadata">

**Author:** ![HRG](https://avatars.discourse-cdn.com/v4/letter/h/a4c791/32.png) [@HRG](https://discuss.elastic.co/u/HRG)\
**Post date:** [April 25, 2018, 6:23pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/41 "2018-04-25T18:23:28Z")

</div>

"LogEventTimeStamp"←[0;37m =\> ←[0m2018-04-19T04:15:08.790Z  
"BusinesDateTimeConverted"←[0;37m =\> ←[0m2018-04-20T07:00:00.000Z

When output showing is " "LateFileStatus"←[0;37m =\> ←[0m←[0;33m"LateArrival"←[0m"

The expected outcome is "OnTime".

**Here is logic**

```
if "LogEventTimeStamp" >= "BusinesDateTimeConverted" {
		mutate {
			add_field => {
				"LateFileStatus" => "LateArrival"
		    }	
		}	
    } else {
		mutate {
			add_field => {
				"LateFileStatus" => "On-Time"
		    }	
		} 	
}
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 25, 2018, 6:38pm UTC](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168/42 "2018-04-25T18:38:19Z")

</div>

> [@HRG](#):
>
> if "LogEventTimeStamp" \>= "BusinesDateTimeConverted" {

That is doing a string comparison. Try

```auto
if [LogEventTimeStamp] >= [BusinesDateTimeConverted] {
```

[Previous page](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168.md?page=1)

[Next page](https://discuss.elastic.co/t/adding-1-day-to-the-date/129168.md?page=3)