# Replace timestamp from logfile - dateparsefailure

**URL:** https://discuss.elastic.co/t/replace-timestamp-from-logfile-dateparsefailure/114800
**Category:** Logstash
**Created:** [January 10, 2018, 6:57am UTC](https://discuss.elastic.co/t/replace-timestamp-from-logfile-dateparsefailure/114800 "2018-01-10T06:57:17Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![A.Klos](https://avatars.discourse-cdn.com/v4/letter/a/977dab/32.png) [@A.Klos](https://discuss.elastic.co/u/A.Klos)
#### Post date: [January 10, 2018, 6:57am UTC](https://discuss.elastic.co/t/replace-timestamp-from-logfile-dateparsefailure/114800/1 "2018-01-10T06:57:18Z")

</div>

Hi,

my logs starting with:

10.01.2018 | 07:43:13,961 | INFO |  
09.01.2018 | 23:59:02,119 | DEBUG |

I tried following:

%{DATE\_EU:Datum} | (%{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second}%{ISO8601\_TIMEZONE:timezone})

But hour, minute etc are empty (null)

How could I replace timestamp with this Date?

Regards

---

<div class="post-metadata">

### Author: ![Sjaak01](https://avatars.discourse-cdn.com/v4/letter/s/73ab20/32.png) [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)
#### Post date: [January 10, 2018, 8:20am UTC](https://discuss.elastic.co/t/replace-timestamp-from-logfile-dateparsefailure/114800/2 "2018-01-10T08:20:25Z")

</div>

Something like `(?<date>[\d.\d.\d]{10}) (?<time>[\d:\d:\d,\d]{12})` should work but in the grok debugger the | doesn't play nice so you might want to use a gsub filter first to remove/replace that character.

Then use mutate to create a new field  
`mutate { add_field => { "TIMEDATE" => "%{date} %{time}" } }`

And date to replace it with the timestamp  
date {  
match =\> ["TIMEDATE", "yyyy-MM-dd HH:mm:sss"] (change this)  
timezone =\> "UTC"  
target =\> "@timestamp"  
}

Not pretty but it will work.

---

<div class="post-metadata">

### Author: ![A.Klos](https://avatars.discourse-cdn.com/v4/letter/a/977dab/32.png) [@A.Klos](https://discuss.elastic.co/u/A.Klos)
#### Post date: [January 10, 2018, 10:09am UTC](https://discuss.elastic.co/t/replace-timestamp-from-logfile-dateparsefailure/114800/3 "2018-01-10T10:09:33Z")

</div>

Thank you,

in grogdebugger I got matches for:

09.01.2018 | 23:59:02,119 | DEBUG |

Pattern:

(?[\d.\d.\d]{10}) | (?[\d:\d:\d,\d]{12})

Now I created following config file (for Test I created an input file)

input {  
file {  
path =\> ["/tmp/app"]  
type =\> "FilePluginInput"  
start\_position =\> beginning  
sincedb\_path =\> "/dev/null"  
}  
}

filter {  
if [type] in ["FilePluginInput"] {  
grok {  
mutate {  
gsub =\> [  
"message", "|", "", // =\> remove |  
]  
}  
match =\> [  
"message" , "(?[\d.\d.\d]{10}) | (?[\d:\d:\d,\d]{12})"  
]  
}  
mutate {  
add\_field =\> { "TIMEDATE" =\> "%{date} %{time}" }  
add\_tag =\> [app]  
}  
date {  
match =\> ["TIMEDATE", "yyyy-MM-dd HH:mm:sss"]  
timezone =\> "UTC"  
target =\> "@timestamp"  
}  
}  
}

Bevor | there is a backslash , this one isnt displayd in my thread entry.  
Any Idea?

---

<div class="post-metadata">

### Author: ![A.Klos](https://avatars.discourse-cdn.com/v4/letter/a/977dab/32.png) [@A.Klos](https://discuss.elastic.co/u/A.Klos)
#### Post date: [January 10, 2018, 1:03pm UTC](https://discuss.elastic.co/t/replace-timestamp-from-logfile-dateparsefailure/114800/4 "2018-01-10T13:03:06Z")

</div>

I think I'm nearly to finish. Now I got dateparse faillure:

Following Fields:

my filter:

input {  
file {  
path =\> ["/tmp/app"]  
type =\> "FilePluginInput"  
start\_position =\> beginning  
sincedb\_path =\> "/dev/null"  
}  
}

filter {  
grok {  
match =\> ["message" , "%{DATE\_EU:Datum} | (?[\d:\d:\d,\d]{12})" ]  
}  
mutate {  
add\_field =\> { "TIMEDATE" =\> "%{Datum} %{Uhrzeit}" }  
}  
date {  
match =\> ["TIMEDATE", "YYYY-MM-dd'T' HH:mm:ss.SSSZ"]  
target =\> "@timestamp"  
}  
}

Fields:  
"TIMEDATE" =\> "10.01.2018 13:37:02,942",  
"path" =\> "/tmp/app",  
"@timestamp" =\> 2018-01-10T12:45:42.417Z,  
"Datum" =\> "10.01.2018",  
"@version" =\> "1",  
"host" =\> "XXX",  
"message" =\> "10.01.2018 | 13:37:02,942 | DEBUG | yyyyy",  
"type" =\> "FilePluginInput",  
"Uhrzeit" =\> "13:37:02,942",  
"tags" =\> [  
[0] "\_dateparsefailure"

Regards

---

<div class="post-metadata">

### Author: ![A.Klos](https://avatars.discourse-cdn.com/v4/letter/a/977dab/32.png) [@A.Klos](https://discuss.elastic.co/u/A.Klos)
#### Post date: [January 10, 2018, 2:21pm UTC](https://discuss.elastic.co/t/replace-timestamp-from-logfile-dateparsefailure/114800/5 "2018-01-10T14:21:53Z")

</div>

It's an hardway 😉 Now I think I'm at 99% finished.

My Filter:

filter {  
grok {  
match =\> ["message" , "%{DATE\_EU:Datum} | (?[\d:\d:\d,\d]{12})" ]  
}  
mutate {  
add\_field =\> { "TIMEDATE" =\> "%{Datum} %{Uhrzeit}" }  
}  
date {  
timezone =\> "Europe/Berlin"  
match =\> ["TIMEDATE", "ISO8601", "dd.MM.yyyy HH:mm:ss','SSSS"]  
target =\> "@timestamp"  
}  
}

Output:

```
  "TIMEDATE" => "09.01.2018 23:59:02,119",
      "path" => "/tmp/app",
"@timestamp" => 2018-01-09T22:59:02.119Z,
     "Datum" => "09.01.2018",
  "@version" => "1",

```

Timestamp is 1 hour earlier than TIMEDATE, what can I do?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 10, 2018, 2:34pm UTC](https://discuss.elastic.co/t/replace-timestamp-from-logfile-dateparsefailure/114800/6 "2018-01-10T14:34:43Z")

</div>

> [@A.Klos](#):
>
> Output:
> 
> "TIMEDATE" =\> "09.01.2018 23:59:02,119",  
> "path" =\> "/tmp/app",  
> "@timestamp" =\> 2018-01-09T22:59:02.119Z,  
> "Datum" =\> "09.01.2018",  
> "@version" =\> "1",
> 
> Timestamp is 1 hour earlier than TIMEDATE, what can I do?

Elastic always stores times as UTC. You told it that timestamp was in the Europe/Berlin timezone, which is UTC +1:00, so this is working as designed.

---

<div class="post-metadata">

### Author: ![A.Klos](https://avatars.discourse-cdn.com/v4/letter/a/977dab/32.png) [@A.Klos](https://discuss.elastic.co/u/A.Klos)
#### Post date: [January 10, 2018, 2:47pm UTC](https://discuss.elastic.co/t/replace-timestamp-from-logfile-dateparsefailure/114800/7 "2018-01-10T14:47:12Z")

</div>

thank you,

in the meantime I also found it.

thanks everyone for help.

Regards

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 7, 2018, 3:01pm UTC](https://discuss.elastic.co/t/replace-timestamp-from-logfile-dateparsefailure/114800/8 "2018-02-07T15:01:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.