# Replace @timestamp with source log timestamp

**URL:** <https://discuss.elastic.co/t/replace-timestamp-with-source-log-timestamp/162346>\
**Category:** Logstash\
**Created:** [December 28, 2018, 1:33pm UTC](https://discuss.elastic.co/t/replace-timestamp-with-source-log-timestamp/162346 "2018-12-28T13:33:46Z")\
**Posts on this page:** 8\
**Page:** 2

<div class="post-metadata">

**Author:** ![wholzgruber](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wholzgruber/32/39267_2.png) [@wholzgruber](https://discuss.elastic.co/u/wholzgruber)\
**Post date:** [January 2, 2019, 5:33pm UTC](https://discuss.elastic.co/t/replace-timestamp-with-source-log-timestamp/162346/21 "2019-01-02T17:33:07Z")

</div>

> [@admlko](#):
>
> As you can see, there is a syntax error with your config.

Well this could be but I do not have a config file larger than 33 lines. Any hints where this line 80 could be?

Regards

---

<div class="post-metadata">

**Author:** ![admlko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/admlko/32/21787_2.png) [@admlko](https://discuss.elastic.co/u/admlko)\
**Post date:** [January 3, 2019, 6:33am UTC](https://discuss.elastic.co/t/replace-timestamp-with-source-log-timestamp/162346/22 "2019-01-03T06:33:48Z")

</div>

Config file means all the files in conf.d if you are using default structure. So you can just print them sequentially. Or you can re-read the error and notice that it actually states exactly where the error is:

> Expected one of #, else, if, ", ', } at line 80, column 4 (byte 1795) **after output {\n if "citrix-netscaler-vor" in [tags]{\n stdout {\n codec =\> rubydebug\n }\n }**"

---

<div class="post-metadata">

**Author:** ![wholzgruber](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wholzgruber/32/39267_2.png) [@wholzgruber](https://discuss.elastic.co/u/wholzgruber)\
**Post date:** [January 3, 2019, 9:10am UTC](https://discuss.elastic.co/t/replace-timestamp-with-source-log-timestamp/162346/23 "2019-01-03T09:10:58Z")

</div>

Dear Guys,  
I have now reviewed everything i had.

\_grokparsefailure is correct because some source lines had different resonspes:

**Normal one:**

```
2018-11-23 00:00:00 192.168.116.163 - HTTP 192.168.10.78 80 GET /maintenance/pacfile.dat - 304 146 235 0 HTTP/1.1 - - -

```

**Bad one**

```
2018-11-23 00:00:00 192.168.116.163 - HTTP http://iambad.com 80 GET /maintenance/pacfile.dat - 304 146 235 0 HTTP/1.1 - - -

```

So my pattern did not work because i expect there an IPV4 address. This "\_grokparsefailures" are moved in an index with current datestamp. So this messages were the first which were shown in discover.

If you choose an other time-range you will find all propper messages parsed exactly. 😶

OK! First issue solved.

The second one whats still current is that every source log file starts with following 4 lines:

```
#Version: 1.0
#Software: Netscaler Web Logging(NSWL)
#Date: 2018-11-23 00:00:00 --> for sure this field changes : )
#Fields: date time c-ip cs-username sc-servicename s-ip s-port cs-method cs-uri-stem cs-uri-query sc-status cs-bytes sc-bytes time-taken cs-version cs(User-Agent) cs(Cookie) cs(Referer)

```

and of course those are also entrys with \_grokparsefailure. So I had to get rid of them.

With four separate filters like:

```
filter {
if "#Version: 1.0" in [message] {
drop { }
     }
}

```

And after getting rid of all those crap I got follwing result in kibana discover:

 ![final](https://us1.discourse-cdn.com/elastic/original/3X/e/c/ecdc1db43bbdec2d26ec9c317e9c5963dadfc84e.png)  
Do you see this beauty...😍 event\_timestamp matches with @timestamp

So I want to thank you for your help!! Really! Thanks a lot.

---

<div class="post-metadata">

**Author:** ![admlko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/admlko/32/21787_2.png) [@admlko](https://discuss.elastic.co/u/admlko)\
**Post date:** [January 3, 2019, 9:30am UTC](https://discuss.elastic.co/t/replace-timestamp-with-source-log-timestamp/162346/24 "2019-01-03T09:30:41Z")

</div>

Glad you got it working, also thanks for the explanation, makes sense.

> With four separate filters like:
> 
> ```auto
> filter {
> if "#Version: 1.0" in [message] {
> drop { }
> }
> }
> 
> ```

Just a note, I would replace those four filters with one:

```auto
filter {
  if [message] =~ /^#.*/ {
    drop {}
  }
}

```

Also, now that you got the @timestamp to work properly, you could add remove\_fields -option in the date filter which would remove event\_timestamp field if the date filter succeeds.

---

<div class="post-metadata">

**Author:** ![wholzgruber](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wholzgruber/32/39267_2.png) [@wholzgruber](https://discuss.elastic.co/u/wholzgruber)\
**Post date:** [January 3, 2019, 9:34am UTC](https://discuss.elastic.co/t/replace-timestamp-with-source-log-timestamp/162346/25 "2019-01-03T09:34:42Z")

</div>

> [@admlko](#):
>
> Just a note, I would replace those four filters with one:

Adapted... 🙂  
Thanks!

---

<div class="post-metadata">

**Author:** ![wholzgruber](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wholzgruber/32/39267_2.png) [@wholzgruber](https://discuss.elastic.co/u/wholzgruber)\
**Post date:** [January 23, 2019, 5:06pm UTC](https://discuss.elastic.co/t/replace-timestamp-with-source-log-timestamp/162346/26 "2019-01-23T17:06:03Z")

</div>

Dear Community,

I have an other problem with matching timestamps. Maybe you have a right type:

Source Logfile Timestamp:

```
2019-01-23T11:08:16+01:00

```

I tried following:

```
match => ["timestamp", "ISO8601"]    

```

or

```
match => ["timestamp", "yyyy-MM-dd'T'HH:mm:ssZZ:ZZ"]

```

or

```
match => ["timestamp", "yyyy-MM-dd'T'HH:mm:ssZZZZ"]

```

without success.

Maybe there is an other problem but please be so kind and tell me which format should match.

Regards  
Wilhelm

---

<div class="post-metadata">

**Author:** ![wholzgruber](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wholzgruber/32/39267_2.png) [@wholzgruber](https://discuss.elastic.co/u/wholzgruber)\
**Post date:** [January 23, 2019, 6:23pm UTC](https://discuss.elastic.co/t/replace-timestamp-with-source-log-timestamp/162346/27 "2019-01-23T18:23:07Z")

</div>

is solved.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 20, 2019, 6:23pm UTC](https://discuss.elastic.co/t/replace-timestamp-with-source-log-timestamp/162346/28 "2019-02-20T18:23:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/replace-timestamp-with-source-log-timestamp/162346.md?page=1)