# Can't replace real log time with @timestamp

**URL:** <https://discuss.elastic.co/t/cant-replace-real-log-time-with-timestamp/189669>\
**Category:** Logstash\
**Created:** [July 10, 2019, 6:47am UTC](https://discuss.elastic.co/t/cant-replace-real-log-time-with-timestamp/189669 "2019-07-10T06:47:54Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mohammad\_hossein\_Taj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohammad_hossein_taj/32/44550_2.png) [@Mohammad\_hossein\_Taj](https://discuss.elastic.co/u/Mohammad_hossein_Taj)\
**Post date:** [July 10, 2019, 6:47am UTC](https://discuss.elastic.co/t/cant-replace-real-log-time-with-timestamp/189669/1 "2019-07-10T06:47:54Z")

</div>

hi  
I'm trying to replace my log time with @timestamp but it doesn't work.  
my log sample:  
`2019-07-08 01:00:13,564 INFO ir.ac.ut.sdrwebservice.SDRWebService @ batchAddStdDoc, System:G, User:25117, StudentIDs:[450188215], GroupID:4501, DocType:1349, returned 1562531413462298`  
and here is my logstash config:

```auto
input {
  beats {
    client_inactivity_timeout => 1200
    port => 5044
  }
}

filter {
  if ([message] !~ /batchAddStdDoc/) {
    drop { }
  }
  if ([message] !~ /returned/) {
    drop { }
  }
  grok {
    match => { "message" => "%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:day} %{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second},%{INT:milisecond} %{LOGLEVEL:loglevel} %{NOTSPACE:webService} @ %{NOTSPACE:function}, System:(?<systemName>.), User:%{NOTSPACE:userId}, StudentIDs:\[%{NUMBER:studentId}\], GroupID:%{GREEDYDATA:groupId}, DocType:%{NOTSPACE:docType}, returned %{INT:returnedCode}" }
  }
  date {
        locale => "en"
        match => ["message", "yyyy-MM-dd HH:mm:ss,SSS"]
        timezone => "Asia/Tehran"
        target => "@timestamp"
        add_field => { "debug" => "timestampMatched"}
   }
}

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
    #user => "elastic"
    #password => "changeme"
  }
}

```

thanks for your help.

---

<div class="post-metadata">

**Author:** ![Rom1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rom1/32/49699_2.png) [@Rom1](https://discuss.elastic.co/u/Rom1)\
**Post date:** [July 10, 2019, 7:01am UTC](https://discuss.elastic.co/t/cant-replace-real-log-time-with-timestamp/189669/2 "2019-07-10T07:01:29Z")

</div>

Hi Mohammad,

The problem is that you're parsing the whole message to create the timestamp:

> match =\> [" **message**", "yyyy-MM-dd HH:mm:ss,SSS"]

Try to grok the date with the following pattern:

> %{TIMESTAMP\_ISO8601: **timestamp** } %{LOGLEVEL:loglevel} ...

and use it with the date plugin:

> match =\> [" **timestamp**", "yyyy-MM-dd HH:mm:ss,SSS"]

Fyi: target =\> "@timestamp" is not necessary (by default)

---

<div class="post-metadata">

**Author:** ![Mohammad\_hossein\_Taj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohammad_hossein_taj/32/44550_2.png) [@Mohammad\_hossein\_Taj](https://discuss.elastic.co/u/Mohammad_hossein_Taj)\
**Post date:** [July 10, 2019, 7:12am UTC](https://discuss.elastic.co/t/cant-replace-real-log-time-with-timestamp/189669/3 "2019-07-10T07:12:45Z")

</div>

thanks for your help.  
I changed my filter like this:

```auto
filter {
  if ([message] !~ /batchAddStdDoc/) {
    drop { }
  }
  if ([message] !~ /returned/) {
    drop { }
  }
  grok {
    match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:loglevel} %{NOTSPACE:webService} @ %{NOTSPACE:function}, System:(?<systemName>.), User:%{NOTSPACE:userId}, StudentIDs:\[%{NUMBER:studentId}\], GroupID:%{GREEDYDATA:groupId}, DocType:%{NOTSPACE:docType}, returned %{INT:returnedCode}" }
  }
  date {
    match => ["timestamp", "yyyy-MM-dd HH:mm:ss,SSS"]
   }
}

```

but it's not working anymore and nothing in Discover part of Kibana!

---

<div class="post-metadata">

**Author:** ![Mohammad\_hossein\_Taj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohammad_hossein_taj/32/44550_2.png) [@Mohammad\_hossein\_Taj](https://discuss.elastic.co/u/Mohammad_hossein_Taj)\
**Post date:** [July 10, 2019, 7:57am UTC](https://discuss.elastic.co/t/cant-replace-real-log-time-with-timestamp/189669/4 "2019-07-10T07:57:01Z")

</div>

I'm one step forward now. when I use this filter it works:

```auto
filter {
  if ([message] !~ /batchAddStdDoc/) {
    drop { }
  }
  if ([message] !~ /returned/) {
    drop { }
  }
  grok {
    match => { "message" => "%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{LOGLEVEL:loglevel} %{NOTSPACE:webService} @ %{NOTSPACE:function}, System:(?<systemName>.), User:%{NOTSPACE:userId}, StudentIDs:\[%{NUMBER:studentId}\], GroupID:%{GREEDYDATA:groupId}, DocType:%{NOTSPACE:docType}, returned %{INT:returnedCode}" }
  }

```

but when I use date filter it doesn't work anymore and nothing to Discover with Kibana!

```auto
filter {
  if ([message] !~ /batchAddStdDoc/) {
    drop { }
  }
  if ([message] !~ /returned/) {
    drop { }
  }
  grok {
    match => { "message" => "%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{LOGLEVEL:loglevel} %{NOTSPACE:webService} @ %{NOTSPACE:function}, System:(?<systemName>.), User:%{NOTSPACE:userId}, StudentIDs:\[%{NUMBER:studentId}\], GroupID:%{GREEDYDATA:groupId}, DocType:%{NOTSPACE:docType}, returned %{INT:returnedCode}" }
  }
  date {
    match => ["timestamp", "yyyy-MM-dd HH:mm:ss,SSS"]
    timezone => "Asia/Tehran"
    target => "@timestamp"
  }
}

```

do you have any idea @Rom1?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 10, 2019, 2:13pm UTC](https://discuss.elastic.co/t/cant-replace-real-log-time-with-timestamp/189669/5 "2019-07-10T14:13:27Z")

</div>

In your grok pattern you have two spaces between

```
%{LOGLEVEL:loglevel} %{NOTSPACE:webService}

```

However, your message only has one, so your get a \_grokparsefailure. Remove one of the spaces and you will get

```
   "timestamp" => "2019-07-08 01:00:13,564",
    "loglevel" => "INFO",

```

etc.

---

<div class="post-metadata">

**Author:** ![Mohammad\_hossein\_Taj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohammad_hossein_taj/32/44550_2.png) [@Mohammad\_hossein\_Taj](https://discuss.elastic.co/u/Mohammad_hossein_Taj)\
**Post date:** [July 11, 2019, 4:26am UTC](https://discuss.elastic.co/t/cant-replace-real-log-time-with-timestamp/189669/6 "2019-07-11T04:26:48Z")

</div>

thanks Badger.  
my logs has 2 space there but I don't know why it's not visible here!  
finally, I solved It. my filter was OK and **I need to change time range** because my logs were at least for two days ago and I was looking for them in today time range in Kibana.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 8, 2019, 4:26am UTC](https://discuss.elastic.co/t/cant-replace-real-log-time-with-timestamp/189669/7 "2019-08-08T04:26:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.