# Replacing @timestamp in logstash using grok fails \[Solved\]

**URL:** <https://discuss.elastic.co/t/replacing-timestamp-in-logstash-using-grok-fails-solved/87532>\
**Category:** Logstash\
**Created:** [May 30, 2017, 9:11am UTC](https://discuss.elastic.co/t/replacing-timestamp-in-logstash-using-grok-fails-solved/87532 "2017-05-30T09:11:51Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![b.thoor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b.thoor/32/18624_2.png) [@b.thoor](https://discuss.elastic.co/u/b.thoor)\
**Post date:** [May 30, 2017, 9:11am UTC](https://discuss.elastic.co/t/replacing-timestamp-in-logstash-using-grok-fails-solved/87532/1 "2017-05-30T09:11:51Z")

</div>

Hi.

The setup is this filebeat (ws2012r2) 5.4.0 -\> logstash (rhel7) 5.4.0 -\> elasticsearch (rhel7) 5.4.0 and the presentation layer is kibana (rhel7) 5.4.0.  
This is a sample line from the logs:

`2017-04-19 15:25:40,378 [80] DEBUG [something] [(null)] - Message was handled without exception 974cc6f7dd91407dbe435439058cda27be4`

The resulting @timestamp field is this:  
@timestamp:May 30th 2017, 10:35:11.786

And here are the relevant bits of the logstash config:

```
filter {
   mutate {
     remove_field => ["[beat][version]" ]
     remove_field => ["[offset]" ]
     remove_field => ["[type]" ]
     remove_field => ["[@version]" ]
     remove_field => ["[input_type]" ]
     remove_field => ["[beat][name]" ]
     remove_field => ["[beat][hostname]" ]
     remove_field => ["[tags]" ]
     rename => {'[host]' => '[hostname]'}
   }
   grok {
     match => ["message", "%{TIMESTAMP_ISO8601}"]
     overwrite => ["@timestamp"]
     add_field => {"TimestampTest" => "grok"}
   }
 }

```

The grok filter triggers because the "TimestampTest" shows up in ES but @timestamp is not replaced, I've checked the logs but there's nothing interesting in them.

Any idea as to what might be wrong?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 30, 2017, 10:10am UTC](https://discuss.elastic.co/t/replacing-timestamp-in-logstash-using-grok-fails-solved/87532/2 "2017-05-30T10:10:48Z")

</div>

`%{TIMESTAMP_ISO8601}` doesn't capture the timestamp into a field. You need `%{TIMESTAMP_ISO8601:@timestamp}`, but I'm not sure such a direct assignment will work. One typically extracts the timestamp to a temporary field and use the date filter to process it.

---

<div class="post-metadata">

**Author:** ![b.thoor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b.thoor/32/18624_2.png) [@b.thoor](https://discuss.elastic.co/u/b.thoor)\
**Post date:** [May 30, 2017, 11:43am UTC](https://discuss.elastic.co/t/replacing-timestamp-in-logstash-using-grok-fails-solved/87532/3 "2017-05-30T11:43:21Z")

</div>

This is the config I tried:

```
    grok {
        match => ["message", "%{TIMESTAMP_ISO8601:@timestamp}"]
        overwrite => ["@timestamp"]
        add_field => {"TimestampTest" => "grok"}
      }

```

And it resulted in this:

> 2017-05-30T12:19:35,234][WARN][logstash.filters.grok] Grok regexp threw exception {:exception=\>"wrong argument type String (expected LogStash::Timestamp)", :backtrace=\>["org/logstash/ext/JrubyEventExtLibrary.java:124:in `set'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.1/lib/logstash/filters/grok.rb:351:in `handle'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.1/lib/logstash/filters/grok.rb:338:in `match_against_groks'", "org/jruby/RubyProc.java:281:in `call'", "(eval):3:in `compile_captures_func'", "org/jruby/RubyProc.java:281:in `call'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.4/lib/grok-pure.rb:202:in `capture'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.1/lib/logstash/filters/grok.rb:338:in `match\_against\_groks'", "org/jruby/RubyArray.java:1613:in `each'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.1/lib/logstash/filters/grok.rb:333:in `match\_against\_groks'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.1/lib/logstash/filters/grok.rb:322:in `match'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.1/lib/logstash/filters/grok.rb:288:in `filter'", "org/jruby/RubyHash.java:1342:in `each'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.1/lib/logstash/filters/grok.rb:287:in `filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:145:in `do_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:164:in `multi\_filter'", "org/jruby/RubyArray.java:1613:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:161:in `multi\_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filter\_delegator.rb:43:in `multi_filter'", "(eval):72:in `filter\_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:370:in `filter_batch'", "org/jruby/RubyProc.java:281:in `call'", "/usr/share/logstash/logstash-core/lib/logstash/util/wrapped\_synchronous\_queue.rb:224:in `each'", "org/jruby/RubyHash.java:1342:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/util/wrapped\_synchronous\_queue.rb:223:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:369:in `filter\_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:350:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:317:in `start\_workers'"], :class=\>"TypeError"}

I did however replace the grok-block with the following:

```
 date {
   match => ["message", "%{TIMESTAMP_ISO8601:@timestamp}"]
   target => ["@timestamp"]
   add_field => {"DateTest" => "true"}
  }

```

which resulted in this in the logs:

> [ERROR][logstash.agent] Cannot create pipeline {:reason=\>"translation missing: en.logstash.agent.configuration.invalid\_plugin\_register"}

I've installed ELK from the rpm packages in the official repo.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 30, 2017, 1:05pm UTC](https://discuss.elastic.co/t/replacing-timestamp-in-logstash-using-grok-fails-solved/87532/4 "2017-05-30T13:05:38Z")

</div>

Yes, as I suspected you can't capture straight into `@timestamp`. Keep your grok filter but capture the timestamp to a different field than `@timestamp`. Use that field in your date filter and a use date pattern that matches your timestamp, like "ISO8601".

---

<div class="post-metadata">

**Author:** ![b.thoor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b.thoor/32/18624_2.png) [@b.thoor](https://discuss.elastic.co/u/b.thoor)\
**Post date:** [May 31, 2017, 12:01pm UTC](https://discuss.elastic.co/t/replacing-timestamp-in-logstash-using-grok-fails-solved/87532/5 "2017-05-31T12:01:12Z")

</div>

Right, so I tried this:

```
filter {
  mutate {
# Remove redundant information
    remove_field => ["[beat][version]" ]
    remove_field => ["[offset]" ]
    remove_field => ["[type]" ]
    remove_field => ["[@version]" ]
    remove_field => ["[input_type]" ]
    remove_field => ["[beat][name]" ]
    remove_field => ["[beat][hostname]" ]
    remove_field => ["[type]" ]
    remove_field => ["[tags]" ]

    rename => {'[host]' => '[hostname]'}

    add_field => { "TimestampTemp" => "x" }
  }
  grok {
    match => { "message" => "%{TIMESTAMP_ISO8601:@timestamp}" }
    overwrite => ["TimestampTemp"]
  }
  date {
    match => ["TimestampTemp", "%{TIMESTAMP_ISO8601:@timestamp}"]
    target => ["@timestamp"]
    remove_field => ["TimestampTemp"]
  }
}

```

With and without @timestamp in the match clause in the date block only to be greeted by this:

> [ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Illegal pattern component: T"}

With no reference to what line contains the offending clause I can only guess that the match in the date block is the problem so I tried a blunt regex like so:

`match => ["TimestampTemp", "\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}i,\d{3}"]`

That did remove the diffuse error message from the logs but it did nothing in terms of replacing the @timestamp.  
As it turns out the problem is that TimestampTemp is never overwritten, checking the documentation for grok -\> overwrite claims that one needs a match clause then overwrite like the one in the grok block above.

At least it seems I've found the issue but unfortunately that doesn't bring me any closer to a solution.

Any ideas?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 31, 2017, 1:51pm UTC](https://discuss.elastic.co/t/replacing-timestamp-in-logstash-using-grok-fails-solved/87532/6 "2017-05-31T13:51:00Z")

</div>

You're not following any of the recommendations in my previous post. This is what I meant:

```nohighlight
grok {
  match => { "message" => "%{TIMESTAMP_ISO8601:TimestampTemp}" }
}
date {
  match => ["TimestampTemp", "ISO8601"]
  remove_field => ["TimestampTemp"]
}

```

---

<div class="post-metadata">

**Author:** ![b.thoor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/b.thoor/32/18624_2.png) [@b.thoor](https://discuss.elastic.co/u/b.thoor)\
**Post date:** [May 31, 2017, 2:03pm UTC](https://discuss.elastic.co/t/replacing-timestamp-in-logstash-using-grok-fails-solved/87532/7 "2017-05-31T14:03:07Z")

</div>

I sort of did what you recommended like so:

```
filter {
  grok {
    match => { "message" => "(?m)%{TIMESTAMP_ISO8601:timestamp}" }
  }
  date {
    match => ["timestamp", "ISO8601", "yyyy-MM-dd HH:mm:ss,SSS"]
    remove_field => ["timestamp"]
  }
}

```

And it works, thanks for your help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 28, 2017, 2:03pm UTC](https://discuss.elastic.co/t/replacing-timestamp-in-logstash-using-grok-fails-solved/87532/8 "2017-06-28T14:03:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.