# Update @timestamp as per log data

**URL:** <https://discuss.elastic.co/t/update-timestamp-as-per-log-data/83927>\
**Category:** Logstash\
**Created:** [April 27, 2017, 6:51pm UTC](https://discuss.elastic.co/t/update-timestamp-as-per-log-data/83927 "2017-04-27T18:51:15Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![sumitbiswas](https://avatars.discourse-cdn.com/v4/letter/s/a3d4f5/32.png) [@sumitbiswas](https://discuss.elastic.co/u/sumitbiswas)\
**Post date:** [April 27, 2017, 6:51pm UTC](https://discuss.elastic.co/t/update-timestamp-as-per-log-data/83927/1 "2017-04-27T18:51:15Z")

</div>

My log looks like below

`Apr 27 10:35:44.001204 xxxClient(106809) NOTICE: {970020} ABC CLIENT_MIN_LATENCY in the last period for host abc4.example.com was 4906`

Can you please help me to know how do I update event's @timestamp with log data?

I am using this. Logstash - v5.3.1.

Logstash.config is as below

input {  
beats {  
port =\> 5044  
}  
}

filter {  
grok {  
match =\> { "message" =\> "%{MONTH} %{MONTHDAY} %{TIME} %{WORD:ProcessName}(%{INT:ProcessId}) %{WORD:Status}: {%{INT:ProcessId}} %{WORD:SubProcessName} %{WORD:LatencyDetails} \in the last period for host %{GREEDYDATA:DgwHost} \was %{INT:Latency}" }  
}

date {  
#locale =\> "en"  
#timezone =\> "America/New\_York"  
#match =\> ["@timestamp", "MMM dd HH:mm:ss.SSS"]  
#add\_tag =\> ["tsmatch"]

```
  #match => ["event_timestamp" , "MMM dd HH:mm:ss.SSS" , "ISO8601"]
  match => ["event_timestamp" , "YYYY-MM-dd HH:mm:ss"]
  #target => "@timestamp"
  #add_tag => ["tmatch"]
}

```

}

output {  
elasticsearch {  
hosts =\> ["[http://localhost:9200](http://localhost:9200)"]  
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 28, 2017, 5:14am UTC](https://discuss.elastic.co/t/update-timestamp-as-per-log-data/83927/2 "2017-04-28T05:14:00Z")

</div>

Why reinvent the wheel when Logstash ships with patterns for syslog files? See the examples section in the Logstash documentation.

---

<div class="post-metadata">

**Author:** ![sumitbiswas](https://avatars.discourse-cdn.com/v4/letter/s/a3d4f5/32.png) [@sumitbiswas](https://discuss.elastic.co/u/sumitbiswas)\
**Post date:** [April 28, 2017, 5:36am UTC](https://discuss.elastic.co/t/update-timestamp-as-per-log-data/83927/3 "2017-04-28T05:36:04Z")

</div>

Thanks Magnus for your response. Actually I tried various options but my end goal to assign date from Log files to Logstash Timestap didn't work.

Can you please help me for the above log line?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 28, 2017, 5:46am UTC](https://discuss.elastic.co/t/update-timestamp-as-per-log-data/83927/4 "2017-04-28T05:46:01Z")

</div>

Looking more closely I see that your log isn't quite in syslog format so using a custom grok filter makes sense. But you're trying to parse the `event_timestamp` field which doesn't exist. In fact, you're not extracting _any_ fields from the timestamp. This should give you an `event_timestamp` field that you can parse:

```
^(?<event_timestamp>%{MONTH} %{MONTHDAY} %{TIME}) %{WORD:ProcessName} ...

```

I strongly suggest that while you're debugging your filters use a `stdout { codec => rubydebug }` output instead of your current elasticsearch output.

---

<div class="post-metadata">

**Author:** ![sumitbiswas](https://avatars.discourse-cdn.com/v4/letter/s/a3d4f5/32.png) [@sumitbiswas](https://discuss.elastic.co/u/sumitbiswas)\
**Post date:** [April 28, 2017, 5:50am UTC](https://discuss.elastic.co/t/update-timestamp-as-per-log-data/83927/5 "2017-04-28T05:50:58Z")

</div>

I think I could make it partially work. I.e "timestamp" is replaced partially. Everything except "hour" is replaced properly. Please let me know if I am doing something wrong.

Below is my logstash config.

```
filter {
  grok {
    match => { "message" => "%{SYSLOGTIMESTAMP:timestamp} %{WORD:ProcessName}\(%{INT:ProcessId}\) %{WORD:Status}\: \{%{INT:ProcessId}\} %{WORD:SubProcessName} %{WORD:LatencyDetails} \in the last period for host %{GREEDYDATA:DgwHost} \was %{INT:Latency}" }
  }

  mutate {
    convert => { "Latency" => "integer" }
  }

  date {
    match => ["timestamp" , "MMM dd HH:mm:ss.SSSSSS"]
  }
}

```

And my output is as below.

```
"message" : "Mar 29 11:56:15.410045 diameterBeClient(16044) NOTICE: {970020} DCD DIAMETER_MIN_LATENCY in the last period for host dgw1.example.com was 5845"
"@timestamp" : "2017-03-29T15:56:15.410Z"
"timestamp" : "Mar 29 11:56:15.410045"
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 28, 2017, 5:55am UTC](https://discuss.elastic.co/t/update-timestamp-as-per-log-data/83927/6 "2017-04-28T05:55:36Z")

</div>

What's the timezone of the machine where Logstash runs? If it's UTC-4 then things are working just fine. `@timestamp` is always UTC.

---

<div class="post-metadata">

**Author:** ![sumitbiswas](https://avatars.discourse-cdn.com/v4/letter/s/a3d4f5/32.png) [@sumitbiswas](https://discuss.elastic.co/u/sumitbiswas)\
**Post date:** [April 28, 2017, 6:00am UTC](https://discuss.elastic.co/t/update-timestamp-as-per-log-data/83927/7 "2017-04-28T06:00:18Z")

</div>

Oh that explains it. My Logstash is in ET "Eastern Daylight Time" timezone.

Is there any possibility to set @timestamp with exact value of Log date irrespective of what timezone my Application server is running?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 28, 2017, 6:04am UTC](https://discuss.elastic.co/t/update-timestamp-as-per-log-data/83927/8 "2017-04-28T06:04:08Z")

</div>

As I said `@timestamp` is always UTC.

---

<div class="post-metadata">

**Author:** ![sumitbiswas](https://avatars.discourse-cdn.com/v4/letter/s/a3d4f5/32.png) [@sumitbiswas](https://discuss.elastic.co/u/sumitbiswas)\
**Post date:** [April 28, 2017, 6:20am UTC](https://discuss.elastic.co/t/update-timestamp-as-per-log-data/83927/9 "2017-04-28T06:20:42Z")

</div>

Got it now. Thanks Magnus for all your help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 26, 2017, 6:29am UTC](https://discuss.elastic.co/t/update-timestamp-as-per-log-data/83927/10 "2017-05-26T06:29:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.