# How to replace logstash read time with log timing?

**URL:** <https://discuss.elastic.co/t/how-to-replace-logstash-read-time-with-log-timing/285834>\
**Category:** Logstash\
**Created:** [October 4, 2021, 2:52pm UTC](https://discuss.elastic.co/t/how-to-replace-logstash-read-time-with-log-timing/285834 "2021-10-04T14:52:46Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pradeep\_Kumar2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pradeep_kumar2/32/85040_2.png) [@Pradeep\_Kumar2](https://discuss.elastic.co/u/Pradeep_Kumar2)\
**Post date:** [October 4, 2021, 2:52pm UTC](https://discuss.elastic.co/t/how-to-replace-logstash-read-time-with-log-timing/285834/1 "2021-10-04T14:52:46Z")

</div>

Hi, I am trying to configure logstash file which will replace the logstash read timestamp or system time with actual log time. In the kibana discover dashboard logs are displayed with logstash read time. How to replace this logstash read time. with actual log time.

```auto
  file {
    path => "C:/Users/eaampnr/ELK/elk/csi/cic-1/log/events/**/*.log"
    start_position => "beginning"
    type => "event"
  }
}

filter{
  date {
    match => ["msg_timestamp", "yyyy-MM-dd'T'HH:mm:ss','SSS"]
    target => "msgtime"
  }
      
  mutate {
    remove_field => ["@timestamp"]
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
  }
  stdout { codec => rubydebug }
}

```

actual log time - 2020-08-15T14:31:54,899 which is in the "message"

_output_

```auto
	{
      "@version" => "1",
       "message" => "2020-08-15T14:31:54,899 | ELAN-DpnInterface, ADD DPN 222845117715277 Instance e183ffcf-2a5b-4aa8-8821-33fbe508ea77",
          "path" => "C:/Users/eaampnr/ELK/elk/csi/cic-1/log/events/netvirt/netvirt.log",
       "@timestamp" => 2021-10-04T14:28:36.233Z, --- logstash read time, laptop time 
          "type" => "event",
          "host" => "XXXXXXX"
   }

```

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [October 4, 2021, 3:05pm UTC](https://discuss.elastic.co/t/how-to-replace-logstash-read-time-with-log-timing/285834/2 "2021-10-04T15:05:00Z")

</div>

When you add a `target` it takes that value and puts it into that field. If you remove the `target` then it will put it in `@timestamp`. It sounds like you are looking to do this.

```auto
filter{
  date {
    match => ["msg_timestamp", "yyyy-MM-dd'T'HH:mm:ss','SSS"]
  }
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 4, 2021, 3:26pm UTC](https://discuss.elastic.co/t/how-to-replace-logstash-read-time-with-log-timing/285834/3 "2021-10-04T15:26:59Z")

</div>

You have not show any filter that creates the [msg\_timestamp] field. If that field does not exist then the date filter is a no-op.

---

<div class="post-metadata">

**Author:** ![Pradeep\_Kumar2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pradeep_kumar2/32/85040_2.png) [@Pradeep\_Kumar2](https://discuss.elastic.co/u/Pradeep_Kumar2)\
**Post date:** [October 4, 2021, 4:50pm UTC](https://discuss.elastic.co/t/how-to-replace-logstash-read-time-with-log-timing/285834/4 "2021-10-04T16:50:16Z")

</div>

> [@aaron-nimocks](#):
>
> ```auto
> filter{
> date {
> match => ["msg_timestamp", "yyyy-MM-dd'T'HH:mm:ss','SSS"]
> }
> }
> 
> ```

Thanks for the quick response. I tried what is suggested it did not work.

the output  
' {  
"message" =\> "2021-05-01T18:01:46,993 | ITM-TunnelInventoryState,REMOVE DTCN received for tun4fdba39c604",  
"host" =\> "IN-00211777",  
"type" =\> "event",  
"@timestamp" =\> 2021-10-04T16:44:54.778Z,  
"@version" =\> "1",  
"path" =\> "C:/Users/eaampnr/ELK/elk/csi/cic-1/log/events/genius/genius.log"  
}  
'  
My expectation of output is below

{  
"message" =\> "2021-05-01T18:01:46,993 | ITM-TunnelInventoryState,REMOVE DTCN received for tun4fdba39c604", --- i want to remove the timestamp from the log messages  
"host" =\> "IN-00211777",  
"type" =\> "event",  
"@timestamp" =\> 2021-05-01T18:01:46,993, ----- it must be the log timestamp  
"@version" =\> "1",  
"path" =\> "C:/Users/eaampnr/ELK/elk/csi/cic-1/log/events/genius/genius.log"  
}

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [October 4, 2021, 4:57pm UTC](https://discuss.elastic.co/t/how-to-replace-logstash-read-time-with-log-timing/285834/5 "2021-10-04T16:57:03Z")

</div>

I was thinking you didn't post your entire conf but as @badger pointed out you aren't processing your message. You will need to parse your `message` field to break it down into individual fields before you use them. One method you can use is [grok](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html).

Something like this before you use the `msg_timestamp` field.

```auto
filter { 
    
 grok {
  match => { "message" => "%{DATA:msg_timestamp} \| %{GREEDYDATA:msg}" }
 }

 date {
  match => ["msg_timestamp", "yyyy-MM-dd'T'HH:mm:ss','SSS"]
 }
      
 mutate {
  remove_field => ["message"]
 }     

}

```

Output

```auto
{
    "msg": "ITM-TunnelInventoryState,REMOVE DTCN received for tun4fdba39c604",
    "msg_timestamp": "2021-05-01T18:01:46,993",
    "@timestamp": " 2021-05-01T22:01:46.993Z" <--- adjusted for timezone that's saved in Zulu time
}

```

---

<div class="post-metadata">

**Author:** ![Pradeep\_Kumar2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pradeep_kumar2/32/85040_2.png) [@Pradeep\_Kumar2](https://discuss.elastic.co/u/Pradeep_Kumar2)\
**Post date:** [October 4, 2021, 5:35pm UTC](https://discuss.elastic.co/t/how-to-replace-logstash-read-time-with-log-timing/285834/6 "2021-10-04T17:35:49Z")

</div>

> [@Badger](#):
>
> msg\_timestamp

Hi Badger,

thank you for the inputs. I used the 'timestamp' instead of 'msg\_timestamp'. But still I see the below output. Log timestamp is not overriding in the output.

output:  
{  
"message" =\> "2021-05-03T20:19:22,084 | IFM-InterfaceInventoryState,REMOVE tun988ddd2ffe9",  
"path" =\> "C:/Users/eaampnr/ELK/elk/csi/cic-1/log/events/genius/genius.log",  
"host" =\> "IN-00211777",  
"type" =\> "event",  
"@version" =\> "1",  
"@timestamp" =\> 2021-10-04T17:06:15.346Z  
}

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [October 4, 2021, 5:41pm UTC](https://discuss.elastic.co/t/how-to-replace-logstash-read-time-with-log-timing/285834/7 "2021-10-04T17:41:10Z")

</div>

Use the filter I posted above. This should work.

```auto
filter { 
    
 grok {
  match => { "message" => "%{DATA:msg_timestamp} \| %{GREEDYDATA:msg}" }
 }

 date {
  match => ["msg_timestamp", "yyyy-MM-dd'T'HH:mm:ss','SSS"]
 }
      
 mutate {
  remove_field => ["message"]
 }     

}

```

---

<div class="post-metadata">

**Author:** ![Pradeep\_Kumar2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pradeep_kumar2/32/85040_2.png) [@Pradeep\_Kumar2](https://discuss.elastic.co/u/Pradeep_Kumar2)\
**Post date:** [October 4, 2021, 6:03pm UTC](https://discuss.elastic.co/t/how-to-replace-logstash-read-time-with-log-timing/285834/8 "2021-10-04T18:03:23Z")

</div>

Thank you for config. the filter file you provided is working fine. The only change required is the "timestamp" to be in the same time zone of msg\_timestamp.

```
"host" => "IN-00211777",
"msg_timestamp" => "2021-05-03T20:19:26,610",
   "@timestamp" => 2021-05-03T14:49:26.610Z,
          "msg" => "Node added to oper ovsdb://uuid/8a932fd8-1e29-4d25-89c0-d97acd295df1/bridge/br-sdnc-sbi",
     "@version" => "1",
         "path" => "C:/Users/eaampnr/ELK/elk/csi/cic-1/log/events/ovsdb/ovsdb.log",
         "type" => "event"
```

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [October 4, 2021, 6:08pm UTC](https://discuss.elastic.co/t/how-to-replace-logstash-read-time-with-log-timing/285834/9 "2021-10-04T18:08:03Z")

</div>

All times in Elastic are stored in Zulu time and that's why you see the shift. Notice the `Z` at the end of the time.

If you need to set your [timezone](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html#plugins-filters-date-timezone) where the timezone in your message is generated from you can. But either way it will converted to Zulu time.

---

<div class="post-metadata">

**Author:** ![Pradeep\_Kumar2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pradeep_kumar2/32/85040_2.png) [@Pradeep\_Kumar2](https://discuss.elastic.co/u/Pradeep_Kumar2)\
**Post date:** [October 4, 2021, 6:25pm UTC](https://discuss.elastic.co/t/how-to-replace-logstash-read-time-with-log-timing/285834/10 "2021-10-04T18:25:56Z")

</div>

ok thank. we can close this topic.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 1, 2021, 6:26pm UTC](https://discuss.elastic.co/t/how-to-replace-logstash-read-time-with-log-timing/285834/11 "2021-11-01T18:26:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.