# Replacing @timestamp with logs having custom timestamp

**URL:** <https://discuss.elastic.co/t/replacing-timestamp-with-logs-having-custom-timestamp/323656>\
**Category:** Logstash\
**Created:** [January 21, 2023, 4:55pm UTC](https://discuss.elastic.co/t/replacing-timestamp-with-logs-having-custom-timestamp/323656 "2023-01-21T16:55:23Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![sahoo35](https://avatars.discourse-cdn.com/v4/letter/s/c6cbf5/32.png) [@sahoo35](https://discuss.elastic.co/u/sahoo35)\
**Post date:** [January 21, 2023, 4:55pm UTC](https://discuss.elastic.co/t/replacing-timestamp-with-logs-having-custom-timestamp/323656/1 "2023-01-21T16:55:23Z")

</div>

Hello Everyone,  
I am newbie in ELK stack and i am still learning the logstash, kibana and its further uses . Currently i am stuck at a point where i want to extract the time stamp from my logs and replace it with @timestamp of the kibana dashboard . I have seen there are various other people who faced the same issue but none of their solution seem to be working for me .

Here is the grok filter that i am using for my logs

```auto
filter {
  if [fields][Component] == "Campaign Director" {
    grok {
      match => {
        'message' => '(?<LogLevel>[%{WORD}]+)([-])?(?<Category>[%{WORD}]+)? (?<TimeStamp>%{MONTHDAY:Day} %{MONTH:Month} %{YEAR:Year} %{HOUR}:%{MINUTE}:%{SECOND}\.%{WORD:Milliseconds}) \[%{DATA:ThreadName}\|%{DATA:ClassName}\.%{DATA:FunctionName}(:%{NUMBER:LineNumber})?] *- %{GREEDYDATA:LogMessage}'
      }
    }
    date {
      match => ["timestamp", "dd MMM yyyy HH:mm:ss.SSS"]
      target=>@timestamp
         }
    mutate {
      add_field => {
        Component => "%{[fields][Component]}"
      }
      gsub => ["Category", "TSK", "TASKS"]
      gsub => ["Category", "RST", "REST"]
      gsub => ["Category", "ZNE", "ZONES"]
      gsub => ["Category", "DSH", "DASHBOARD"]
      gsub => ["Category", "HST", "HISTORY"]
      gsub => ["Category", "SCD", "SCHEDULES"]
      gsub => ["Category", "IMP", "IMPORT"]
      gsub => ["Category", "CLP", "CLEANUP"]
      gsub => ["Category", "TSK", "TASK"]
      gsub => ["Category", "ENTEXT", "ENTRYEXIT"]
      gsub => ["Category", "IMPVRB", "IMPORTVERBOSE"]
      gsub => ["LogLevel", "FST", "FINEST"]
      gsub => ["LogLevel", "FNR", "FINER"]
      gsub => ["LogLevel", "FNE", "FINE"]
      gsub => ["LogLevel", "IFO", "INFO"]
      gsub => ["LogLevel", "WRN", "WARN"]
      gsub => ["LogLevel", "FTL", "FATAL"]
      gsub => ["LogLevel", "ERR", "ERROR"]
      gsub => ["Component", "CmpDir", "Campaign Director"]

 	}     
    }
    
  } 

```

This is my sample log  
FST-SCD 11 Dec 2022 07:39:50.527 [Cleanup-Thread|CleanupThread.cleanDanglingSchedules:] - CleanDanglingSchedules - inside for loop index=1, TriggerName=Trigger1, TriggerState=NORMAL

Currently it is showing the current date on the @timestamp field and not the log date  
Any help would be appreciated . Thanks Everyone

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [January 21, 2023, 6:35pm UTC](https://discuss.elastic.co/t/replacing-timestamp-with-logs-having-custom-timestamp/323656/2 "2023-01-21T18:35:09Z")

</div>

Grok pattern is OK, parsing is correct.  
The fields `TimeStamp` and `timestamp` are different, the naming is case sensitive.

```auto
      date {
        match => ["TimeStamp", "dd MMM yyyy HH:mm:ss.SSS"]
        # timezone => "Europe/Berlin"
        # target=>"@timestamp" No need, it's default
      }

```

Result:  
`"@timestamp" => 2022-12-11T06:39:50.527Z`

Optionally you can set your time zone. Default TZ is from the LS host. It's useful in cloud environments.

---

<div class="post-metadata">

**Author:** ![sahoo35](https://avatars.discourse-cdn.com/v4/letter/s/c6cbf5/32.png) [@sahoo35](https://discuss.elastic.co/u/sahoo35)\
**Post date:** [January 21, 2023, 7:37pm UTC](https://discuss.elastic.co/t/replacing-timestamp-with-logs-having-custom-timestamp/323656/3 "2023-01-21T19:37:14Z")

</div>

Thanks for the reply, I have tried that using TimeStamp then it does not display anything on the kibana dashboard at all.

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [January 21, 2023, 7:55pm UTC](https://discuss.elastic.co/t/replacing-timestamp-with-logs-having-custom-timestamp/323656/4 "2023-01-21T19:55:55Z")

</div>

TimeStamp is a string.  
If you want, use `target=>"@TimeStamp"` for but TimeStamp as date. You have to recreate index pattern or just leave @timestamp as the _Timestamp_ field in Kibana.

---

<div class="post-metadata">

**Author:** ![sahoo35](https://avatars.discourse-cdn.com/v4/letter/s/c6cbf5/32.png) [@sahoo35](https://discuss.elastic.co/u/sahoo35)\
**Post date:** [January 21, 2023, 8:11pm UTC](https://discuss.elastic.co/t/replacing-timestamp-with-logs-having-custom-timestamp/323656/5 "2023-01-21T20:11:02Z")

</div>

But @timestamp is not showing the correct date and time if i leave it as it is.

Also if i use target=\> @TimeStamp will it convert it to date ?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 21, 2023, 8:59pm UTC](https://discuss.elastic.co/t/replacing-timestamp-with-logs-having-custom-timestamp/323656/6 "2023-01-21T20:59:55Z")

</div>

> [@sahoo35](#):
>
> I have tried that using TimeStamp then it does not display anything on the kibana dashboard at all.

Perhaps that is because the time range on the dashboard does not extend far back enough to show early December. That may indicate it is actually working.

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [January 21, 2023, 10:44pm UTC](https://discuss.elastic.co/t/replacing-timestamp-with-logs-having-custom-timestamp/323656/7 "2023-01-21T22:44:15Z")

</div>

As Badger said, send the screen from Kibana where is data is not OK and how the message looks like.

---

<div class="post-metadata">

**Author:** ![sahoo35](https://avatars.discourse-cdn.com/v4/letter/s/c6cbf5/32.png) [@sahoo35](https://discuss.elastic.co/u/sahoo35)\
**Post date:** [January 22, 2023, 10:22am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-logs-having-custom-timestamp/323656/8 "2023-01-22T10:22:39Z")

</div>

OHHHH....now i get it, so dumb of me to not check that before hand. Thanks @Rios and @Badger for helping i got what i wanted.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 19, 2023, 10:23am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-logs-having-custom-timestamp/323656/9 "2023-02-19T10:23:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.