# Logstash cannot assign correct date to log timestamp

**URL:** <https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249>\
**Category:** Logstash\
**Created:** [June 13, 2017, 5:30pm UTC](https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249 "2017-06-13T17:30:52Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![John\_06](https://avatars.discourse-cdn.com/v4/letter/j/7c8e57/32.png) [@John\_06](https://discuss.elastic.co/u/John_06)\
**Post date:** [June 13, 2017, 5:30pm UTC](https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249/1 "2017-06-13T17:30:53Z")

</div>

Could anybody clarify if it's possible to add date part to log timestamp in Logstash not in 'UTC' but in servers time zone.

For example, if I have log record timestamp like this (without date part):

`21:17:43,124 INFO [...`

and I want to add todays date to every log record timestamp and assign that value to `@timestamp` field.

```
filter {
    grok {
      match => {"message" => "(?<logTime>[0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3}) "}
    }
    mutate {
      add_field => {"logTimestamp" => "%{+YYYY-MM-dd} %{logTime}"}
    }
    date {
      match => ["logTimestamp", "YYYY-MM-dd HH:mm:ss,SSS"]
      timezone => "..."
    }
}

```

However, it will add date in 'UTC' time zone and if host machine is in different time zone, `@timestamp` field will show tomorrow date instead of today.

Can it be Logstash bug?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 13, 2017, 6:22pm UTC](https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249/2 "2017-06-13T18:22:29Z")

</div>

> Could anybody clarify if it's possible to add date part to log timestamp in Logstash not in 'UTC' but in servers time zone.

No, that's not possible. `@timestamp` is always UTC.

> and I want to add todays date to every log record timestamp and assign that value to `@timestamp` field.

That assumes that you always process the file in real time on the exact same day that the log record was produced. If that really is an okay assumption to make you can use a ruby filter to produce today's date in your local timezone.

---

<div class="post-metadata">

**Author:** ![John\_06](https://avatars.discourse-cdn.com/v4/letter/j/7c8e57/32.png) [@John\_06](https://discuss.elastic.co/u/John_06)\
**Post date:** [June 13, 2017, 7:04pm UTC](https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249/3 "2017-06-13T19:04:42Z")

</div>

@magnusbaeck Thanks.

Is this correct config:

```
filter {
    grok {
      match => {"message" => "(?<logTime>[0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3}) "}
    }
    ruby {
      code => "event['logTimestamp'] = Time.now.strftime('%Y-%m-%d') + ' ' + event['logTime']"
    }
    date {
      match => ["logTimestamp", "YYYY-MM-dd HH:mm:ss,SSS"]
      timezone => "..."
    }
}

```

I replaced `mutate` filter with `ruby` one to get the current date of the log.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 13, 2017, 7:19pm UTC](https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249/4 "2017-06-13T19:19:14Z")

</div>

That looks okay for Logstash 1.x and 2.x but for 5.x and later you need to use the new event API ([https://www.elastic.co/guide/en/logstash/current/event-api.html](https://www.elastic.co/guide/en/logstash/current/event-api.html)).

---

<div class="post-metadata">

**Author:** ![John\_06](https://avatars.discourse-cdn.com/v4/letter/j/7c8e57/32.png) [@John\_06](https://discuss.elastic.co/u/John_06)\
**Post date:** [June 13, 2017, 8:01pm UTC](https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249/5 "2017-06-13T20:01:48Z")

</div>

I think ruby filter should look like this:

```
ruby {
  code => 'event.set("logTimestamp", Time.now.strftime('%Y-%m-%d') + ' ' + event.get("logTime"))'
}

```

But Logstash complains about `Expected one of #, {, }` for date pattern. Any hints what can be wrong?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 13, 2017, 8:06pm UTC](https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249/6 "2017-06-13T20:06:44Z")

</div>

Can't tell without seeing the whole configuration, but you can try commenting out pieces of it to narrow the problem down.

---

<div class="post-metadata">

**Author:** ![John\_06](https://avatars.discourse-cdn.com/v4/letter/j/7c8e57/32.png) [@John\_06](https://discuss.elastic.co/u/John_06)\
**Post date:** [June 13, 2017, 8:15pm UTC](https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249/7 "2017-06-13T20:15:50Z")

</div>

This is complete filter section:

```
filter {
  grok {
    match => {"message" => "(?<logTime>[0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3})"}
  }
  ruby {
    code => 'event.set("logTimestamp", Time.now.strftime('%Y-%m-%d') + ' ' + event.get("logTime"))'
  }
  date {
    match => ["logTimestamp", "YYYY-MM-dd HH:mm:ss,SSS"]
    timezone => "America/Chicago"
  }
}

```

And Logstash complains about this part`('%Y-%m-%d')`

I cannot find any examples in docs, not sure what could be wrong with syntax.

---

<div class="post-metadata">

**Author:** ![John\_06](https://avatars.discourse-cdn.com/v4/letter/j/7c8e57/32.png) [@John\_06](https://discuss.elastic.co/u/John_06)\
**Post date:** [June 14, 2017, 1:05am UTC](https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249/8 "2017-06-14T01:05:34Z")

</div>

I changed it to:

```
filter {
    grok {
      match => {"message" => "(?<logTime>[0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3})"}
    }
    ruby {
      code => 'event.set("currentDate", Time.now.strftime("%Y.%m.%d"))'
    }
    ruby {
      code => 'event.set("logTimestamp", event.get("currentDate") + " " + event.get("logTime"))'
    }
    date {
      match => ["logTimestamp", "YYYY.MM.dd HH:mm:ss,SSS"]
      timezone => "America/Chicago"
    }
}

```

but it still shows date in 'UTC' time zone.

I also tried this to get current date:

```
ruby {
  code => 'event.set("currentTime", Time.now.getlocal("-07:00"))'
}

```

but the date is in 'UTC' time zone.

What' is the correct way to get current local date?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 14, 2017, 5:32am UTC](https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249/9 "2017-06-14T05:32:24Z")

</div>

> And Logstash complains about this part `('%Y-%m-%d')`

Aha. Don't use single quotes within the single-quoted Ruby code snippet. Use double quotes everywhere within the string.

> but it still shows date in 'UTC' time zone.

As I said `@timestamp` is always UTC.

---

<div class="post-metadata">

**Author:** ![John\_06](https://avatars.discourse-cdn.com/v4/letter/j/7c8e57/32.png) [@John\_06](https://discuss.elastic.co/u/John_06)\
**Post date:** [June 14, 2017, 3:02pm UTC](https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249/10 "2017-06-14T15:02:41Z")

</div>

> As I said @timestamp is always UTC.

Does that mean there is no way to change `@timestamp` to another time zone?

I would like to see my logs time in my current timezone, not in UTC  
If the log was created today at `21:00:00,000` I don't want to add tomorrow date to its timestamp, It should have today date in timestamp.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 15, 2017, 5:57am UTC](https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249/11 "2017-06-15T05:57:10Z")

</div>

> Does that mean there is no way to change `@timestamp` to another time zone?

On the ES side it'll always be stored as UTC and that's what the date filter produces (not configurable). Reading the ES documentation for the date type it seems it accepts timestamps with a non-zero UTC offset, i.e. your documents can contain timestamps in local time as long as the offset is specified. I haven't tried it because I think UTC is a good thing that people should stop fighting.

> I would like to see my logs time in my current timezone, not in UTC

See _where_? Kibana converts the timestamps to the browser's timezone.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 13, 2017, 5:57am UTC](https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249/12 "2017-07-13T05:57:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.