# Best Method to Manipulate Date & TimeStamp

**URL:** <https://discuss.elastic.co/t/best-method-to-manipulate-date-timestamp/152992>\
**Category:** Logstash\
**Created:** [October 18, 2018, 12:22pm UTC](https://discuss.elastic.co/t/best-method-to-manipulate-date-timestamp/152992 "2018-10-18T12:22:37Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![londonx](https://avatars.discourse-cdn.com/v4/letter/l/da6949/32.png) [@londonx](https://discuss.elastic.co/u/londonx)\
**Post date:** [October 18, 2018, 12:22pm UTC](https://discuss.elastic.co/t/best-method-to-manipulate-date-timestamp/152992/1 "2018-10-18T12:22:37Z")

</div>

Hi all, looking for a way to efficiently convert an ISO8601 date/time stamp into the full ISO8601 format containing the "T" and "Z"

The log contains a date timestamp in the following format:

```
  2018-10-16 00:10:01.7764

```

ISO8601 grok pattern recognises the date & time stamp fields correctly:

```
  %{TIMESTAMP_ISO8601:timestamp}

{
"timestamp": [
 [
"2018-10-16 00:10:01.7764"
]
],
"YEAR": [
[
"2018"
]
],
"MONTHNUM": [
 [
"10"
]
],
"MONTHDAY": [
[
"16"
]
],
"HOUR": [
[
"00",
null
]
],
"MINUTE": [
[
"10",
null
]
],
"SECOND": [
[
"01.7764"
]
],
"ISO8601_TIMEZONE": [
[
null
]
]
}

```

So the captured field looks fine:

```
 "timestamp": "2018-09-13 16:00:52.1074",

```

Unfortunately the output demands the full ISO8601 format with 'T' separator and 'Z' timezone  
e.g.

```
 2018-10-16T00:10:01.7764Z

```

what would the most efficient way be of trying to massage the T and Z into the timestamp field? I am assuming I would still want Logstash to know it is a timestamp rather than end up just being a string!

There are two ways I can think of,

1. Use a filter \> mutate (will it still recognise the internal Grok variables from the previous step? Or would I have to split up the initial Grok pattern %{TIMESTAMP\_ISO8601:timestamp}" into something like %{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND} "

then I could mutate and add the T & Z:

```
 mutate {
    update => {
         "timestamp" => "%{YEAR}-%{MONTHNUM}-%{MONTHDAY}T%{HOUR}:%{MINUTE}:%{SECOND}Z"

```

Seems a tad convoluted and expensive though!

1. Use the dedicated "date" filter but there doesn't seem to be a way to convert it once captured unless I am missing something!

> ```
> date {
> match => ["timestamp", "yyyy-MM-dd HH:mm:ss.SSSS"]
> target => "timestamp2"
> 
> ```

Thanks for reading!

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [October 18, 2018, 1:02pm UTC](https://discuss.elastic.co/t/best-method-to-manipulate-date-timestamp/152992/3 "2018-10-18T13:02:10Z")

</div>

Use the date filter to convert is to a Logstash Timestamp object. If you want it to be an object in the output, you are done at this point. If you want it to be a String, just use

`mutate { convert => { "timestamp" => string}}`

The default string representation of timestamps in Logstash meets the ISO8601 criteria. (But it uses 3 digits for the fractional seconds as Logstash doesn't support any more precision. If that fourth digit is important to you, you'll have to stick to string manipulations, I guess.)

---

<div class="post-metadata">

**Author:** ![londonx](https://avatars.discourse-cdn.com/v4/letter/l/da6949/32.png) [@londonx](https://discuss.elastic.co/u/londonx)\
**Post date:** [October 18, 2018, 1:31pm UTC](https://discuss.elastic.co/t/best-method-to-manipulate-date-timestamp/152992/4 "2018-10-18T13:31:00Z")

</div>

Thanks yeah that's an interesting point, so I have gone with keeping the simple ISO8601 Grok pattern

```
%{TIMESTAMP_ISO8601:timestamp}

```

And used the date filter to convert it into the full ISO8601 output

```
filter {
  date {
    match => ["timestamp", "yyyy-MM-dd HH:mm:ss.SSSS"]
    target => "@timestamp"

```

So I'll keep both the "@timestamp" full ISO8601 field and retain the "timestamp" with its 4 digit fractional output just in case!

Seems fairly clean and not too much mutating!

Output:

```
"@timestamp":"2018-07-01T02:13:35.030Z"
"timestamp":"2018-07-01 02:13:35.0304"
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 15, 2018, 1:31pm UTC](https://discuss.elastic.co/t/best-method-to-manipulate-date-timestamp/152992/5 "2018-11-15T13:31:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.