# Updating the @timestamp is not working

**URL:** <https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192>\
**Category:** Logstash\
**Created:** [June 17, 2018, 4:22am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192 "2018-06-17T04:22:54Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pradana](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@Pradana](https://discuss.elastic.co/u/Pradana)\
**Post date:** [June 17, 2018, 4:22am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/1 "2018-06-17T04:22:54Z")

</div>

I am trying to updating the timestamp using below :

date {  
match =\> ["smsdate", "dd/MMM/yyyy:HH:mm:ss Z"]  
target =\> ["@timestamp"]  
}

but in kibana still showing the current date. How to make it right ? my objective is to using smsdate in @timestamp

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 17, 2018, 7:27am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/2 "2018-06-17T07:27:39Z")

</div>

`@timestamp` is the default target field, so you should not need to specify this. Do you have a `smsdate` field at the time you call the date filter? If so, what is the format of this field?

---

<div class="post-metadata">

**Author:** ![Pradana](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@Pradana](https://discuss.elastic.co/u/Pradana)\
**Post date:** [June 17, 2018, 8:12am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/3 "2018-06-17T08:12:32Z")

</div>

smsdate format is a text. how to change it to date ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 17, 2018, 8:20am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/4 "2018-06-17T08:20:05Z")

</div>

Can you show us what the contents of the field looks like?

---

<div class="post-metadata">

**Author:** ![Pradana](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@Pradana](https://discuss.elastic.co/u/Pradana)\
**Post date:** [June 17, 2018, 8:41am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/5 "2018-06-17T08:41:37Z")

</div>

{  
"\_index": "filebeat-2018.06.17",  
"\_type": "doc",  
"\_id": "eocxDGQBruHXTGIb-XoF",  
"\_version": 1,  
"\_score": null,  
"\_source": {  
"destaddr": "628285333724631",  
"@timestamp": "2018-06-17T05:21:32.834Z",  
"smstype": "Incoming",  
"input": {  
"type": "log"  
},  
"sourcenpi": "1",  
"operator\_name": "Telkomsel",  
"sourceaddr": "6282340713802",  
"logdate": "2018-06-11 23:56:45,972",  
"sourceton": "1",  
"operator": "62823",  
"host": {  
"name": "smsclog"  
},  
"status": "success\_esme",  
"offset": 1887268,  
"@version": "1",  
"prospector": {  
"type": "log"  
},  
"source": [  
"/home/pradana/Data/cdr.log.2018-06-11",  
"[org.mobicents.smsc.library.CdrGenerator]"  
],  
"fields": {  
"sourcelog": "cdrlog"  
},  
"_smsdate_": "2018-06-11 23:56:45.916",  
"tags": [  
"beats\_input\_codec\_plain\_applied",  
"\_dateparsefailure"  
],  
"addrnpi": "1",  
"debugtype": "DEBUG",  
"message": "2018-06-11 23:56:45,972 DEBUG [org.mobicents.smsc.library.CdrGenerator] 2018-06-11 23:56:45.916,6282340713802,1,1,628285333724631,1,1,success\_esme,SS7\_HR,message,null,103791,0,null,null,null,null,6281107908,null,0,15,null,0,0,,,,5,"Sepi bae ndk arak ce","",,,",  
"addrton": "1",  
"beat": {  
"version": "6.3.0",  
"hostname": "localhost.localdomain",  
"name": "smsclog"  
}  
},  
"fields": {  
"@timestamp": [  
"2018-06-17T05:21:32.834Z"  
]  
},  
"sort": [  
1529212892834  
]  
}

Please see above :

smsdate : "2018-06-11 23:56:45.916"

---

<div class="post-metadata">

**Author:** ![Pradana](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@Pradana](https://discuss.elastic.co/u/Pradana)\
**Post date:** [June 17, 2018, 8:45am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/6 "2018-06-17T08:45:53Z")

</div>

btw I dont have the smsdate field before i call the date filter.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 17, 2018, 8:55am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/7 "2018-06-17T08:55:46Z")

</div>

You have to have the `smsdate` field parsed before you can use it. The format also do not match what you have specified in the date filter, which means it will fail even if you have it.

---

<div class="post-metadata">

**Author:** ![Pradana](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@Pradana](https://discuss.elastic.co/u/Pradana)\
**Post date:** [June 17, 2018, 9:23am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/8 "2018-06-17T09:23:41Z")

</div>

I have change it to logdate with having this content : "logdate": "2018-06-11 00:00:29,900"

on the grok i put like below : grok {  
match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:logdate:date} %{LOGLEVEL:debugtype} %{DATA:source} %{TIMESTAMP\_ISO8601:smsdate},%{WORD:sourceaddr},%{NUMBER:addrton},%{NUMBER:addrnpi},%{WORD:destaddr},%{NUMBER:sourceton},%{NUMBER:  
sourcenpi},%{WORD:status}" }}

here i am expecting the logdate should be in the type of date but in the mapping, it still showing me type of text:

"logdate": {  
"type": "text",

and the timestamp also still refer to current date.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 17, 2018, 9:31am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/9 "2018-06-17T09:31:08Z")

</div>

Please show what the resulting event looks like. Did you use the date filter on that field?

---

<div class="post-metadata">

**Author:** ![Pradana](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@Pradana](https://discuss.elastic.co/u/Pradana)\
**Post date:** [June 17, 2018, 9:50am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/10 "2018-06-17T09:50:31Z")

</div>

Below is the JSON result :

{  
"\_index": "filebeat-2018.06.17",  
"\_type": "doc",  
"\_id": "AYcKDWQBruHXTGIbKI0t",  
"\_version": 1,  
"\_score": 1,  
"\_source": {  
"logdate": "2018-06-11 02:52:48,889",  
"destaddr": "62828987319211",  
"smstype": "Incoming",  
"debugtype": "DEBUG",  
"message": "2018-06-11 02:52:48,889 DEBUG [org.mobicents.smsc.library.CdrGenerator] 2018-06-11 02:52:48.873,6281390136127,1,1,62828987319211,1,1,success\_esme,SS7\_HR,message,null,100752,0,null,null,null,null,6281107908,null,0,15,null,0,0,,,,3,"Ibu baru sampe karaw","",,,",  
"source": [  
"/home/pradana/Data/cdr.log.2018-06-11",  
"[org.mobicents.smsc.library.CdrGenerator]"  
],  
"addrnpi": "1",  
"operator\_name": "Telkomsel",  
"host": {  
"name": "smsclog"  
},  
"sourceaddr": "6281390136127",  
"smsdate": "2018-06-11 02:52:48.873",  
"status": "success\_esme",  
"beat": {  
"version": "6.3.0",  
"name": "smsclog",  
"hostname": "localhost.localdomain"  
},  
"@version": "1",  
"fields": {  
"sourcelog": "cdrlog"  
},  
"@timestamp": "2018-06-17T09:17:01.722Z",  
"sourcenpi": "1",  
"operator": "62813",  
"input": {  
"type": "log"  
},  
"sourceton": "1",  
"addrton": "1",  
"offset": 164237,  
"tags": [  
"beats\_input\_codec\_plain\_applied",  
"\_dateparsefailure"  
],  
"prospector": {  
"type": "log"  
}  
},  
"fields": {  
"@timestamp": [  
"2018-06-17T09:17:01.722Z"  
]  
}  
}

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 17, 2018, 10:03am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/11 "2018-06-17T10:03:14Z")

</div>

`logdate` is mapped as a string in Elasticsearch as it is not in a format that allows dynamic mapping to identify it as a date field. If you use the `date` filter with a correct format it will set the correct format for you.

---

<div class="post-metadata">

**Author:** ![Pradana](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@Pradana](https://discuss.elastic.co/u/Pradana)\
**Post date:** [June 17, 2018, 10:13am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/12 "2018-06-17T10:13:09Z")

</div>

As i told you .. during processing below is my grok :

grok {  
match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:logdate:date} %{LOGLEVEL:debugtype} %{DATA:source} %{TIMESTAMP\_ISO8601:smsdate},%{WORD:sourceaddr},%{NUMBER:addrton},%{NUMBER:addrnpi},%{WORD:destaddr},%{NUMBER:sourceton},%{NUMBER:  
sourcenpi},%{WORD:status}" }}

but somehow the logdate still have the format of "Text" not date. How to change this ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 17, 2018, 10:15am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/13 "2018-06-17T10:15:23Z")

</div>

You need to use the date filter to format it correctly. After that you will need to start with a fresh index as existing mappings can not be updated.

---

<div class="post-metadata">

**Author:** ![Pradana](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@Pradana](https://discuss.elastic.co/u/Pradana)\
**Post date:** [June 17, 2018, 10:40am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/14 "2018-06-17T10:40:01Z")

</div>

Can you elaborate how to use the date filter correctly ? I have refresh the data many time but still not giving the desire result

---

<div class="post-metadata">

**Author:** ![Pradana](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@Pradana](https://discuss.elastic.co/u/Pradana)\
**Post date:** [June 17, 2018, 11:01am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/15 "2018-06-17T11:01:52Z")

</div>

after changing the date filter into this ...its working ..

date {  
match =\> ["logdate", "ISO8601"]  
target =\> ["@timestamp"]  
}

the timestamp pointing to correct date but the logdate still on string type

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 17, 2018, 11:02am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/16 "2018-06-17T11:02:15Z")

</div>

The example in your initial post is close, but the format `"dd/MMM/yyyy:HH:mm:ss Z"` does not match `"2018-06-11 02:52:48,889"`. Try using the format `yyyy-MM-dd HH:mm:ss,SSS` instead.

---

<div class="post-metadata">

**Author:** ![Pradana](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@Pradana](https://discuss.elastic.co/u/Pradana)\
**Post date:** [June 17, 2018, 11:05am UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/17 "2018-06-17T11:05:19Z")

</div>

Thanks christian. I am using ISO8601 and its working but the logdate still on the type of string not date.  
How to fix this ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 17, 2018, 2:13pm UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/18 "2018-06-17T14:13:41Z")

</div>

Have a look at this example:

```auto
input {
  generator {
    lines => ['2018-06-11 23:56:45,972']
    count => 1
  } 
} 

filter {
    date {
        match => ["message", "yyyy-MM-dd HH:mm:ss,SSS"]
    }
}

output {
  stdout { codec => rubydebug }
}

```

Change the `target` field as required.

---

<div class="post-metadata">

**Author:** ![Pradana](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@Pradana](https://discuss.elastic.co/u/Pradana)\
**Post date:** [June 17, 2018, 6:26pm UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/19 "2018-06-17T18:26:34Z")

</div>

After making below change in my logstash configuration :

date {  
match =\> ["smsdate", "ISO8601"]  
target =\> ["@timestamp"]  
}

```
date {
  match => ["smsdate", "ISO8601"]
  target => ["smsdate"]
}

```

everything is working fine as expected. timestamp follow the event date, and the smsdate field type is date.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 15, 2018, 6:26pm UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192/20 "2018-07-15T18:26:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.