# @timestamp cannot be deleted - overwriting does not work

**URL:** <https://discuss.elastic.co/t/timestamp-cannot-be-deleted-overwriting-does-not-work/93512>\
**Category:** Logstash\
**Created:** [July 18, 2017, 7:02am UTC](https://discuss.elastic.co/t/timestamp-cannot-be-deleted-overwriting-does-not-work/93512 "2017-07-18T07:02:12Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![dschmitz](https://avatars.discourse-cdn.com/v4/letter/d/a9adbd/32.png) [@dschmitz](https://discuss.elastic.co/u/dschmitz)\
**Post date:** [July 18, 2017, 7:02am UTC](https://discuss.elastic.co/t/timestamp-cannot-be-deleted-overwriting-does-not-work/93512/1 "2017-07-18T07:02:12Z")

</div>

Hi,  
I know, that the field @timestamp can not be deleted. So I want to override @timestamp with the timestamp of logs. I've created a new field. The timestamp in logfile looks like "13.07.2017 07:11:10 ..." my new field LOGTIMESTAMP "13.07.2017 07:11:10", but @timestamp "17.07.2017 11:29:24".  
I parse the logfile in this way -\>  
grok {  
match =\> ["message", "%{DATESTAMP:LOGTIMESTAMP}"]  
}  
date {  
match =\> ["LOGTIMESTAMP", "dd.MMM.YYYY hh:mm:ss"]  
target =\> "@timestamp"  
locale =\> "de"  
remove\_field =\> ["LOGTIMESTAMP"]  
}  
What am I doing wrong? Thanks for your help.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 18, 2017, 7:07am UTC](https://discuss.elastic.co/t/timestamp-cannot-be-deleted-overwriting-does-not-work/93512/2 "2017-07-18T07:07:20Z")

</div>

If the date filter fails the log contains clues. I can immediately see that MMM in your date pattern should be MM and, unless you use a 12-hour clock with no am/pm marker, hh should be HH.

---

<div class="post-metadata">

**Author:** ![dschmitz](https://avatars.discourse-cdn.com/v4/letter/d/a9adbd/32.png) [@dschmitz](https://discuss.elastic.co/u/dschmitz)\
**Post date:** [July 18, 2017, 8:10am UTC](https://discuss.elastic.co/t/timestamp-cannot-be-deleted-overwriting-does-not-work/93512/3 "2017-07-18T08:10:30Z")

</div>

Thanks for your answer. I've made the adjustments as suggested, without success. I can not get any error messages from the logfiles. Kibana shows me an \_dateparsefailure, otherwise no further error-informations ...

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 18, 2017, 10:03am UTC](https://discuss.elastic.co/t/timestamp-cannot-be-deleted-overwriting-does-not-work/93512/4 "2017-07-18T10:03:03Z")

</div>

Works fine for me with both Logstash 2.3 and 5.4. Please copy/paste an example event from Kibana. Use the JSON tab in the Discover view so we get the raw JSON representation.

```nohighlight
$ cat test.config
input { stdin { } }
output { stdout { codec => rubydebug } }
filter {
  date {
    match => ["message", "dd.MM.YYYY HH:mm:ss"]
    locale => "de"
  }
}
$ echo '13.07.2017 07:11:10' | logstash -f test.config
Settings: Default pipeline workers: 8
Pipeline main started
{
       "message" => "13.07.2017 07:11:10",
      "@version" => "1",
    "@timestamp" => "2017-07-13T05:11:10.000Z",
          "host" => "bertie"
}
Pipeline main has been shutdown
stopping pipeline {:id=>"main"}

```

---

<div class="post-metadata">

**Author:** ![dschmitz](https://avatars.discourse-cdn.com/v4/letter/d/a9adbd/32.png) [@dschmitz](https://discuss.elastic.co/u/dschmitz)\
**Post date:** [July 18, 2017, 10:47am UTC](https://discuss.elastic.co/t/timestamp-cannot-be-deleted-overwriting-does-not-work/93512/5 "2017-07-18T10:47:24Z")

</div>

If i reproduce your example, I get the same (correct) result. But only for manual execution.

Here my JSON-Output:

{  
"\_index": "filebeat-2017.07.18",  
"\_type": "test",  
"\_id": "AV1UtBnpzzBeDCd4SQD4",  
"\_score": null,  
"\_source": {  
"HOST": "test.lan",  
"PID": "20896",  
"message": "13.07.2017 00:51:10 [20896] [INFO] [main::() line(63)] staging dirs:",  
"tags": [  
"\_dateparsefailure"  
],  
"@timestamp": "2017-07-18T07:57:34.263Z",  
"LOGMESSAGE": "staging dirs:",  
"PRODUCT": "test",  
"beat": {},  
"SOURCE": "main::() line(63)",  
"INPUT\_TYPE": "log",  
"FILE": "/tmp/test.log",  
"LOGTIMESTAMP": "13.07.2017 00:51:10",  
"TYPE": "test"  
},  
"fields": {  
"@timestamp": [  
1500364654263  
]  
},  
"sort": [  
1500364654263  
]  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 18, 2017, 1:57pm UTC](https://discuss.elastic.co/t/timestamp-cannot-be-deleted-overwriting-does-not-work/93512/6 "2017-07-18T13:57:52Z")

</div>

I can't reproduce.

```nohighlight
$ cat data
13.07.2017 00:51:10 [20896] [INFO] [main::() line(63)] staging dirs:
$ cat test.config 
input { stdin { } }
output { stdout { codec => rubydebug } }
filter {
  grok {
    match => ["message", "%{DATESTAMP:LOGTIMESTAMP}"]
  }
  date {
    match => ["LOGTIMESTAMP", "dd.MM.YYYY HH:mm:ss"]
    locale => "de"
  }
}
$ logstash -f test.config < data 
Sending Logstash's logs to /home/magnus/logstash/logstash-5.4.1/logs which is now configured via log4j2.properties
[2017-07-18T15:56:07,689][INFO][logstash.pipeline] Starting pipeline {"id"=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>1000}
[2017-07-18T15:56:07,715][INFO][logstash.pipeline] Pipeline main started
[2017-07-18T15:56:07,765][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
{
      "@timestamp" => 2017-07-12T22:51:10.000Z,
        "@version" => "1",
            "host" => "bertie",
         "message" => "13.07.2017 00:51:10 [20896] [INFO] [main::() line(63)] staging dirs:",
    "LOGTIMESTAMP" => "13.07.2017 00:51:10"
}
[2017-07-18T15:56:10,733][WARN][logstash.agent] stopping pipeline {:id=>"main"}

```

I find it very hard to believe that the date filter is silent about why it's failing.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 15, 2017, 1:58pm UTC](https://discuss.elastic.co/t/timestamp-cannot-be-deleted-overwriting-does-not-work/93512/7 "2017-08-15T13:58:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.