# Filebeat touches my @timestamp even I've explicitly said don't do this!

**URL:** https://discuss.elastic.co/t/filebeat-touches-my-timestamp-even-ive-explicitly-said-dont-do-this/235127
**Category:** Beats
**Tags:** filebeat
**Created:** [June 1, 2020, 8:09am UTC](https://discuss.elastic.co/t/filebeat-touches-my-timestamp-even-ive-explicitly-said-dont-do-this/235127 "2020-06-01T08:09:33Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![alchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alchy/32/47743_2.png) [@alchy](https://discuss.elastic.co/u/alchy)
#### Post date: [June 1, 2020, 8:09am UTC](https://discuss.elastic.co/t/filebeat-touches-my-timestamp-even-ive-explicitly-said-dont-do-this/235127/1 "2020-06-01T08:09:34Z")

</div>

Hi,

thank you for reading, the story goes:

1. generated **event** in file  
`{"@timestamp": "20200601T070018-0100", "src_type": ""...} ... ... ... all the lines with in the same @timestamp format`

2. **filebeat** 7.7 grabs file on the host and sends content to logstash, I don't need filebeat even barely [touches] my data (filebeat.yml):

3. **logstash** 7.7 gets the data, no rules for @timestap, I don't need logstash touches my data either, especially the @timestamp, I suppose this field should not be modified when no date filter...

4. elasticsearch has the respective field set as [basic\_date\_time\_no\_millis](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html), the source format fits, even manual ingestion via POST works, but the format gets garbled somewhere on the road:

5. The indexing error from logstash-plain.log:  
'2020-06-01T07:40:08.669Z'", "caused\_by"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"failed to parse date field [2020-06-01T07:40:08.669Z] with format [basic\_date\_time\_no\_millis]", "caused\_by"=\>{"type"=\>"date\_time\_parse\_exception", "reason"=\>"date\_time\_parse\_exception: Failed to parse with all enclosed parsers"}}}}}}

Who and why WTF manipulates my @timestamp when I've explicitly said don't do this? How can I be more explicit?

any idea will be appreciated  
thanks

---

<div class="post-metadata">

### Author: ![alchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alchy/32/47743_2.png) [@alchy](https://discuss.elastic.co/u/alchy)
#### Post date: [June 1, 2020, 8:37am UTC](https://discuss.elastic.co/t/filebeat-touches-my-timestamp-even-ive-explicitly-said-dont-do-this/235127/2 "2020-06-01T08:37:45Z")

</div>

filebeat does that! debug output to file proves that.  
I've not idea how to prevent this when overwrite\_keys: false is set already...

---

<div class="post-metadata">

### Author: ![alchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alchy/32/47743_2.png) [@alchy](https://discuss.elastic.co/u/alchy)
#### Post date: [June 1, 2020, 9:22am UTC](https://discuss.elastic.co/t/filebeat-touches-my-timestamp-even-ive-explicitly-said-dont-do-this/235127/3 "2020-06-01T09:22:59Z")

</div>

> [@alchy](#):
>
> "20200601T070018-0100"

according to [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/processor-timestamp.html), there is the processor for timestamp, which should enable tweaking logstash's @timestamp behaviour. according to documentation, this should work (in plain, please use the @timestamp, but write your modification elsewhere):

```
processors:
  - timestamp:
      field: @timestamp
      target_field: @filebeat_timestamp
      layouts: 
        - "Mon Jan 2 15:04:05 MST 2006"

```

but this won't work, the logstash won't start  
as a bonus, seems nothing relevant is logged

---

<div class="post-metadata">

### Author: ![alchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alchy/32/47743_2.png) [@alchy](https://discuss.elastic.co/u/alchy)
#### Post date: [June 1, 2020, 11:41am UTC](https://discuss.elastic.co/t/filebeat-touches-my-timestamp-even-ive-explicitly-said-dont-do-this/235127/4 "2020-06-01T11:41:33Z")

</div>

> [@alchy](#):
>
> 20200601T070018-0100

the best answer yet seems to be avoid using @timestamp in your data while transfering them with filebeat. even if you manage filebeat won't touch your data, there are other metadata with @timestamp around your data and it seems those metadata are used while indexing in the elastic as the primary @timestamp. these @timestap/s are related to time, when filebeat reads the file from disk.

---

<div class="post-metadata">

### Author: ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [June 2, 2020, 2:21pm UTC](https://discuss.elastic.co/t/filebeat-touches-my-timestamp-even-ive-explicitly-said-dont-do-this/235127/5 "2020-06-02T14:21:05Z")

</div>

Indeed, `@timestamp` is reserved field name in Beats. That's where the time of reading is saved when reading events from arbitrary sources.

However, if you use the `json.*` option in the `log` input, you can overwrite the value in `@timestamp`. See more: [https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html#filebeat-input-log-config-json](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html#filebeat-input-log-config-json)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 30, 2020, 2:30pm UTC](https://discuss.elastic.co/t/filebeat-touches-my-timestamp-even-ive-explicitly-said-dont-do-this/235127/6 "2020-06-30T14:30:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
