# Parsing metricbeat json log in filebeat -\> got processing timestamp, not log timestamp

**URL:** <https://discuss.elastic.co/t/parsing-metricbeat-json-log-in-filebeat-got-processing-timestamp-not-log-timestamp/76610>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 27, 2017, 11:08am UTC](https://discuss.elastic.co/t/parsing-metricbeat-json-log-in-filebeat-got-processing-timestamp-not-log-timestamp/76610 "2017-02-27T11:08:11Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![asp](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@asp](https://discuss.elastic.co/u/asp)\
**Post date:** [February 27, 2017, 11:08am UTC](https://discuss.elastic.co/t/parsing-metricbeat-json-log-in-filebeat-got-processing-timestamp-not-log-timestamp/76610/1 "2017-02-27T11:08:11Z")

</div>

Hi,

I use following workstream:

metricbeat for gathering metrics -\> using file output to buffer on disk -\> transfer the log via filebeat to logstash -\> do further enrichment -\> send to elasticsearch.

Here is my filebeat prospector configuration for the metricbeat log:

```
# metricbeat probes
- input_type: log

  # Paths that should be crawled and fetched. Glob based paths.
  # To fetch all ".log" files from a specific level of subdirectories
  # /var/log/*/*.log can be used.
  # For each file found under this path, a harvester is started.
  # Make sure not file is defined twice as this can lead to unexpected behaviour.
  paths:
    - /var/log/metricbeat/metricbeat_probes.log

  encoding: utf-8
  ignore_older: 24h
  document_type: metricsets
  scan_frequency: 10s
  symlinks: true
  
# json.message_key: message
  json.keys_under_root: true

```

here is an example log line of the metricbeat log:

```
{"@timestamp":"2017-02-27T10:55:14.839Z","beat":{"hostname":"xxxx","name":"xxxx","version":"5.2.1"},"hostName":"xxxx","metricset":{"module":"system","name":"process","rtt":25517},"serverType":"control","stage":"Production","system":{"process":{"cmdline":"/usr/share/metricbeat/bin/metricbeat -c /etc/metricbeat/metricbeat.yml -path.home /usr/share/metricbeat -path.config /etc/metricbeat -path.data /var/lib/metricbeat -path.logs /var/log/metricbeat","cpu":{"start_time":"2017-02-27T10:04:43.000Z","total":{"pct":0.001300}},"fd":{"limit":{"hard":4096,"soft":1024},"open":7},"memory":{"rss":{"bytes":12972032,"pct":0.001600},"share":5967872,"size":499920896},"name":"metricbeat","pgid":28560,"pid":28560,"ppid":1,"state":"sleeping","username":"root"}},"type":"metricsets"}

```

Now I encountered the issue, that the @timestamp is not correct. Some processing time is used.

There are no error tags on the event, just the normal "beats\_input\_raw\_event".

Any Ideas? What is my mistake?

Thanks, Andreas

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [February 27, 2017, 4:26pm UTC](https://discuss.elastic.co/t/parsing-metricbeat-json-log-in-filebeat-got-processing-timestamp-not-log-timestamp/76610/2 "2017-02-27T16:26:57Z")

</div>

See `json.overwrite_keys` in the [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#config-json). I think you need to have this set in order to allow `@timestamp` from the Filebeat event to be overwritten by the decoded `@timestamp` value.

---

<div class="post-metadata">

**Author:** ![asp](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@asp](https://discuss.elastic.co/u/asp)\
**Post date:** [February 28, 2017, 10:23am UTC](https://discuss.elastic.co/t/parsing-metricbeat-json-log-in-filebeat-got-processing-timestamp-not-log-timestamp/76610/3 "2017-02-28T10:23:07Z")

</div>

many thanks. That solved the problem.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 28, 2017, 10:23am UTC](https://discuss.elastic.co/t/parsing-metricbeat-json-log-in-filebeat-got-processing-timestamp-not-log-timestamp/76610/4 "2017-03-28T10:23:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.