Filebeat - json - elastic search - decoding time stamps

Elastic Stack Beats
1

I have setup the filebeat to forward json logfiles to the elasticsearch server. Here is my filebeat configuration:

filebeat.prospectors:
- input_type: log
  paths:
    - /data/log/1.log
  json.keys_under_root: true
  json.add_error_key: true
  tail_files: false
  close_removed: true
  clean_removed: true

output.elasticsearch:
  hosts: ["http://XXXXX:9200"]
  template.enabled: true
  template.path: "/etc/filebeat/filebeat.template.json"
  template.overwrite: false
  index: "filebeat"

Everything works and the logs are parsed and stored in elasticsearch, however, the @timestamp of the elastic search is not the same as the timestamp in the file.

Here is the first line of the log file:
{"time":"2017-10-26T12:55:32.772000Z", "field1":"value1"}

and here is the entry in elasticsearch:

@timestamp:October 31st 2017, 21:54:21.687 beat.hostname:XXX beat.name:XXX beat.version:5.6.3 field1:value1 input_type:log offset:63 source:/data/log/1.log time:2017-10-26T12:55:32.772000Z type:log _id:AV91SWr9ZOS8mHYUUnB1 _type:doc _index:filebeat _score: -

Is it possible to have elasticsearch load the timestamp from the log file itself?

2

You can do this through an ingest pipeline containing a date processor.

3

This topic was automatically closed after 21 days. New replies are no longer allowed.