# Timestamp in json logs **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [July 31, 2018, 8:00pm UTC](https://discuss.elastic.co/t/timestamp-in-json-logs/142438 "2018-07-31T20:00:52Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![PandKing](https://avatars.discourse-cdn.com/v4/letter/p/258eb7/32.png) [@PandKing](https://discuss.elastic.co/u/PandKing)\ **Post date:** [July 31, 2018, 8:00pm UTC](https://discuss.elastic.co/t/timestamp-in-json-logs/142438/1 "2018-07-31T20:00:52Z")
Hi, I am using Ossec [https://www.ossec.net/docs/](https://www.ossec.net/docs/) to log alerts locally to alerts.json which already has a timestamp in the log. Can I use this timestamp in the logstash output to elastic ? The input is below. All my beats aggregate to logstash then split or direct events to SUMO, Elastic or whatever is needed. {"timestamp":"2018-07-31T15:56:06.373-0400","rule":{"level":3,"description":"Windows: Service startup type was changed." ...., "blaa"} filebeat.inputs: - type: log json.keys\_under\_root: true json.add\_error\_key: true json.overwrite\_keys: true enabled: true paths: - /var/ossec/logs/alerts/alerts.json ---
**Author:** ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)\ **Post date:** [July 31, 2018, 8:44pm UTC](https://discuss.elastic.co/t/timestamp-in-json-logs/142438/2 "2018-07-31T20:44:46Z")
How is the alerts.json file being created in this scenario. By default Logstash will use the `@timestamp` field as the main timestamp associated with a given Event. ---
**Author:** ![PandKing](https://avatars.discourse-cdn.com/v4/letter/p/258eb7/32.png) [@PandKing](https://discuss.elastic.co/u/PandKing)\ **Post date:** [July 31, 2018, 8:48pm UTC](https://discuss.elastic.co/t/timestamp-in-json-logs/142438/3 "2018-07-31T20:48:35Z")
The file is generated directly from ossec/bin/ossec-analysisd daemon on a linux host. What I am seeing in Kibana and when I dump to file from Logstash are two timestamps. The second coming from logstash. ---
**Author:** ![PandKing](https://avatars.discourse-cdn.com/v4/letter/p/258eb7/32.png) [@PandKing](https://discuss.elastic.co/u/PandKing)\ **Post date:** [July 31, 2018, 8:57pm UTC](https://discuss.elastic.co/t/timestamp-in-json-logs/142438/4 "2018-07-31T20:57:06Z")
{ "\_index": "filebeat-6.3.2-2018.07.31", "\_type": "doc", "\_id": "6n0U8mQBl-NcLeOlLsVz", "\_version": 1, "\_score": null, "\_source": { "@timestamp": "2018-07-31T20:52:29.452Z", "tags": [ "MOC", "OSSEC", "beats\_input\_raw\_event" ], "predecoder": { "program\_name": "WinEvtLog", "timestamp": "2018 Jul 31 16:52:11", "hostname": "##################" }, "offset": 57557220, "timestamp": "2018-07-31T16:52:29.62-0400", "beat": { "version": "6.3.2", "hostname": "###############", "name": "################" }, "location": "WinEvtLog", "id": "1533070349.403263", "full\_log": "2018 Jul 31 16:52:11 WinEvtLog: System: INFORMATION(7040): Service Control Manager: SYSTEM: NT AUTHORITY: #####################: The start type of the Background Intelligent Transfer Service service was changed from auto start to demand start. ", "input": { "type": "log" }, "prospector": { "type": "log" }, "manager": { "name": "################" }, "data": { "dstuser": "SYSTEM", "data": "Service Control Manager", "id": "7040", "type": "System", "system\_name": "########################", "status": "INFORMATION" }, "source": "/var/ossec/logs/alerts/alerts.json", "@version": "1", "host": { "name": "############" }, "decoder": { "parent": "windows", "name": "windows" }, "agent": { "id": "114" }, "rule": { "groups": [ "windows", "policy\_changed", "hipaa\_164.308.a.3.ii.B", "hipaa\_164.308.a.4.i", "hipaa\_164.308.a.4.ii.B", "hipaa\_164.308.a.4.ii.C", "hipaa\_164.312.a.2.i", "hipaa\_164.312.a.2.iv", "nist\_800\_53\_ac-02", "hipaa\_164.308.a.5.ii.C", "hipaa\_164.312.b", "nist\_800\_53\_au-06", "hipaa\_164.308.a.1.ii.D", "hipaa\_164.312.b", "nist\_800\_53\_au-07", "hipaa\_164.308.a.5.ii.B", "nist\_800\_53\_si-04" ], "pci\_dss": [ "10.6" ], "gdpr": [ "2.3" ], "mail": false, "id": "18145", "level": 3, "firedtimes": 699, "description": "Windows: Service startup type was changed.", "info": "This does not appear to be logged on Windows 2000." } }, "fields": { "@timestamp": [ "2018-07-31T20:52:29.452Z" ] }, "sort": [ 1533070349452 ] } ---
**Author:** ![PandKing](https://avatars.discourse-cdn.com/v4/letter/p/258eb7/32.png) [@PandKing](https://discuss.elastic.co/u/PandKing)\ **Post date:** [July 31, 2018, 9:29pm UTC](https://discuss.elastic.co/t/timestamp-in-json-logs/142438/5 "2018-07-31T21:29:39Z")
I found this in another thread. Do I need json.overwitekeys in my beats-pipeline.conf ? > ---
**Author:** ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)\ **Post date:** [July 31, 2018, 9:34pm UTC](https://discuss.elastic.co/t/timestamp-in-json-logs/142438/6 "2018-07-31T21:34:49Z")
You just need the processor section. You can set overwrite keys in that if you need it, but you shouldn't set it in the top level. See the [Decode JSON fields](https://www.elastic.co/guide/en/beats/filebeat/current/decode-json-fields.html) docs. ---
**Author:** ![PandKing](https://avatars.discourse-cdn.com/v4/letter/p/258eb7/32.png) [@PandKing](https://discuss.elastic.co/u/PandKing)\ **Post date:** [July 31, 2018, 9:51pm UTC](https://discuss.elastic.co/t/timestamp-in-json-logs/142438/7 "2018-07-31T21:51:04Z")
Ok that may be my problem, it's in the input section and have no processor section. ``` filebeat.inputs: - type: log json.keys_under_root: true json.add_error_key: true json.overwrite_keys: true enabled: true paths: - /var/ossec/logs/alerts/alerts.json #============================= Filebeat modules =============================== filebeat.config.modules: # Glob pattern for configuration loading path: ${path.config}/modules.d/*.yml # Set to true to enable config reloading reload.enabled: false #----------------------------- Logstash output -------------------------------- output.logstash: # The Logstash hosts hosts: ["###.###.###.###:5044"] ``` ---
**Author:** ![PandKing](https://avatars.discourse-cdn.com/v4/letter/p/258eb7/32.png) [@PandKing](https://discuss.elastic.co/u/PandKing)\ **Post date:** [July 31, 2018, 9:52pm UTC](https://discuss.elastic.co/t/timestamp-in-json-logs/142438/8 "2018-07-31T21:52:59Z")
Followed this thread > [@6.0.0-rc1: json.overwrite\_keys not working with @timestamp](https://discuss.elastic.co/t/6-0-0-rc1-json-overwrite-keys-not-working-with-timestamp/105189): > > Hi, we are trying to forward all json messages in a log file to logstash using Filebeat, but the timestamps are already off on the Filebeat side. With 5.0.1 it is working, with 6.0.0-rc1 we get duplicate @timestamp fields. Sample JSON: { "@timestamp": "2017-10-17T10:03:14.301Z", "request": "/" } Sample Filebeat Config 5.0.1 (working): filebeat.prospectors: - input\_type: log paths: - serverlogs/apache.json json.keys\_under\_root: true json.add\_error\_key: true json.overwrite\_keys: … ---
**Author:** ![PandKing](https://avatars.discourse-cdn.com/v4/letter/p/258eb7/32.png) [@PandKing](https://discuss.elastic.co/u/PandKing)\ **Post date:** [August 1, 2018, 6:49pm UTC](https://discuss.elastic.co/t/timestamp-in-json-logs/142438/9 "2018-08-01T18:49:54Z")
Little confused json.overwrite\_keys in filebeat.reference.yml has it in the **input** section as well as much of the discussions I have looked at. Release notes have it in the processor section but there is no context of the difference. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 29, 2018, 6:50pm UTC](https://discuss.elastic.co/t/timestamp-in-json-logs/142438/10 "2018-08-29T18:50:06Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.