# Pass full original timestamp from Zeek log to Elasticsearch **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [May 22, 2020, 6:06pm UTC](https://discuss.elastic.co/t/pass-full-original-timestamp-from-zeek-log-to-elasticsearch/233944 "2020-05-22T18:06:00Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![matthewerobison](https://avatars.discourse-cdn.com/v4/letter/m/8edcca/32.png) [@matthewerobison](https://discuss.elastic.co/u/matthewerobison)\ **Post date:** [May 22, 2020, 6:06pm UTC](https://discuss.elastic.co/t/pass-full-original-timestamp-from-zeek-log-to-elasticsearch/233944/1 "2020-05-22T18:06:01Z")
I have an Elastic cluster setup with a server hosting Zeek and Filebeat. We are manually running PCAP files through Zeek, Filebeat is picking up the logs, and data is being parsed/indexed by Elastic and we can see all the Zeek log data. I would like to see if anyone has been able to get Filebeat to pass along the full timestamp field from the Zeek log to Elastic. Currently, it cuts off part of the timestamp and only goes to the millisecond, but I would like it to keep all the way to the nano second. I've tried changing the fields.yml file to say the date field is type date\_nanos, but that did not work. I've also tried to create a new field and pass it the zeek.conn.ts field, which is the "ts" field from the Zeek connection log and it creates the new field in Elastic, but it is still the same truncated timestamp with no nano seconds. It's almost as though Filebeat cannot handle date/time that goes to the nano second. ---
**Author:** ![Andre\_Letterer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andre_letterer/32/42248_2.png) [@Andre\_Letterer](https://discuss.elastic.co/u/Andre_Letterer)\ **Post date:** [May 31, 2020, 9:53pm UTC](https://discuss.elastic.co/t/pass-full-original-timestamp-from-zeek-log-to-elasticsearch/233944/2 "2020-05-31T21:53:36Z")
Hi Matthew, We have a log ongoing work around nanosecond. For beats and logstash > > > Now that Elasticsearch 7.x supports nanosecond timestamps, it would be nice if they could pass through logstash as well (in my... > > > The @timestamp field only has microsecond precisions. Internally, or when collecting from external sources, timestamps might have higher precisions. Elasticsearch supports... One of the PRs adds a `fraction of second` formatting possibility for time formatting: > ```auto // S fraction of second nanoseconds yes 978000 // f fraction of seconds nanoseconds yes 123456789 ``` Should be merged in 7.7.0. Would you mind to share, how exactly does your field from the log file look like? ---
**Author:** ![matthewerobison](https://avatars.discourse-cdn.com/v4/letter/m/8edcca/32.png) [@matthewerobison](https://discuss.elastic.co/u/matthewerobison)\ **Post date:** [June 2, 2020, 2:52pm UTC](https://discuss.elastic.co/t/pass-full-original-timestamp-from-zeek-log-to-elasticsearch/233944/3 "2020-06-02T14:52:34Z")
Hi Andre Here is a snipit of a line from our Zeek Connection log. It shows the full timestamp. We are using the default config and Elastic pipelines that come with Filebeat. ```auto {"ts":1548711854.189404,"uid":"CyrSdJ3kqGRU2bOr04","id.orig_h":"172.17.8.109","id.orig_p":49157,"id.resp_h":"172.17.8.2","id.resp_p":88,"proto":"tcp","service":"krb_tcp","duration":0.0032958984375,"orig_bytes":248,"resp_bytes":304,"conn_state":"RSTR","missed_bytes":0,"history":"ShADdFar","orig_pkts":4,"orig_ip_bytes":420,"resp_pkts":4,"resp_ip_bytes":476} ``` ---
**Author:** ![Andre\_Letterer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andre_letterer/32/42248_2.png) [@Andre\_Letterer](https://discuss.elastic.co/u/Andre_Letterer)\ **Post date:** [June 3, 2020, 11:37pm UTC](https://discuss.elastic.co/t/pass-full-original-timestamp-from-zeek-log-to-elasticsearch/233944/4 "2020-06-03T23:37:55Z")
Ok, this was helpful, and these are not nanoseconds ```auto https://en.wikipedia.org/wiki/Nanosecond 10^-9 s java format SSSSSSSSS https://en.wikipedia.org/wiki/Microsecond 10^-6 s java format SSSSSS https://en.wikipedia.org/wiki/Millisecond 10^-3 s java format SSS ``` So this is more what you have ```auto 1548711854.189404 Epoch sec ms µs ``` These are µs actually. Normally date timestamp conversion can be done in ingest pipelines in elasticsearch or in logstash using date filter. Here an example of how to convert the timestamp in ingest pipeline: ```auto GET /_ingest/pipeline/_simulate { "pipeline" : { "description": "_description", "processors": [ { "date" : { "field" : "timestamp", "target_field": "convertedtimestamp", "formats": ["UNIX"] } } ] }, "docs": [ { "_index": "testindex", "_id": "id", "_source": { "timestamp": "1548711854.189404", "@timestamp": "2020-05-27T11:37:25.559Z" } } ] } ``` The converted timestamp is then in field `convertedtimestamp`: Result would be like: ```auto ... "_source" : { "convertedtimestamp" : "2016-03-19T09:07:21.909Z", "@timestamp" : "2020-05-27T11:37:25.559Z", "timestamp" : "1458378441.909527" }, ... ``` If you like you can transfer the timestamp into @timestamp field as well. Hope that helps. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 1, 2020, 11:51pm UTC](https://discuss.elastic.co/t/pass-full-original-timestamp-from-zeek-log-to-elasticsearch/233944/5 "2020-07-01T23:51:51Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.