Hello all,
I had opened a similar issue before to get sub second precision in Elasticsearch. I wasn't able to send logs in epoch millseconds to Elasticsearch which now I have figured out how to do.
Now I have two issues I would like some help with.
1 : I get the time in epoch format (1640879589134). How to convert in to human readable format?
I tried to make changes to date format in the template but it didn't help much (I got it working once but not after that and I do not remember what format was it).
2 : With milliseconds, Elasticsearch creates another field called "time"(which is present in the logs), which is not present when it's only seconds. With time in second only precision I get "@timestamp" which is defined in the template but with milliseconds a "time" field is created.
How to make the "@timestamp" field give time in milliseconds?
Thank you
The answers and comments here assume you are using Elasticsearch as Opensearch is not supported here. If you are using Opensearch I would recommend you reach out to that community instead as details may differ.
The mappings define how Elasticsearch parses the documents. Elasticsearch will however not modify the indexed documents. If you therefore index a document with an epoch timestamp you will be able to query this using date syntax if the mappings are correct but the returned document will always contain exactly what you indexed. You will therefore need to transform the format in your application.
How are you indexing the data? What tool are you using?
Hello @Christian_Dahlqvist , thank you for your reply.
I 'had' this issue with both Elasticsearch and opensearch. I am using fluentd for indexing data. After your comment, I tried to change the logs in apache to emit logs in a human readable(2021-12-31T00:10:14.799+0000) format and it solved the issue. I used to produce the apache logs in epoch milliseconds before.
I still do not understand one thing. Apache logs before this change were in epoch second format("@timestamp":1640833640) and in kibana the logs were in human readable. Why is that so?
Kibana does some transformation before displaying it, e.g. adjusting time zone based on user location.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.