First, while this is the .csv export in Excel, all cells are formatted as text so this is exactly as it appears in say, Notepad++ or any other raw text editor.
RayID was included as a way to identify specific events and can be ignored otherwise.
EdgeStartTimestamp is on here twice as the timefield for the index is this field.
EdgeStartTimestamp and EdgeEndTimestamp in Elasticsearch are in epoch seconds.
I've configured Kibana to quote values in exports, which resolved a previous issue with date formatting
Below is the same report in Kibana, why are the exported values incorrect?
Changed date format to 'X' which would display the values as epoch time in the interface, which also changes the exported data...not sure I agree but I digress. It looks like the exported values are being truncated for some reason...
It's almost certainly a field formatting issue in the CSV export. The problem is that epoch in unix and most systems is in seconds, but in Javascript time is expressed in milliseconds. So 1544159 in unix is 1544159000 in Javascript, and new Date(1544159) is some time in 1970, while new Date(1544159000) would be Dec 6, 2018, as you'd expect.
My guess here is that the date formatter in the CSV export isn't converting your time (which is in seconds, as it's stored in Elasticsearch) correctly before trying to format the number into a date string (in Javascript, which expects milliseconds), and that's why you are seeing the wrong date.
I'm making a lot of assumptions here, but the output seems to line up with them. @tsullivan@Brandon_Kobel you two have more experience than I do... is this a bug, or is there maybe some setting that can be used here to produce the correct date output?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
