Hi,
I am fully aware that the issue i am going to write about was already discussed multiple times (e.g. see https://github.com/elastic/elasticsearch/issues/10005 , https://github.com/elastic/logstash/issues/3530 etc).
Current version of Logstash (2.2.x) is able to correctly process log files with milliseconds precision.
When the log file contains higher precision timestamps (RFC 3339 format) all parts after milliseconds are truncated.
Because of this Kibana sorting is not working correctly. Messages from log files are displayed in wrong sequence.
This problem is even more apparent when multiline filter was applied (misplaced events that occured in the same milliseconds)
Many real world application needs higher precision.
Like databases (microseconds) or network infrastructure (nanoseconds).
If i understood correctly the problem lies inside the limits of the code handling timestamps.
In Logstash it is Joda-time classes package which limits precision to milliseconds.
Joda-time classes were used mainly because of not-so-great Java date and time classes. This situation is no longer true.
Hope you know, that in fact Joda-time classes authors are asking user to migrate to the new java.time classes (see http://www.joda.org/joda-time/).
The authors of Joda-time greatly contributed to the all new java.time (JSR-310) classes which are core part of JDK onwards Java SE 8.
And JSR-310 java.time classes are based on nanoseconds.
I don't know about the requirements or roadmap of the Elastic/Logstash/Kibana package.
But i think simple solution of the problem (truncating timestamp after milliseconds part) could be replacing usage of Joda-Time classes with the java.time JSR-310 - which is, by design, working with nanoseconds.
Of course that would result in higher requirement of Java SE 8 for the Elastic/Logstash/Kibana package.
On the Elastic webpage i see that in requirements for Logstash (2.2.x) JRE 8 is highly recommended, but JRE 6/7 is still supported.
Java JRE 6 (Feb 2013) and JRE 7 (Apr 2015) had already reached End of life support period.
Thanks for any reply.
Best regards.