Sorting events with identical millis timestamps by order events were recorded

bublik-hpt · February 21, 2022, 1:07pm

Hi i have logs where multiple lines have the same timestamp. In Kibana discover they are in opposite order (if i follow the timestamp) then in the logfile, but when I check surrounding documents they are in correct order. Is it possible to see documents in discovery in correct (time based) order?

example:
logfile:

2022-02-21T13:52:15.840 [bookie-io-1-7] INFO  org.apache.bookkeeper.proto.AuthHandler - Authentication success on server side
2022-02-21T13:52:15.840 [bookie-io-1-7] INFO  org.apache.bookkeeper.proto.BookieRequestHandler - Channel connected  [id: 0x3b9d22a3, L:/10.15.24.43:3181 - R:/192.168.255.94:38216]
2022-02-21T13:52:15.840 [bookie-io-1-7] INFO  org.apache.bookkeeper.proto.BookieRequestHandler - Channels disconnected: [id: 0x3b9d22a3, L:/10.15.24.43:3181 ! R:/192.168.255.94:38216]

discover:

surrounding docs:

Thank you very much.

matw · February 24, 2022, 9:19am

Hi and welcome to our community

So surrounding docs are sorting by timefield and _doc, Discover itself doesn't.

We might implement this in the future:

github.com/elastic/kibana

[Discover] Add _doc as field you can sort by

opened 08:53AM - 11 Oct 19 UTC

kertal

discuss Feature:Discover Team:DataDiscovery

If you have huge amounts of data with many lots of documents sharing the same ti…mestamp up to the millisecond you might wanna use `_doc` for faster sorting. We should consider adding an option for the user to sort by `_doc` Knowhow about `_doc`: > _doc has no real use-case besides being the most efficient sort order. So if you don’t care about the order in which documents are returned, then you should sort by _doc. This especially helps when scrolling. We should also provide the option to use a tiebreaker in Discover (by default `_doc`), here's an older PR for that #32426

The only way to currently solve this is if you have an own sequence field you can sort by additionally. Or we also support sorting by a timestamp with nanoseconds

But of course most loggers don't support this

Best,
Matthias

system · March 24, 2022, 9:19am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.