Hi. Is there a way to sort by a different field in the Observability/Logs app? Since we have multiple logging servers feeding a kafka queue, our logs don't necessarily arrive in the same order they're generated. We'd like to use Filebeat's log.offset field as the tiebreaker, but the tiebreaker feature seems to be deprecated. Can someone point me to how to sort the logs in an order other than strictly timestamp?
Thanks in advance,
Mike
Hi @ippolito,
there is currently no support for other sorting keys since several optimizations rely on the fact that it's a time series. If the timestamps in your indexed documents are not accurate, maybe you could improve your ingestion pipeline to extract the correct timestamps (i.e. those generated by the source) from the log message payload?
OK, thanks Felix. There really isn't a way for us to change the ingestion pipeline, because many of the logs arrive with the same timestamp down to the millisecond, in which case we'd need the tiebreaker.
Hm, I understand that the limitation is annoying. For clarification, the tiebreaker is not deprecated, but the ability to configure it is. We found that it's easy to misunderstand and therefore wanted to stick with the lucene _doc value.
I'll think about whether we can introduce an additional monotonic field from ECS to use as the tiebreaker in the future, which works reliably for all use-cases.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.