How to make ElasticSearch return results in sorted time order

Elastic Stack Elasticsearch
1

Hello,

I have an index whose documents have a "timestamp" field. Each document represents a syslog line and the timestamp corresponds to the timestamp of the line. Now in my system, sometimes the log lines are printed so quickly that two or more log lines have the same timestamp.

Im developing a log parser in which I query ElasticSearch for log lines that match a certain filter and then walk through the log lines and implement some business logic. For the logic to work, it is important that the lines retrieved from ElasticSearch are in exactly the same order they were put in (the process should align with just parsing a syslog file line by line in sequence). Ive noticed that sometimes ElasticSearch will deliver lines out of order if the lines share the same timestamp.

My question is: Is there a hidden field in ES that captures a very granular timestamp at which a document is inserted and can I make ES return documents in that order?

2

Elasticsearch can do multiple sorts. For example, "first sort on timestamp and then sort on document ID." Kibana actually has a view surrounding documents feature which can make use of this as well. If you want to preserve the specific order at a granularity beyond what's contained in your logs, you do need to have some field which identifies the canonical order.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.