Hi, total ELK newbie here.
I think I'm missing some vital basic information. And it doesn't help that I can't search for e.g. "_uid" and "@timestamp", only "uid" and "timestamp" in both Google and here in Discuss... And searching for "id" and "timestamp" gives overwhelmingly false positives
We have a process that would like to "subscribe" to an Elasticsearch query. So periodically poll Elasticsearch and only get documents that were introduced ( == indexed? ) since last poll.
Is that even possible? I don't mean just query everything in
[ now() - 5min, now() ], because if timestamps are determined by the sender, there could easily be timestamps out of sync. And multiple documents can arrive in the same millisecond. I'd like to get every new document exactly once in arrival order.
Will order by _uid do what I want? And can I combine that with
where _id > $lastQueriedID somehow? (Sorry, I'm not yet fluent in Query DSL). Does that guarantee sorting by arrival order, or is that just sort by
ASCII(concat(_type, _id)) which is somehow unrelated to arrival order?
And what happens when we move from a single sandbox stand-alone machine to a cluster? Are there queries of this sort that will be deterministic on a single machine but that'll give misleading replies when moving to a cluster in the future?