Hi, I use EKB to process log data, and FileBeat sends data to ElasticSearch directly. The objective is to have some way to render the sequence of events in the same order they were read out of the log file. With the "offset" field, I can do that inside a file. But in my case, all the "source" fil…

An other idea I had was using harvester_limit: 1 to make sure one log file is finished reading before the next one starts. The close_* variables would have to be adjusted to your use case to make sure the file is not close too early. Then you could use the timestamp generated by filebeat itself for …

Discuss the Elastic Stack

Restore the sequence of the events

Elastic Stack Beats

ginger (jiangguoqiang) December 12, 2016, 11:45am 10

Thanks @ruflin so much.

Topic		Replies	Views
Filebeat \| Ordering Messages Beats	3	1389	October 31, 2016
How to maintain the order of logs Logstash	9	7372	July 6, 2017
How can i maintain the sequence of log data while parsing it in logstash? Logstash	4	1005	October 24, 2018
How to control the filebeat read file sequence Beats filebeat	8	4430	July 5, 2017
Regarding @timestamp Elasticsearch	7	1508	July 5, 2017

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.

Restore the sequence of the events

Related topics