I am reading input from a .txt file that is 24 hours old, is there any way to have it read in as live data rather than read in all at once? Basically trying to mimic real-time data with yesterday's data. I am running logstash outputted to elasticsearch then displayed in kibana (all versions 5.6.2).
Have you looked at the throttle filter for logstash ? It won't give a synthetic reproduction of live data being ingested, but it will allow you to stagger the volume and frequency of the data as it travels across your workflow.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.