I am an ELK newbie. I have created some text files using a python script and saved them in a folder. I have configured Filebeat to send to Logstash and Logstash to Elasticsearch. I have set the path as
- /home/vboxuser/temp/output_text_files2/*.txt
My problem is that when i start filebeat and logstash, it does not immediately send the files. Once I enter the file, do anything in it (not changing the final file) and save it, then it gets indexed. Why do I need to save it once again for it to be sent? Also I do not even know if it is an filebeat problem or logstash problem. I tried using the ignore_older: 1m option, I tried using scan_frequency:1m option, still does not work.
Any or all help is appreciated.