data.json file is used by Filebeat to store states of harvested files. If Filebeat is not harvesting from files then this data.json file will be empty. So what you watch is symptom and not the reason.
Could you please start filebeat in debug mode (./filebeat -e -d "*") and share the logs with us?
@ChrsMark yes, you are correct... I have changed the filebeat yml to following config and it worked. althought still i coudnlt see the logs in filebeat but its being sent to logstash directly in real time
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.