I am new to ELK stack and have recently installed logstash, Elasticsearch and Kibana (all version 5.4 on CentOS 6.5).
Objective is to monitor a directory for new csv files and parse them using logstash and index them into Elasticsearch for report generation.
I can parse the files through the command line (/usr/share/logstash/bin/logstash -f logstash.conf) and it gets successfully parsed and indexed in ES, however when i put the conf file under /etc/logstash/conf.d folder, it does not process the csv files.
my logstash.conf is as below
path => "/root/ELK/wallet-data/*.csv"
start_position => "beginning"
sincedb_path => "/dev/null"
separator => ","
columns => ["date","description","moneyIn","moneyOut","tx"]
hosts => "http://localhost:9200"
index => "wallet-index"
I have enabled debug for the logfile and i can see it trying to process the conf file and get the column-names etc but nothing gets indexed in ES.
Not sure what am i doing wrong over here. Any help would be really appreciated.