Hello Team,
I am new to using this platform, hence posting my query here.
I have a logstash conf file which is taking input from a csv file and I am parsing the data using csv filter plugin. Now the issue is I am using elasticsearch output plugin. At first when I run the pipeline without using the csv filter , only input and output plugin , the indices was being created and the data was getting loaded to elasticsearch indices. After I modified my conf file by adding csv filter the same is not working means the indices is not getting created in elasticsearch. I tried to output the result manually into the terminal it was working using stdout but the same is not working when output is being written to elasticsearch. here is my conf file:
input {
file {
path => "/home/ec2-user/elastic_weblog_new.csv"
start_position => beginning
}
}
filter {
csv {
separator => ","
columns => ["IP","Time","URL","Staus"]
}
}
output {
#stdout {
#codec => rubydebug }
elasticsearch {
hosts => ["http://172.31.50.249:9200"]
index => "weblog-%{+YYYY.MM.dd}"
}
}
Please let me know my mistake what I am doing wrong, I don't see any error in the log as well.