Here are configuration details,
CSV will uploaded every 30 mins into S3 bucket,
I am currently running logstash from command prompt and keep command prompt up and running so if any new file uploaded into S3, Logstash start reading logs and send it to Elastic Search,
Note: I havent install logstash as a service,
Issue: Logs are missing in Elastic search, (Logstash missing CSVs)
I am also using sincedb so logstash will not import duplicate data, Also CSV's are very small size, and i noticed that some files are uploaded same datetime, Is this main reason logstash missing logs/CSVs?