I am trying to insert data in elasticsearch using logstash but when i update a file and again process it the events get duplicated when i only want the newly added entries to get indexed, for this i am trying to use sincedb_path but i am making some mistake. Can anybody help me in finding the mistake?. Any help is appreciated. Thanks
My input filter looks like this:
input{
file
{
mode => "read"
file_completed_action => "log"
file_completed_log_path => "C:/logstash-7.1.1/log.txt"
path => "F:/PRANAY/project/Store Mg C++/sales.csv"
sincedb_path => "F:/PRANAY/project/Store Mg C++/salesincedb.txt"
start_position => "beginning"
}}
It would be really great if someone could explain when to use start_position and mode and which value to use.