How to update data automatically in elastic search when we deleted files in log stash input path

Elastic Stack Elasticsearch
1

i uploaded source files into elastic search by log stash. whenever new file is added in source path (input) it automatically reflecting in elastic search index . but if any files we deleted in source path , it showing same like previous . that deleted file result also showing in elastic search index.

source : /input having 2 csv files on time T1 then these 2 files are pushed into elastic search index .

then at , T2 i deleted a single csv file from source /input , but it will showing 2 csv result in elastic search index .

so , how to make reflection of update data in elastic search when any files were deleted in source path .

2

Logstash is designed to stream log files, not sync files in a directory structure with Elasticsearch. If you want to handle deleted files I suspect you may need to create a custom solution.

3

I think that FSCrawler might be able to handle this. Not sure though.

4

thank you @ christian

5

Thank you @dadoonet

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.