How to Remove deleted data from index data

Elastic Stack Logstash
1

Hi
I have an input file having 10 records which is already indexed. Now out of 10 records 4 records removed from file and 6 records are updated. I have used document_id to replace all the records in that index.
Now
Those 6 records correctly updated in kibana but it will also displaying that 4 removed records.
So please let me know how to delete those 4 records from index data.
records of csv file
0,col1,string,client1.json
1,col2,string,client1.json
2,col3,string,client1.json
3,col4,string,client1.json
4,col5,string,client1.json
5,col6,string,client1.json
6,col7,string,client1.json
7,col8,string,client1.json
8,col9,string,client1.json
9,col10,string,client1.json

6 Updated records :
0,col1,timestamp,client1.json
1,col2,timestamp,client1.json
2,col3,timestamp,client1.json
3,col4,timestamp,client1.json
8,col9,timestamp,client1.json
9,col10,timestamp,client1.json

4 Deleted records :
4,col2,string,client1.json
5,col3,string,client1.json
6,col1,string,client1.json
7,col2,string,client1.json

Here is the config file that I'm using

input {
file {
path => "Z:/ELK/*.csv"
start_position => "beginning"
sincedb_path => "NUL"
type => "normalized_jsons"
}

}
filter {

if [type] == "normalized_jsons"
	{
			csv {
			separator => ","
			columns => ["Index","ColumnName","DataType","JSONName"]
		}
	}

}
output {

if [type] == "normalized_jsons"
	{
			elasticsearch {
			hosts => ["localhost:9200"]
			index => ["normalized_jsons"]
			document_id => "%{JSONName}_%{Index}"
			
			}
	stdout {}
	}

}

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.