In logstash, How can we check if a data exists another elastic search index?

[meghavinam]

I want to sync set of data from mysql to elastic search. That time i need to remove if same data exists in another index

[Wolfram_Haussig]

Hi,

Welcome to this forum!

You can create a LogStash pipeline with the jdbc-input to read the data from mysql and then use the elasticsearch-filter to query ElasticSearch if it exists already.

Based on this information you can use the action setting of the elasticsearch-output to either index or delete the document from your target index.

Best regards
Wolfram

[meghavinam]

Is there any example for this?
How can use the result in output?

[Wolfram_Haussig]

There is - as far as I know - no exact example for what you want but you can check the links above to get a general direction: You can store the results of you query in a field and use an if clause to switch between the output for creating and the output for deletion.

[meghavinam]

I got the solution

mutate { add_field => { "data_occure" => "" } }
elasticsearch {
hosts => ["localhost:9200"]
index => ["active_users_%{[domain_alias]}"]
query => "_id:%{[email]}"
result_size => 1
fields => {"email" =>"data_occure"}
}

output {

    if   [data_occure]=="" {
  stdout { codec => json }

}

}