Update document without id

Elastic Stack Logstash
1

I'm pretty new to logstash. I have an index called person in Kibana. The index includes firstname, lastname, email, address... Now I need to update person's address by using a table in database. My table only have emails and new addresses. I want to retrieve id in the elasticserach filter.
Here's my config file:

input {
jdbc{
#jdbc library, connection string...
}

statement => "select email, address from person_table"

}
filter{
elasticsearch {
host =>"myhost"
index => "person"
query => "email_field:%{email}"
fields => ["id"] # I want to retrieve id here and use it later, but couldn't make it work.
}
}

output{
elasticsearch {
host =>"myhost"
index => "person"
action => "update"
doc_as_upsert => "true"
document_id => "%{id}"
}

docinfo is not availble in filter. How can I get the id?

2

Why not add a unique key to your database table that you can use as document id in Elasticsearch?

3

My table has an integer id. The document id in Elasticsearch for the index is a 33 character string. They are not equal. So I couldn't do document_id =>%{id} in output.

4

I would recommend reindexing the data with this id as document id as it will make updates much easier and more efficient.

5

If it's possible to retrieve document id and set it as parameter in filter and use it in output?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.