Hi!
I'm trying to solve the following problem.
I have some information in the database that I want to send to an Elasticsearch index through Logstash.
It is something that I have already done on some occasion but the problem in this case is that the id of the index is auto-generated by Elasticsearch, so I cannot perform the update action through the id.
I would need something similar to functionality
but I understand that the plugin for elastic
can not stand it. Is that correct?
In this link an alternative is mentioned using the http plugin:
but I'm not sure how to perform this in my scenario.
The post I should make is something like this:
POST myindex/_update_by_query
{
"script": {
"source": "ctx._source.itemValue = params.itemValue",
"lang": "painless",
"params": {
"itemValue": "itemValue"
}
},
"query": {
"term": {
"myField": "myField"
}
}
}
How can I send the results of my query being that I should perform a POST for each retrieved row.
It's possible?
Should I do it in another way?
I upload a pseudo configuration to graph the problem.
input {
jdbc {
jdbc_driver_class => ""
jdbc_driver_library => ""
jdbc_connection_string => ""
jdbc_user => ""
jdbc_password => ""
jdbc_paging_enabled => true
schedule => ""
statement => "SELECT A as myField, B AS valueToUpdate FROM TABLE"
}
}
filter {
mutate {
add_field => {
"[script] [lang]" => "painless"
"[script] [source]" => "ctx._source.itemValue = params.itemValue"
"[script] [params] [itemValue]" => // This should go from the valueToUpdate field retrieved from the database
"[query] [term] [myField]" => // Here should go the information of the myField field retrieved from the database
}
remove_field => [""]
}
}
output {
http {
url => "http://IP:9200/index/doc/_update_by_query"
http_method => "post"
format => "json"
}
}