Update by query in Logstash

Elastic Stack Logstash
1

Hi!
I'm trying to solve the following problem.
I have some information in the database that I want to send to an Elasticsearch index through Logstash.
It is something that I have already done on some occasion but the problem in this case is that the id of the index is auto-generated by Elasticsearch, so I cannot perform the update action through the id.

I would need something similar to functionality

but I understand that the plugin for elastic

can not stand it. Is that correct?

In this link an alternative is mentioned using the http plugin:

but I'm not sure how to perform this in my scenario.

The post I should make is something like this:

POST myindex/_update_by_query
{
  "script": {
    "source": "ctx._source.itemValue = params.itemValue",
    "lang": "painless",
    "params": {
      "itemValue": "itemValue"
    }
  },
  "query": {
    "term": {
      "myField": "myField"
    }
  }
}

How can I send the results of my query being that I should perform a POST for each retrieved row.
It's possible?
Should I do it in another way?

I upload a pseudo configuration to graph the problem.

input {
  jdbc {
    jdbc_driver_class => ""
    jdbc_driver_library => ""
    jdbc_connection_string => ""
    jdbc_user => ""
    jdbc_password => ""
    jdbc_paging_enabled => true
    schedule => ""
    statement => "SELECT A as myField, B AS valueToUpdate FROM TABLE"
  }
}
filter {
  mutate {
    add_field => {
      "[script] [lang]" => "painless"
      "[script] [source]" => "ctx._source.itemValue = params.itemValue"
      "[script] [params] [itemValue]" => // This should go from the valueToUpdate field retrieved from the database
      "[query] [term] [myField]" => // Here should go the information of the myField field retrieved from the database
    }
    remove_field => [""]
  }
}
output {
  http {
    url => "http://IP:9200/index/doc/_update_by_query"
    http_method => "post"
    format => "json"
  }
}
1 Like
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.