Update by query with Logstash http output message

Elastic Stack Logstash
1

I am using Logstash to update by query existing Elasticsearch documents with an additional field that contains aggregate values extracted from Potgresql table. I use elastichsearch output to load one index using document_id and http output to update another index that have different document_id. Elasticsearch output works fine but http output give errors:

[2023-02-08T17:58:12,086][ERROR][logstash.outputs.http ][main][b64f19821b11ee0df1bd165920785876cd6c5fab079e27d39bb7ee19a3d642a4] [HTTP Output Failure] Encountered non-2xx HTTP code 400 {:response_code=>400, :url=>"http://localhost:9200/medico/_update_by_query", :event=>#LogStash::Event:0x19a14c08}

This is my pipeline configuration:

input {
    jdbc {
        # Postgres jdbc connection string to our database, mydb
        jdbc_connection_string => "jdbc:postgresql://handel:5432/mydb"
        statement_filepath => "D:\ProgrammiUnsupported\logstash-7.15.2\config\nota_sede.sql"
    }
}

filter {
    aggregate {
        task_id => "%{idCso}"
        code => "
            map['idCso'] = event.get('idCso')
            map['noteSede'] ||= []
            map['noteSede'] << {
                'id' => event.get('idNota'),
                'tipo' => event.get('tipoNota'),
                'descrizione' => event.get('descrizione'),
                'data' => event.get('data'),
                'dataInizio' => event.get('dataInizio'),
                'dataFine' => event.get('dataFine')
            }
            event.cancel()"
        push_previous_map_as_event => true
        timeout => 60
        timeout_tags => ['_aggregatetimeout']       
    }
   }
}

output {

    stdout { codec => rubydebug { metadata => true } }

#       this works
    elasticsearch {
        hosts => "https://localhost:9200"
        document_id => "STRUTTURA_%{idCso}" 
        index => "struttura"
        action => "update"
        user => "user"
        password => "password"
        ssl => true
        cacert => "/usr/share/logstash/config/ca.crt"   
    }
    
    http {
        url => "http://localhost:9200/medico/_update_by_query"
        user => "elastic"
        password => "changeme"
        http_method => "post"
        format => "message"
        content_type => "application/json"
        message => '{
                        "query":{
                            "term":{
                                "idCso":"%{idCso}"
                            }
                        },
                        "script":{
                            "source":"ctx._source.noteSede=params.noteSede",
                            "lang":"painless",
                            "params":{
                                "noteSede":"%{noteSede}"
                                }
                            }
                        }
                    }'
    }
}

Could you help please, thx.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.