Logstash Elasticsearch output plugin fails to establish a connection

Elastic Stack Logstash
1

Hi all,

I am facing an error with Logstash which is not able to connect to Elasticsearch. I am getting the following error:

[2023-01-11T12:10:15,365][WARN ][logstash.outputs.elasticsearch][continuous] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://elastic:xxxxxx@elasticsearch:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401' contacting Elasticsearch at URL 'https://elasticsearch:9200/'"}

I use the following .conf:

input {
    jdbc {
        jdbc_driver_library => "./src/postgresql-42.3.6.jar"
        jdbc_driver_class => "org.postgresql.Driver"
        jdbc_connection_string => "jdbc:postgresql://${PG_HOST}:${PG_PORT}/${PG_DB}?options=-c%20statement_timeout=900000"
        last_run_metadata_path => "${LS_META_PATH}/${ENVIRONMENT}_continuous_all.yml"
        jdbc_user => "${PG_USER}"
        jdbc_password => "${PG_PASSWORD}"
        statement_filepath => "./src/queries/${ENVIRONMENT}_continuous.sql"
        use_column_value => true
        tracking_column_type => "numeric"
        tracking_column => "load_id"
        schedule => "*/8 * * * * *"
        jdbc_paging_enabled => true
        jdbc_paging_mode => "explicit"
        jdbc_page_size => "${BATCH_SIZE}"
    }
}
filter {
    json {
        source => "members"
        target => "members"
    }
    ruby {
        path =>"./src/scripts/add_representative.rb"
    }
}
output {
    elasticsearch {
        hosts => "${ES_HOST}"
        index => "${ES_INDEX}"
        user => "${ES_USER}"
        password => "${ES_USER}"
        document_id => "%{family_id}"
        doc_as_upsert => true
        action => "update"
        manage_template => true
        cacert => "${SSL_CERT_FILE:.}"
        ssl => true
    }
    http {
        url => "${ES_HOST}/${ES_INDEX}/_delete_by_query"
        user => "${ES_USER}"
        password => "${ES_PASSWORD}"
        http_method => "post"
        format => "message"
        content_type => "application/json"
        message => '{"query": {"ids": {"values": "%{[@metadata][member_ids]}"}}}'
        cacert => "${SSL_CERT_FILE:.}"
    }
}

Where ES_HOST=https://elasticsearch:9200. I am running the Logstash job on Kubernetes as well as the Elasticsearch cluster itself. I have manually queried Elasticsearch from the container using curl which is able to make a successful request.

Could someone help me here?

2

You are getting error 401, this means that your user or password is wrong.

In the config you shared you put the wrong variable for the password option.

1 Like
3

Alright, that is extremely stupid of me... :roll_eyes: I have been looking at it for so long that I completely missed that.

Anyways, thanks a lot!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.