Logstash Elasticsearch output plugin fails to establish a connection

[Thijsvdp (Thijs)]

Hi all,

I am facing an error with Logstash which is not able to connect to Elasticsearch. I am getting the following error:

[2023-01-11T12:10:15,365][WARN ][logstash.outputs.elasticsearch][continuous] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://elastic:xxxxxx@elasticsearch:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401' contacting Elasticsearch at URL 'https://elasticsearch:9200/'"}

I use the following .conf:

input {
    jdbc {
        jdbc_driver_library => "./src/postgresql-42.3.6.jar"
        jdbc_driver_class => "org.postgresql.Driver"
        jdbc_connection_string => "jdbc:postgresql://${PG_HOST}:${PG_PORT}/${PG_DB}?options=-c%20statement_timeout=900000"
        last_run_metadata_path => "${LS_META_PATH}/${ENVIRONMENT}_continuous_all.yml"
        jdbc_user => "${PG_USER}"
        jdbc_password => "${PG_PASSWORD}"
        statement_filepath => "./src/queries/${ENVIRONMENT}_continuous.sql"
        use_column_value => true
        tracking_column_type => "numeric"
        tracking_column => "load_id"
        schedule => "*/8 * * * * *"
        jdbc_paging_enabled => true
        jdbc_paging_mode => "explicit"
        jdbc_page_size => "${BATCH_SIZE}"
    }
}
filter {
    json {
        source => "members"
        target => "members"
    }
    ruby {
        path =>"./src/scripts/add_representative.rb"
    }
}
output {
    elasticsearch {
        hosts => "${ES_HOST}"
        index => "${ES_INDEX}"
        user => "${ES_USER}"
        password => "${ES_USER}"
        document_id => "%{family_id}"
        doc_as_upsert => true
        action => "update"
        manage_template => true
        cacert => "${SSL_CERT_FILE:.}"
        ssl => true
    }
    http {
        url => "${ES_HOST}/${ES_INDEX}/_delete_by_query"
        user => "${ES_USER}"
        password => "${ES_PASSWORD}"
        http_method => "post"
        format => "message"
        content_type => "application/json"
        message => '{"query": {"ids": {"values": "%{[@metadata][member_ids]}"}}}'
        cacert => "${SSL_CERT_FILE:.}"
    }
}

Where ES_HOST=https://elasticsearch:9200. I am running the Logstash job on Kubernetes as well as the Elasticsearch cluster itself. I have manually queried Elasticsearch from the container using curl which is able to make a successful request.

Could someone help me here?

[leandrojmp (Leandro Pereira)]

@Thijsvdp

Got response code '401' contacting Elasticsearch

You are getting error 401, this means that your user or password is wrong.

In the config you shared you put the wrong variable for the password option.

@Thijsvdp

        user => "${ES_USER}"
        password => "${ES_USER}"

[Thijsvdp (Thijs)]

Alright, that is extremely stupid of me... I have been looking at it for so long that I completely missed that.

Anyways, thanks a lot!