Fail to send data from logstash to elasticsearch after installing x-pack

Hi,

I tried to install x-pack to get marvel functionality in ES 5.0.0.
Unfortunately did it force me to use security.

I am currently having problmes sending data from logstash to elastic search.
[2016-11-15T20:03:15,183][WARN ][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=>#<LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError: No Available connections>}
[2016-11-15T20:03:15,183][WARN ][logstash.outputs.elasticsearch] Elasticsearch output attempted to sniff for new connections but cannot. No living connections are detected. Pool contains the following current URLs {:url_info=>{}}
[2016-11-15T20:03:20,067][WARN ][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=>#<LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError: No Available connections>}
[2016-11-15T20:03:20,068][ERROR][logstash.outputs.elasticsearch] Attempted to send a bulk request to elasticsearch, but no there are no living connections in the connection pool. Perhaps Elasticsearch is unreachable or down? {:error_message=>"No Available connections", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError", :will_retry_in_seconds=>64}

There is no problem with basic connectivity:
curl -u elastic:changeme elasticsearch:9200
{
"name" : "5--6n-p",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "JLQqztpqR6C55p4x4baNLg",
"version" : {
"number" : "5.0.0",
"build_hash" : "253032b",
"build_date" : "2016-10-26T04:37:51.531Z",
"build_snapshot" : false,
"lucene_version" : "6.2.0"
},
"tagline" : "You Know, for Search"
}

There are no filebeat indices:
curl http://elasticsearch:9200/_cat/indices/ -u elastic:changeme
yellow open .monitoring-es-2-2016.11.15 RLKhWkBATFCIAvbqPyczgg 1 1 27510 27 11.7mb 11.7mb
yellow open .monitoring-data-2 8yhqiEkqS_a3PLjvEXGZ4w 1 1 3 0 6.9kb 6.9kb
yellow open .kibana alfvdT1hQz-FQNWzFn_LsA 1 1 62 425 348kb 348kb
yellow open .monitoring-kibana-2-2016.11.15 oVGI4CUxTw6pFBHYLlALGg 1 1 5184 0 1.2mb 1.2mb
green open .security 9qs8Gj_NSQW1V8CYF9ylZg 1 0 4 0 16.2kb 16.2kb

logstash output config:
output {
elasticsearch {
user => logstash_internal
password => changeme
hosts => ["elasticsearch.service.consul:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}

Role configuration:
curl -XGET -u elastic:changeme 'elasticsearch:9200/_xpack/security/role/logstash_reader?pretty'
{
"logstash_reader" : {
"cluster" : [ ],
"indices" : [
{
"names" : [
"filebeat-*"
],
"privileges" : [
"read",
"view_index_metadata"
]
}
],
"run_as" : [ ],
"metadata" : { }
}
}

user configuration:
curl -XGET -u elastic:changeme 'elasticsearch:9200/_xpack/security/user/logstash_user?pretty'
{
"logstash_user" : {
"username" : "logstash_user",
"roles" : [
"logstash_reader"
],
"full_name" : "Kibana User",
"email" : null,
"metadata" : { },
"enabled" : true
}
}

Any ideas why elastic search is not taking logstash data?

Br Mat

Hi,

I noticed that even after uninstalling x-pack, logstash was still unsuccessfull sending data to elasticsearch.
I found a work around for the problem. Disabling sniffing in logstash output configuration file made event transfers work again.

Br mat

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.