# X-pack installation with logstash

**URL:** <https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175>\
**Category:** Logstash\
**Created:** [June 13, 2017, 11:14am UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175 "2017-06-13T11:14:31Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![raghvendra](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@raghvendra](https://discuss.elastic.co/u/raghvendra)\
**Post date:** [June 13, 2017, 11:14am UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175/1 "2017-06-13T11:14:31Z")

</div>

hi... I have been trying to install x pack in my ELK cluster of 3 nodes. I have elasticsearch on all 3 nodes in logstash and kibana in 1 node which I operate physically and rest using ssh server . I have installed x pack in ES of all nodes and in kibana and logstash . ES and kibana is working fine and I am able to use new functionalities on pre-loaded data . When I start logstash to pump new data into ES , it shows error.

[2017-06-13T16:37:44,413][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>#\<URI::HTTP:0x7de13e33 URL:[http://logstash\_system:xxxxxx@localhost:9200/\_xpack/monitoring/?system\_id=logstash&system\_api\_version=2&interval=1s](http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s)\>, :error\_type=\>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/)][Manticore::SocketException] Connection refused (Connection refused)"}  
[2017-06-13T16:37:44,596][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/), :path=\>"/"}

In spite of this error .. it is pumping data into ES . Config file is -

input {  
file {  
path =\> "/home/mywavia/Desktop/original/sample.txt"  
start\_position =\> "beginning"  
}  
}  
filter {

grok {  
match =\> { "message" =\> "%{COMBINEDAPACHELOG}"}  
}  
geoip {  
source =\> "clientip"  
}  
}  
output {  
elasticsearch { hosts=\>["master","slave-1","slave-2"]  
user =\> elastic  
password =\> changeme   
}

}  
I had followed this document- [https://www.elastic.co/guide/en/x-pack/current/installing-xpack.html](https://www.elastic.co/guide/en/x-pack/current/installing-xpack.html)  
Can anyone suggest what's the problem ???

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 13, 2017, 11:24am UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175/2 "2017-06-13T11:24:03Z")

</div>

Have you configured the `logstash.ym`l file according to [these instructions](https://www.elastic.co/guide/en/x-pack/current/monitoring-logstash.html)? It seems odd that there is no link to this page though. I will create an issue for this.

---

<div class="post-metadata">

**Author:** ![raghvendra](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@raghvendra](https://discuss.elastic.co/u/raghvendra)\
**Post date:** [June 13, 2017, 11:29am UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175/3 "2017-06-13T11:29:22Z")

</div>

i have added these lines into logstash.yml  
xpack.monitoring.enabled: true  
xpack.monitoring.elasticsearch.url: ["192.168.2.157:9200","192.168.2.158:9200","192.168.2.156:9200"]  
xpack.monitoring.elasticsearch.username: elastic  
xpack.monitoring.elasticsearch.password: changeme

---

<div class="post-metadata">

**Author:** ![raghvendra](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@raghvendra](https://discuss.elastic.co/u/raghvendra)\
**Post date:** [June 14, 2017, 5:59am UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175/4 "2017-06-14T05:59:53Z")

</div>

@Christian_Dahlqvist I have followed your link to configure Logstash .. and I have appended these lines into logstash.yml =\>

xpack.monitoring.enabled: true  
xpack.monitoring.elasticsearch.url: ["http//master:9200","http//slave-2:9200","http//slave-1:9200"]  
xpack.monitoring.elasticsearch.username: logstash\_internal  
xpack.monitoring.elasticsearch.password: changeme

Since , I have not enabled SSL/TLS so I skipped 3rd point of [https://www.elastic.co/guide/en/x-pack/current/monitoring-logstash.html](https://www.elastic.co/guide/en/x-pack/current/monitoring-logstash.html) .

I am still getting the same error .

---

<div class="post-metadata">

**Author:** ![jsvd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvd/32/6203_2.png) [@jsvd](https://discuss.elastic.co/u/jsvd)\
**Post date:** [June 19, 2017, 1:38pm UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175/5 "2017-06-19T13:38:16Z")

</div>

can you start logstash with `--log.level=debug` and send me the logs of the first couple of minutes? you can pm a private pastebin link or a gist if you want

---

<div class="post-metadata">

**Author:** ![jsvd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvd/32/6203_2.png) [@jsvd](https://discuss.elastic.co/u/jsvd)\
**Post date:** [June 19, 2017, 1:49pm UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175/6 "2017-06-19T13:49:09Z")

</div>

> [@raghvendra](#):
>
> xpack.monitoring.elasticsearch.url: ["http//master:9200","http//slave-2:9200","http//slave-1:9200"]

Just to double check, the url hosts you've shown above aren't correct urls since they're missing the ` :` character. So instead of:

```auto
xpack.monitoring.elasticsearch.url: ["http//master:9200","http//slave-2:9200","http//slave-1:9200"]

```

should be

```auto
xpack.monitoring.elasticsearch.url: ["http://master:9200","http://slave-2:9200","http://slave-1:9200"]

```

---

<div class="post-metadata">

**Author:** ![raghvendra](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@raghvendra](https://discuss.elastic.co/u/raghvendra)\
**Post date:** [June 20, 2017, 5:52am UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175/7 "2017-06-20T05:52:47Z")

</div>

Hi .. thanks for your reply.

I have updated my logtash.yml . Now it's working fine . It's pumping data into ES . But it' s still giving some warnings .  
Warnings are -

> [2017-06-20T11:01:07,501][INFO][logstash.pipeline] Pipeline .monitoring-logstash started  
> [2017-06-20T11:01:07,544][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=\>{:removed=\>, :added=\>[[http://master:9200/](http://master:9200/), [http://slave-1:9200/](http://slave-1:9200/), [http://slave-2:9200/](http://slave-2:9200/)]}}  
> [2017-06-20T11:01:07,546][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://master:9200/](http://master:9200/), :path=\>"/"}  
> [2017-06-20T11:01:07,632][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>#\<URI::HTTP:0x180062b8 URL:[http://master:9200/](http://master:9200/)\>}  
> [2017-06-20T11:01:07,632][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://slave-1:9200/](http://slave-1:9200/), :path=\>"/"}  
> [2017-06-20T11:01:07,655][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>#\<URI::HTTP:0x2c86d8f4 URL:[http://slave-1:9200/](http://slave-1:9200/)\>}  
> [2017-06-20T11:01:07,656][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://slave-2:9200/](http://slave-2:9200/), :path=\>"/"}  
> [2017-06-20T11:01:07,709][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>#\<URI::HTTP:0x1095b0b4 URL:[http://slave-2:9200/](http://slave-2:9200/)\>}  
> [2017-06-20T11:01:07,710][INFO][logstash.outputs.elasticsearch] Using mapping template from {:path=\>nil}  
> [2017-06-20T11:01:07,970][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage\_template=\>{"template"=\>"logstash-_", "version"=\>50001, "settings"=\>{"index.refresh\_interval"=\>"5s"}, "mappings"=\>{"default"=\>{"\_all"=\>{"enabled"=\>true, "norms"=\>false}, "dynamic\_templates"=\>[{"message\_field"=\>{"path\_match"=\>"message", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false}}}, {"string\_fields"=\>{"match"=\>"_", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false, "fields"=\>{"keyword"=\>{"type"=\>"keyword"}}}}}], "properties"=\>{"@timestamp"=\>{"type"=\>"date", "include\_in\_all"=\>false}, "@version"=\>{"type"=\>"keyword", "include\_in\_all"=\>false}, "geoip"=\>{"dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip"}, "location"=\>{"type"=\>"geo\_point"}, "latitude"=\>{"type"=\>"half\_float"}, "longitude"=\>{"type"=\>"half\_float"}}}}}}}}  
> [2017-06-20T11:01:07,982][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::Elasticsearch", :hosts=\>[#\<URI::Generic:0x7cc6648c URL://master\>, #\<URI::Generic:0x20fef014 URL://slave-1\>, #\<URI::Generic:0x72e7904d URL://slave-2\>]}  
> [2017-06-20T11:01:08,031][INFO][logstash.filters.geoip] Using geoip database {:path=\>"/home/mywavia/Downloads/logstash-5.4.0/vendor/bundle/jruby/1.9/gems/logstash-filter-geoip-4.0.4-java/vendor/GeoLite2-City.mmdb"}  
> [2017-06-20T11:01:08,106][INFO][logstash.pipeline] Starting pipeline {"id"=\>"main", "pipeline.workers"=\>2, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>5, "pipeline.max\_inflight"=\>250}  
> [2017-06-20T11:01:08,501][INFO][logstash.pipeline] Pipeline main started  
> [2017-06-20T11:01:08,605][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600}  
> [2017-06-20T11:01:12,463][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/), :path=\>"/"}  
> [2017-06-20T11:01:12,468][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>#\<URI::HTTP:0x3e9569e6 URL:[http://logstash\_system:xxxxxx@localhost:9200/\_xpack/monitoring/?system\_id=logstash&system\_api\_version=2&interval=1s](http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s)\>, :error\_type=\>LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/)][Manticore::SocketException] Connection refused (Connection refused)"}  
> [2017-06-20T11:01:12,500][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/), :path=\>"/"}  
> [2017-06-20T11:01:12,504][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>#\<URI::HTTP:0x6f63f27a URL:[http://logstash\_system:xxxxxx@localhost:9200/\_xpack/monitoring/?system\_id=logstash&system\_api\_version=2&interval=1s](http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s)\>, :error\_type=\>LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/)][Manticore::SocketException] Connection refused (Connection refused)"}  
> [2017-06-20T11:01:17,471][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/), :path=\>"/"}  
> [2017-06-20T11:01:17,486][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>#\<URI::HTTP:0x542d53e8 URL:[http://logstash\_system:xxxxxx@localhost:9200/\_xpack/monitoring/?system\_id=logstash&system\_api\_version=2&interval=1s](http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s)\>, :error\_type=\>LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/)][Manticore::SocketException] Connection refused (Connection refused)"}  
> [2017-06-20T11:01:17,506][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/), :path=\>"/"}  
> [2017-06-20T11:01:17,514][WARN][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>#\<URI::HTTP:0x2bb4dc7 URL:[http://logstash\_system:xxxxxx@localhost:9200/\_xpack/monitoring/?system\_id=logstash&system\_api\_version=2&interval=1s](http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s)\>, :error\_type=\>LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/)][Manticore::SocketException] Connection refused (Connection refused)"}  
> [2017-06-20T11:01:17,534][WARN][logstash.outputs.elasticsearch] Marking url as dead. Last error: [LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable:

and So on.  
But I can't see logstash instance in monitoring and management section through kibana.  
here is the scrrenshot -\>

 ![](https://us1.discourse-cdn.com/elastic/original/3X/d/0/d0977092247d6272bba724c2b4ef28e0f307a67b.png)

Monitoring section is giving this-\>

 ![](https://us1.discourse-cdn.com/elastic/original/3X/3/0/30d09d465657169cb457a27622173821ce150212.png)

Can you help me with this \>

---

<div class="post-metadata">

**Author:** ![raghvendra](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@raghvendra](https://discuss.elastic.co/u/raghvendra)\
**Post date:** [June 20, 2017, 5:53am UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175/8 "2017-06-20T05:53:19Z")

</div>

It's corresponding kibana trash is -

> log [05:31:19.843] [info][status][ui settings] Status changed from uninitialized to green - Ready  
> request [05:31:55.504] [error][monitoring-ui] TypeError: Cannot read property 'type' of null  
> at clusters.reduce (/home/mywavia/kibana-5.4.0-linux-x86\_64/plugins/x-pack/plugins/monitoring/server/cluster\_alerts/alerts\_clusters\_aggregation.js:95:73)  
> at Array.reduce (native)  
> at callWithRequest.then.result (/home/mywavia/kibana-5.4.0-linux-x86\_64/plugins/x-pack/plugins/monitoring/server/cluster\_alerts/alerts\_clusters\_aggregation.js:67:21)  
> at tryCatcher (/home/mywavia/kibana-5.4.0-linux-x86\_64/node\_modules/bluebird/js/main/util.js:26:23)  
> at Promise.\_settlePromiseFromHandler (/home/mywavia/kibana-5.4.0-linux-x86\_64/node\_modules/bluebird/js/main/promise.js:503:31)  
> at Promise.\_settlePromiseAt (/home/mywavia/kibana-5.4.0-linux-x86\_64/node\_modules/bluebird/js/main/promise.js:577:18)  
> at Promise.\_settlePromises (/home/mywavia/kibana-5.4.0-linux-x86\_64/node\_modules/bluebird/js/main/promise.js:693:14)  
> at Async.\_drainQueue (/home/mywavia/kibana-5.4.0-linux-x86\_64/node\_modules/bluebird/js/main/async.js:123:16)  
> at Async.\_drainQueues (/home/mywavia/kibana-5.4.0-linux-x86\_64/node\_modules/bluebird/js/main/async.js:133:10)  
> at Immediate.Async.drainQueues (/home/mywavia/kibana-5.4.0-linux-x86\_64/node\_modules/bluebird/js/main/async.js:15:14)  
> at runCallback (timers.js:666:20)  
> at tryOnImmediate (timers.js:639:5)  
> at processImmediate [as \_immediateCallback] (timers.js:611:5)

---

<div class="post-metadata">

**Author:** ![jsvd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvd/32/6203_2.png) [@jsvd](https://discuss.elastic.co/u/jsvd)\
**Post date:** [June 20, 2017, 9:30am UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175/9 "2017-06-20T09:30:09Z")

</div>

can you show me the pipeline configuration you're using, or even just the output section of it?

---

<div class="post-metadata">

**Author:** ![raghvendra](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@raghvendra](https://discuss.elastic.co/u/raghvendra)\
**Post date:** [June 20, 2017, 9:36am UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175/10 "2017-06-20T09:36:33Z")

</div>

yeah ..

Right now , I have disabled x pack security . I have added my hosts file and system recognizes master , slave-1 and slave-2.

![](https://us1.discourse-cdn.com/elastic/original/3X/f/5/f5d6e619c2b5a2f7c62e4e10236cc1ee4b293118.png)

---

<div class="post-metadata">

**Author:** ![jsvd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvd/32/6203_2.png) [@jsvd](https://discuss.elastic.co/u/jsvd)\
**Post date:** [June 20, 2017, 9:42am UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175/11 "2017-06-20T09:42:51Z")

</div>

interesting, I don't see where the localhost is coming from if both your pipeline and xpack settings pointed to master/slave hostnames

---

<div class="post-metadata">

**Author:** ![raghvendra](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@raghvendra](https://discuss.elastic.co/u/raghvendra)\
**Post date:** [June 20, 2017, 9:45am UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175/12 "2017-06-20T09:45:52Z")

</div>

yeah . i haven't mentioned localhost anywhere . I have used IP instead of localhost . I don't know why it is still going to localhost .

---

<div class="post-metadata">

**Author:** ![jsvd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvd/32/6203_2.png) [@jsvd](https://discuss.elastic.co/u/jsvd)\
**Post date:** [June 20, 2017, 9:48am UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175/13 "2017-06-20T09:48:20Z")

</div>

everything (es,ls,kibana) is version 5.4.0 right? I suggest that you run with `--log.level=debug --debug.config` and post the first few minutes of logging. please obscure anything you don't want shown in the logs, or send them privately

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 18, 2017, 9:48am UTC](https://discuss.elastic.co/t/x-pack-installation-with-logstash/89175/14 "2017-07-18T09:48:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.