# Communication Problem between logstash and elasticsearch

**URL:** <https://discuss.elastic.co/t/communication-problem-between-logstash-and-elasticsearch/5714>\
**Category:** Elasticsearch\
**Created:** [October 28, 2011, 4:02am UTC](https://discuss.elastic.co/t/communication-problem-between-logstash-and-elasticsearch/5714 "2011-10-28T04:02:54Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![medas3c\_medas3c](https://avatars.discourse-cdn.com/v4/letter/m/49beb7/32.png) [@medas3c\_medas3c](https://discuss.elastic.co/u/medas3c_medas3c)\
**Post date:** [October 28, 2011, 4:02am UTC](https://discuss.elastic.co/t/communication-problem-between-logstash-and-elasticsearch/5714/1 "2011-10-28T04:02:54Z")

</div>

Hi all,

I have some problems in intergrating logstash with elasticsearch.

## My model is below:

## (input Message Queue) logstash (output elastic search) --\> elasticsearch + logstash web.

Logstash and elasticsearch are installed on two separate servers.

After configuration, something was wrong. I use tcpdump to catch  
packet from logstash server send to elastic server (port 9300). I  
relize that have some metadata send to elastic server, but don't have  
any log data.

## Elasticsearch dump some warning log as:

## [2011-10-28 10:44:13,940][WARN][transport.netty] [Fantastic Four] Exception caught on netty layer [[id: 0x76659bde]] java.net.NoRouteToHostException: No route to host at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java: 592) at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink $Boss.connect(NioClientSocketPipelineSink.java:384) at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink $Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354) at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink $Boss.run(NioClientSocketPipelineSink.java:276) at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java: 108) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker $1.run(DeadLockProofWorker.java:44) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: 1110) at java.util.concurrent.ThreadPoolExecutor $Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:636)

## If I search data from logstash web install on elasticsearch server, i received message

The query 'text:a' resulted the following error:

org.elasticsearch.action.search.SearchPhaseExecutionException: Failed  
to execute phase [initial], No indices / shards to search on,  
requested indices are []

## This may mean you have no data in ElasticSearch.

I think problem not at logstash server because if I stop logstash  
process, and count message queue, I see some element in the queue. If  
i start logstash process, none eleement in the queue.

Below is my configurations:

## Logstash config:

input {  
amqp {  
# ship logs to the 'rawlogs' fanout queue.  
debug =\> true  
type =\> "all"  
host =\> "127.0.0.1"  
exchange\_type =\> "fanout"  
name =\> "elastic"  
}  
amqp {  
# ship logs to the 'rawlogs' fanout queue.  
debug =\> true  
type =\> "all"  
host =\> "127.0.0.1"  
exchange\_type =\> "fanout"  
name =\> "graylog"  
}  
}

output {  
stdout {  
debug =\> true  
}

# If you can't discover using multicast, set the address explicitly

elasticsearch {  
cluster =\> "elasticsearch"  
host =\> "Ip address of Elastic server"  
port =\> "9300"  
index =\> "logstash-%{+YYYY.MM.dd}"  
type =\> "raw\_public\_log"  
}

## }

## Elasticsearch config

# The cluster name

cluster.name: elasticsearch

#cluster:

# name: MyCluster

network:  
host: Ip address of Elastic server

# Path Settings

#path.conf: /path/to/conf  
#path.data: /path/to/data  
path.data: /u03/elasticsearch/data/elasticsearch

path.work: /u03/elasticsearch/work/elasticsearch  
#path.logs: /path/to/logs  
path.logs: /u03/elasticsearch/logs/elasticsearch

# Force all memory to be locked, forcing the JVM to never swap

#bootstrap.mlockall: true

node.master: true  
node.data: true

# Gateway Settings

# Controls when to start the initial recovery process when starting

a new cluster

# allowing for better reused of existing data during recovery.

#gateway.recover\_after\_nodes: 1  
#gateway.recover\_after\_time: 5m  
#gateway.expected\_nodes: 2

# Controls the minimum number of master eligible nodes this node

should "see"

# in order to operate within the cluster.

# Set this to a higher value (2-4) when running more than 2 nodes in

the cluster  
#discovery.zen.minimum\_master\_nodes: 1

# The time to wait for ping responses from other nodes when doing node

discovery  
#discovery.zen.ping.timeout: 3s

# Unicast Discovery (disable multicast)

## discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.enabled: true discovery.zen.ping.unicast.hosts: ["Ip address of Elastic server"]

So I have some questions:

1. How logstash communicate with elasticsearch?
2. What is my problem and how to resolve it?
3. Can you recommend some detail document for elasticsearch (and  
logstash)?

Thanks!

---

<div class="post-metadata">

**Author:** ![Karussell1](https://avatars.discourse-cdn.com/v4/letter/k/50afbb/32.png) [@Karussell1](https://discuss.elastic.co/u/Karussell1)\
**Post date:** [October 31, 2011, 8:28am UTC](https://discuss.elastic.co/t/communication-problem-between-logstash-and-elasticsearch/5714/2 "2011-10-31T08:28:33Z")

</div>

Probably this is more a logstash config related question? Did you ask  
on their mailinglist too?

Peter

On 28 Okt., 05:02, medas3c medas3c [meda...@gmail.com](mailto:meda...@gmail.com) wrote:

> Hi all,
> 
> I have some problems in intergrating logstash with elasticsearch.
> 
> ## My model is below:
> 
> ## (input Message Queue) logstash (output Elasticsearch) --\> elasticsearch + logstash web.
> 
> Logstash and elasticsearch are installed on two separate servers.
> 
> After configuration, something was wrong. I use tcpdump to catch  
> packet from logstash server send to elastic server (port 9300). I  
> relize that have some metadata send to elastic server, but don't have  
> any log data.
> 
> ## Elasticsearch dump some warning log as:
> 
> ## [2011-10-28 10:44:13,940][WARN][transport.netty] [Fantastic Four] Exception caught on netty layer [[id: 0x76659bde]] java.net.NoRouteToHostException: No route to host at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java: 592) at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink $Boss.connect(NioClientSocketPipelineSink.java:384) at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink $Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354) at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink $Boss.run(NioClientSocketPipelineSink.java:276) at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java: 108) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker $1.run(DeadLockProofWorker.java:44) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: 1110) at java.util.concurrent.ThreadPoolExecutor $Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:636)
> 
> ## If I search data from logstash web install on elasticsearch server, i received message
> 
> The query 'text:a' resulted the following error:
> 
> org.elasticsearch.action.search.SearchPhaseExecutionException: Failed  
> to execute phase [initial], No indices / shards to search on,  
> requested indices are
> 
> ## This may mean you have no data in Elasticsearch.
> 
> I think problem not at logstash server because if I stop logstash  
> process, and count message queue, I see some element in the queue. If  
> i start logstash process, none eleement in the queue.
> 
> Below is my configurations:
> 
> ## Logstash config:
> 
> input {  
> amqp {  
> # ship logs to the 'rawlogs' fanout queue.  
> debug =\> true  
> type =\> "all"  
> host =\> "127.0.0.1"  
> exchange\_type =\> "fanout"  
> name =\> "elastic"  
> }  
> amqp {  
> # ship logs to the 'rawlogs' fanout queue.  
> debug =\> true  
> type =\> "all"  
> host =\> "127.0.0.1"  
> exchange\_type =\> "fanout"  
> name =\> "graylog"  
> }
> 
> }
> 
> output {  
> stdout {  
> debug =\> true  
> }
> 
> # If you can't discover using multicast, set the address explicitly
> 
> elasticsearch {  
> cluster =\> "elasticsearch"  
> host =\> "Ip address of Elastic server"  
> port =\> "9300"  
> index =\> "logstash-%{+YYYY.MM.dd}"  
> type =\> "raw\_public\_log"  
> }
> 
> }
> 
> * * *
> 
> ## Elasticsearch config
> 
> # The cluster name
> 
> cluster.name: elasticsearch
> 
> #cluster:
> 
> # name: MyCluster
> 
> network:  
> host: Ip address of Elastic server
> 
> # Path Settings
> 
> #path.conf: /path/to/conf  
> #path.data: /path/to/data  
> path.data: /u03/elasticsearch/data/elasticsearch
> 
> path.work: /u03/elasticsearch/work/elasticsearch  
> #path.logs: /path/to/logs  
> path.logs: /u03/elasticsearch/logs/elasticsearch
> 
> # Force all memory to be locked, forcing the JVM to never swap
> 
> #bootstrap.mlockall: true
> 
> node.master: true  
> node.data: true
> 
> # Gateway Settings
> 
> # Controls when to start the initial recovery process when starting
> 
> a new cluster
> 
> # allowing for better reused of existing data during recovery.
> 
> #gateway.recover\_after\_nodes: 1  
> #gateway.recover\_after\_time: 5m  
> #gateway.expected\_nodes: 2
> 
> # Controls the minimum number of master eligible nodes this node
> 
> should "see"
> 
> # in order to operate within the cluster.
> 
> # Set this to a higher value (2-4) when running more than 2 nodes in
> 
> the cluster  
> #discovery.zen.minimum\_master\_nodes: 1
> 
> # The time to wait for ping responses from other nodes when doing node
> 
> discovery  
> #discovery.zen.ping.timeout: 3s
> 
> # Unicast Discovery (disable multicast)
> 
> ## discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.enabled: true discovery.zen.ping.unicast.hosts: ["Ip address of Elastic server"]
> 
> So I have some questions:
> 
> 1. How logstash communicate with elasticsearch?
> 2. What is my problem and how to resolve it?
> 3. Can you recommend some detail document for elasticsearch (and  
> logstash)?
> 
> Thanks!

---

<div class="post-metadata">

**Author:** ![medas3c\_medas3c](https://avatars.discourse-cdn.com/v4/letter/m/49beb7/32.png) [@medas3c\_medas3c](https://discuss.elastic.co/u/medas3c_medas3c)\
**Post date:** [November 1, 2011, 9:22am UTC](https://discuss.elastic.co/t/communication-problem-between-logstash-and-elasticsearch/5714/3 "2011-11-01T09:22:35Z")

</div>

I have posted it on logstash user groups  
[https://groups.google.com/group/logstash-users/browse\_thread/thread/a78ebf73e26b20b0](https://groups.google.com/group/logstash-users/browse_thread/thread/a78ebf73e26b20b0)

and the problem has been resolved.

Thanks you.

On Oct 31, 3:28 pm, Karussell [tableyourt...@googlemail.com](mailto:tableyourt...@googlemail.com) wrote:

> Probably this is more a logstash config related question? Did you ask  
> on their mailinglist too?
> 
> Peter
> 
> On 28 Okt., 05:02, medas3c medas3c [meda...@gmail.com](mailto:meda...@gmail.com) wrote:
> 
> > Hi all,
> 
> > I have some problems in intergrating logstash with elasticsearch.
> 
> > ## My model is below:
> > 
> > ## (input Message Queue) logstash (output Elasticsearch) --\> elasticsearch + logstash web.
> 
> > Logstash and elasticsearch are installed on two separate servers.
> 
> > After configuration, something was wrong. I use tcpdump to catch  
> > packet from logstash server send to elastic server (port 9300). I  
> > relize that have some metadata send to elastic server, but don't have  
> > any log data.
> 
> > ## Elasticsearch dump some warning log as:
> > 
> > ## [2011-10-28 10:44:13,940][WARN][transport.netty] [Fantastic Four] Exception caught on netty layer [[id: 0x76659bde]] java.net.NoRouteToHostException: No route to host at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java: 592) at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink $Boss.connect(NioClientSocketPipelineSink.java:384) at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink $Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354) at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink $Boss.run(NioClientSocketPipelineSink.java:276) at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java: 108) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker $1.run(DeadLockProofWorker.java:44) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: 1110) at java.util.concurrent.ThreadPoolExecutor $Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:636)
> 
> > ## If I search data from logstash web install on elasticsearch server, i received message
> > 
> > The query 'text:a' resulted the following error:
> 
> > org.elasticsearch.action.search.SearchPhaseExecutionException: Failed  
> > to execute phase [initial], No indices / shards to search on,  
> > requested indices are
> 
> > ## This may mean you have no data in Elasticsearch.
> 
> > I think problem not at logstash server because if I stop logstash  
> > process, and count message queue, I see some element in the queue. If  
> > i start logstash process, none eleement in the queue.
> 
> > Below is my configurations:
> 
> > ## Logstash config:
> > 
> > input {  
> > amqp {  
> > # ship logs to the 'rawlogs' fanout queue.  
> > debug =\> true  
> > type =\> "all"  
> > host =\> "127.0.0.1"  
> > exchange\_type =\> "fanout"  
> > name =\> "elastic"  
> > }  
> > amqp {  
> > # ship logs to the 'rawlogs' fanout queue.  
> > debug =\> true  
> > type =\> "all"  
> > host =\> "127.0.0.1"  
> > exchange\_type =\> "fanout"  
> > name =\> "graylog"  
> > }
> 
> > }
> 
> > output {  
> > stdout {  
> > debug =\> true  
> > }
> 
> > # If you can't discover using multicast, set the address explicitly
> > 
> > elasticsearch {  
> > cluster =\> "elasticsearch"  
> > host =\> "Ip address of Elastic server"  
> > port =\> "9300"  
> > index =\> "logstash-%{+YYYY.MM.dd}"  
> > type =\> "raw\_public\_log"  
> > }
> 
> > }
> 
> > * * *
> 
> > ## Elasticsearch config
> > 
> > # The cluster name
> > 
> > cluster.name: elasticsearch
> 
> > #cluster:
> > 
> > # name: MyCluster
> 
> > network:  
> > host: Ip address of Elastic server
> 
> > # Path Settings
> > 
> > #path.conf: /path/to/conf  
> > #path.data: /path/to/data  
> > path.data: /u03/elasticsearch/data/elasticsearch
> 
> > path.work: /u03/elasticsearch/work/elasticsearch  
> > #path.logs: /path/to/logs  
> > path.logs: /u03/elasticsearch/logs/elasticsearch
> 
> > # Force all memory to be locked, forcing the JVM to never swap
> > 
> > #bootstrap.mlockall: true
> 
> > node.master: true  
> > node.data: true
> > 
> > # Gateway Settings
> > 
> > # Controls when to start the initial recovery process when starting
> > 
> > a new cluster
> > 
> > # allowing for better reused of existing data during recovery.
> > 
> > #gateway.recover\_after\_nodes: 1  
> > #gateway.recover\_after\_time: 5m  
> > #gateway.expected\_nodes: 2
> 
> > # Controls the minimum number of master eligible nodes this node
> > 
> > should "see"
> > 
> > # in order to operate within the cluster.
> > 
> > # Set this to a higher value (2-4) when running more than 2 nodes in
> > 
> > the cluster  
> > #discovery.zen.minimum\_master\_nodes: 1
> 
> > # The time to wait for ping responses from other nodes when doing node
> > 
> > discovery  
> > #discovery.zen.ping.timeout: 3s
> 
> > # Unicast Discovery (disable multicast)
> > 
> > ## discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.enabled: true discovery.zen.ping.unicast.hosts: ["Ip address of Elastic server"]
> 
> > So I have some questions:
> 
> > 1. How logstash communicate with elasticsearch?
> > 2. What is my problem and how to resolve it?
> > 3. Can you recommend some detail document for elasticsearch (and  
> > logstash)?
> 
> > Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:50am UTC](https://discuss.elastic.co/t/communication-problem-between-logstash-and-elasticsearch/5714/4 "2017-07-06T03:50:17Z")

</div>