ES and logstash are nit working well with each other


(David Montgomery) #1

Hi,

I am, rather concerned that ES is not working with logstash index server.

I start logstash index server like this:

/usr/local/share/logstash-1.4.1/bin/logstash -f
/usr/local/share/logstash.indexer.config

Using milestone 2 input plugin 'redis'. This plugin should be stable, but
if you see strange behavior, please let us know! For more information on
plugin milestones, see
http://logstash.net/docs/1.4.1/plugin-milestones{:level=>:warn}
log4j, [2014-05-29T00:33:12.473] WARN: org.elasticsearch.discovery:
[logstash-do-logstash-sf-development-20140527082230-2162-2010] waited for
30s and no initial state was set by the discovery
Exception in thread ">output"
org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(org/elasticsearch/action/support/master/TransportMasterNodeOperationAction.java:180)
at
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(org/elasticsearch/cluster/service/InternalClusterService.java:492)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor.java:615)
at java.lang.Thread.run(java/lang/Thread.java:744)

Here is my config:

input {
redis {
host => "redis.queue.do.development.sf.test.com"
data_type => "list"
key => "logstash"
codec => json
}
}

output {
stdout { }
elasticsearch {
host => "eslogs.do.development.sf.test.com"
}
}

Yes..ES us running and Yes ports are open for logstash

So..whats the deal?

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CACF%2B8rpLhKKFPv%2BZ_JpwiR%3DuNUJLrjm7PGQ0z6dR%2BF04dBytyQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Mark Walkom) #2

You should take this to the LS group for the best response :slight_smile:

But, is ES running?

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 29 May 2014 14:43, David Montgomery davidmontgomery@gmail.com wrote:

Hi,

I am, rather concerned that ES is not working with logstash index server.

I start logstash index server like this:

/usr/local/share/logstash-1.4.1/bin/logstash -f
/usr/local/share/logstash.indexer.config

Using milestone 2 input plugin 'redis'. This plugin should be stable, but
if you see strange behavior, please let us know! For more information on
plugin milestones, see http://logstash.net/docs/1.4.1/plugin-milestones{:level=>:warn}
log4j, [2014-05-29T00:33:12.473] WARN: org.elasticsearch.discovery:
[logstash-do-logstash-sf-development-20140527082230-2162-2010] waited for
30s and no initial state was set by the discovery
Exception in thread ">output"
org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(org/elasticsearch/action/support/master/TransportMasterNodeOperationAction.java:180)
at
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(org/elasticsearch/cluster/service/InternalClusterService.java:492)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor.java:615)
at java.lang.Thread.run(java/lang/Thread.java:744)

Here is my config:

input {
redis {
host => "redis.queue.do.development.sf.test.com"
data_type => "list"
key => "logstash"
codec => json
}
}

output {
stdout { }
elasticsearch {
host => "eslogs.do.development.sf.test.com"
}
}

Yes..ES us running and Yes ports are open for logstash

So..whats the deal?

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CACF%2B8rpLhKKFPv%2BZ_JpwiR%3DuNUJLrjm7PGQ0z6dR%2BF04dBytyQ%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CACF%2B8rpLhKKFPv%2BZ_JpwiR%3DuNUJLrjm7PGQ0z6dR%2BF04dBytyQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624ZwKqR32YuDFhkXmHFKqGciXkZ4WQ437Sn78Mwyd6Yhxg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(David Montgomery) #3

PS...and how is this possible????? I feel so bad I bought the kindle
logstash book:(

I changed to host rather than bind host. I mean..wow..I have ports open.
See? ufw satus ===> 9200:9400/tcp ALLOW
my.logstash.ipaddress

I have ES running
service elasticsearch status

  • elasticsearch is running

/usr/local/share/logstash-1.4.1/bin/logstash -f
/usr/local/share/logstash.indexer.configUsing milestone 2 input plugin
'redis'. This plugin should be stable, but if you see strange behavior,
please let us know! For more information on plugin milestones, see
http://logstash.net/docs/1.4.1/plugin-milestones {:level=>:warn}
Exception in thread ">output"
org.elasticsearch.transport.BindTransportException: Failed to bind to
[9300-9400]
at
org.elasticsearch.transport.netty.NettyTransport.doStart(org/elasticsearch/transport/netty/NettyTransport.java:380)
at
org.elasticsearch.common.component.AbstractLifecycleComponent.start(org/elasticsearch/common/component/AbstractLifecycleComponent.java:85)
at
org.elasticsearch.transport.TransportService.doStart(org/elasticsearch/transport/TransportService.java:92)
at
org.elasticsearch.common.component.AbstractLifecycleComponent.start(org/elasticsearch/common/component/AbstractLifecycleComponent.java:85)
at
org.elasticsearch.node.internal.InternalNode.start(org/elasticsearch/node/internal/InternalNode.java:229)
at
org.elasticsearch.node.NodeBuilder.node(org/elasticsearch/node/NodeBuilder.java:166)
at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:606)
at
RUBY.build_client(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:198)
at
RUBY.client(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:15)
at
RUBY.initialize(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:157)
at
RUBY.register(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch.rb:250)
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
at
RUBY.outputworker(/usr/local/share/logstash-1.4.1/lib/logstash/pipeline.rb:220)
at
RUBY.start_outputs(/usr/local/share/logstash-1.4.1/lib/logstash/pipeline.rb:152)
at java.lang.Thread.run(java/lang/Thread.java:744)

On Thu, May 29, 2014 at 12:43 PM, David Montgomery <
davidmontgomery@gmail.com> wrote:

Hi,

I am, rather concerned that ES is not working with logstash index server.

I start logstash index server like this:

/usr/local/share/logstash-1.4.1/bin/logstash -f
/usr/local/share/logstash.indexer.config

Using milestone 2 input plugin 'redis'. This plugin should be stable, but
if you see strange behavior, please let us know! For more information on
plugin milestones, see http://logstash.net/docs/1.4.1/plugin-milestones{:level=>:warn}
log4j, [2014-05-29T00:33:12.473] WARN: org.elasticsearch.discovery:
[logstash-do-logstash-sf-development-20140527082230-2162-2010] waited for
30s and no initial state was set by the discovery
Exception in thread ">output"
org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(org/elasticsearch/action/support/master/TransportMasterNodeOperationAction.java:180)
at
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(org/elasticsearch/cluster/service/InternalClusterService.java:492)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor.java:615)
at java.lang.Thread.run(java/lang/Thread.java:744)

Here is my config:

input {
redis {
host => "redis.queue.do.development.sf.test.com"
data_type => "list"
key => "logstash"
codec => json
}
}

output {
stdout { }
elasticsearch {
host => "eslogs.do.development.sf.test.com"
}
}

Yes..ES us running and Yes ports are open for logstash

So..whats the deal?

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CACF%2B8rpwG%3D3n3PPpsqBhsDFTm9t9tbMTkhwNSAx%2BuForbGHgrg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(David Montgomery) #4

Before I give up on logstash..I now placed elastic search on the same
server using the below.

input {
redis {
host => "redis.queue.do.development.sf.test.com"
data_type => "list"
key => "logstash"
codec => json
}
}

output {
stdout { }
elasticsearch {
bind_host => "127.0.0.1"
}
}

I get this erorr I am using 1.4.1for logstash and
0.90.9 for es.

'/usr/local/share/logstash-1.4.1/bin/logstash -f
/usr/local/share/logstash.indexer.config
Using milestone 2 input plugin 'redis'. This plugin should be stable, but
if you see strange behavior, please let us know! For more information on
plugin milestones, see
http://logstash.net/docs/1.4.1/plugin-milestones{:level=>:warn}
log4j, [2014-05-29T11:18:31.923] WARN: org.elasticsearch.discovery:
[logstash-do-logstash-sf-development-20140527082230-16645-2010] waited for
30s and no initial state was set by the discovery
Exception in thread ">output"
org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(org/elasticsearch/action/support/master/TransportMasterNodeOperationAction.java:180)
at
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(org/elasticsearch/cluster/service/InternalClusterService.java:492)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor.java:615)
at java.lang.Thread.run(java/lang/Thread.java:744)

On Thu, May 29, 2014 at 1:15 PM, David Montgomery <davidmontgomery@gmail.com

wrote:

PS...and how is this possible????? I feel so bad I bought the kindle
logstash book:(

I changed to host rather than bind host. I mean..wow..I have ports open.
See? ufw satus ===> 9200:9400/tcp ALLOW
my.logstash.ipaddress

I have ES running
service elasticsearch status

  • elasticsearch is running

/usr/local/share/logstash-1.4.1/bin/logstash -f
/usr/local/share/logstash.indexer.configUsing milestone 2 input plugin
'redis'. This plugin should be stable, but if you see strange behavior,
please let us know! For more information on plugin milestones, see
http://logstash.net/docs/1.4.1/plugin-milestones {:level=>:warn}
Exception in thread ">output"
org.elasticsearch.transport.BindTransportException: Failed to bind to
[9300-9400]
at
org.elasticsearch.transport.netty.NettyTransport.doStart(org/elasticsearch/transport/netty/NettyTransport.java:380)
at
org.elasticsearch.common.component.AbstractLifecycleComponent.start(org/elasticsearch/common/component/AbstractLifecycleComponent.java:85)
at
org.elasticsearch.transport.TransportService.doStart(org/elasticsearch/transport/TransportService.java:92)
at
org.elasticsearch.common.component.AbstractLifecycleComponent.start(org/elasticsearch/common/component/AbstractLifecycleComponent.java:85)
at
org.elasticsearch.node.internal.InternalNode.start(org/elasticsearch/node/internal/InternalNode.java:229)
at
org.elasticsearch.node.NodeBuilder.node(org/elasticsearch/node/NodeBuilder.java:166)
at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:606)
at
RUBY.build_client(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:198)
at
RUBY.client(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:15)
at
RUBY.initialize(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:157)
at
RUBY.register(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch.rb:250)
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
at
RUBY.outputworker(/usr/local/share/logstash-1.4.1/lib/logstash/pipeline.rb:220)
at
RUBY.start_outputs(/usr/local/share/logstash-1.4.1/lib/logstash/pipeline.rb:152)
at java.lang.Thread.run(java/lang/Thread.java:744)

On Thu, May 29, 2014 at 12:43 PM, David Montgomery <
davidmontgomery@gmail.com> wrote:

Hi,

I am, rather concerned that ES is not working with logstash index server.

I start logstash index server like this:

/usr/local/share/logstash-1.4.1/bin/logstash -f
/usr/local/share/logstash.indexer.config

Using milestone 2 input plugin 'redis'. This plugin should be stable, but
if you see strange behavior, please let us know! For more information on
plugin milestones, see http://logstash.net/docs/1.4.1/plugin-milestones{:level=>:warn}
log4j, [2014-05-29T00:33:12.473] WARN: org.elasticsearch.discovery:
[logstash-do-logstash-sf-development-20140527082230-2162-2010] waited for
30s and no initial state was set by the discovery
Exception in thread ">output"
org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(org/elasticsearch/action/support/master/TransportMasterNodeOperationAction.java:180)
at
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(org/elasticsearch/cluster/service/InternalClusterService.java:492)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor.java:615)
at java.lang.Thread.run(java/lang/Thread.java:744)

Here is my config:

input {
redis {
host => "redis.queue.do.development.sf.test.com"
data_type => "list"
key => "logstash"
codec => json
}
}

output {
stdout { }
elasticsearch {
host => "eslogs.do.development.sf.test.com"
}
}

Yes..ES us running and Yes ports are open for logstash

So..whats the deal?

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CACF%2B8rpwPQuRkxD12GNp5eS_u5%3D%2Bdz-6f5Go1-pQtjnWa76iMg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(David Pilato) #5

Elasticsearch output is using es 1.x IIRC. You can not mix versions.
You need to upgrade es or downgrade logstash or use elasticsearch HTTP output.

My 2 cents

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 29 mai 2014 à 17:21, David Montgomery davidmontgomery@gmail.com a écrit :

Before I give up on logstash..I now placed elastic search on the same server using the below.

input {
redis {
host => "redis.queue.do.development.sf.test.com"
data_type => "list"
key => "logstash"
codec => json
}
}

output {
stdout { }
elasticsearch {
bind_host => "127.0.0.1"
}
}

I get this erorr I am using 1.4.1for logstash and
0.90.9 for es.

'/usr/local/share/logstash-1.4.1/bin/logstash -f /usr/local/share/logstash.indexer.config
Using milestone 2 input plugin 'redis'. This plugin should be stable, but if you see strange behavior, please let us know! For more information on plugin milestones, see http://logstash.net/docs/1.4.1/plugin-milestones {:level=>:warn}
log4j, [2014-05-29T11:18:31.923] WARN: org.elasticsearch.discovery: [logstash-do-logstash-sf-development-20140527082230-16645-2010] waited for 30s and no initial state was set by the discovery
Exception in thread ">output" org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
at org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(org/elasticsearch/action/support/master/TransportMasterNodeOperationAction.java:180)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(org/elasticsearch/cluster/service/InternalClusterService.java:492)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor.java:615)
at java.lang.Thread.run(java/lang/Thread.java:744)

On Thu, May 29, 2014 at 1:15 PM, David Montgomery davidmontgomery@gmail.com wrote:
PS...and how is this possible????? I feel so bad I bought the kindle logstash book:(

I changed to host rather than bind host. I mean..wow..I have ports open. See? ufw satus ===> 9200:9400/tcp ALLOW my.logstash.ipaddress

I have ES running
service elasticsearch status

  • elasticsearch is running

/usr/local/share/logstash-1.4.1/bin/logstash -f /usr/local/share/logstash.indexer.configUsing milestone 2 input plugin 'redis'. This plugin should be stable, but if you see strange behavior, please let us know! For more information on plugin milestones, see http://logstash.net/docs/1.4.1/plugin-milestones {:level=>:warn}
Exception in thread ">output" org.elasticsearch.transport.BindTransportException: Failed to bind to [9300-9400]
at org.elasticsearch.transport.netty.NettyTransport.doStart(org/elasticsearch/transport/netty/NettyTransport.java:380)
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(org/elasticsearch/common/component/AbstractLifecycleComponent.java:85)
at org.elasticsearch.transport.TransportService.doStart(org/elasticsearch/transport/TransportService.java:92)
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(org/elasticsearch/common/component/AbstractLifecycleComponent.java:85)
at org.elasticsearch.node.internal.InternalNode.start(org/elasticsearch/node/internal/InternalNode.java:229)
at org.elasticsearch.node.NodeBuilder.node(org/elasticsearch/node/NodeBuilder.java:166)
at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:606)
at RUBY.build_client(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:198)
at RUBY.client(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:15)
at RUBY.initialize(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:157)
at RUBY.register(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch.rb:250)
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
at RUBY.outputworker(/usr/local/share/logstash-1.4.1/lib/logstash/pipeline.rb:220)
at RUBY.start_outputs(/usr/local/share/logstash-1.4.1/lib/logstash/pipeline.rb:152)
at java.lang.Thread.run(java/lang/Thread.java:744)

On Thu, May 29, 2014 at 12:43 PM, David Montgomery davidmontgomery@gmail.com wrote:
Hi,

I am, rather concerned that ES is not working with logstash index server.

I start logstash index server like this:

/usr/local/share/logstash-1.4.1/bin/logstash -f /usr/local/share/logstash.indexer.config

Using milestone 2 input plugin 'redis'. This plugin should be stable, but if you see strange behavior, please let us know! For more information on plugin milestones, see http://logstash.net/docs/1.4.1/plugin-milestones {:level=>:warn}
log4j, [2014-05-29T00:33:12.473] WARN: org.elasticsearch.discovery: [logstash-do-logstash-sf-development-20140527082230-2162-2010] waited for 30s and no initial state was set by the discovery
Exception in thread ">output" org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
at org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(org/elasticsearch/action/support/master/TransportMasterNodeOperationAction.java:180)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(org/elasticsearch/cluster/service/InternalClusterService.java:492)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor.java:615)
at java.lang.Thread.run(java/lang/Thread.java:744)

Here is my config:

input {
redis {
host => "redis.queue.do.development.sf.test.com"
data_type => "list"
key => "logstash"
codec => json
}
}

output {
stdout { }
elasticsearch {
host => "eslogs.do.development.sf.test.com"
}
}

Yes..ES us running and Yes ports are open for logstash

So..whats the deal?

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CACF%2B8rpwPQuRkxD12GNp5eS_u5%3D%2Bdz-6f5Go1-pQtjnWa76iMg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7A285E35-E405-456B-B739-C47115D37312%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.


(Mark Walkom) #6

Use the HTTP output and save yourself some trouble.

Also, upgrade ES :slight_smile:

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 30 May 2014 01:38, David Pilato david@pilato.fr wrote:

Elasticsearch output is using es 1.x IIRC. You can not mix versions.
You need to upgrade es or downgrade logstash or use elasticsearch HTTP
output.

My 2 cents

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 29 mai 2014 à 17:21, David Montgomery davidmontgomery@gmail.com a
écrit :

Before I give up on logstash..I now placed elastic search on the same
server using the below.

input {
redis {
host => "redis.queue.do.development.sf.test.com"
data_type => "list"
key => "logstash"
codec => json
}
}

output {
stdout { }
elasticsearch {
bind_host => "127.0.0.1"
}
}

I get this erorr I am using 1.4.1for logstash and
0.90.9 for es.

'/usr/local/share/logstash-1.4.1/bin/logstash -f
/usr/local/share/logstash.indexer.config
Using milestone 2 input plugin 'redis'. This plugin should be stable, but
if you see strange behavior, please let us know! For more information on
plugin milestones, see http://logstash.net/docs/1.4.1/plugin-milestones
{:level=>:warn}
log4j, [2014-05-29T11:18:31.923] WARN: org.elasticsearch.discovery:
[logstash-do-logstash-sf-development-20140527082230-16645-2010] waited for
30s and no initial state was set by the discovery
Exception in thread ">output"
org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(org/elasticsearch/action/support/master/TransportMasterNodeOperationAction.java:180)
at
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(org/elasticsearch/cluster/service/InternalClusterService.java:492)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor.java:615)
at java.lang.Thread.run(java/lang/Thread.java:744)

On Thu, May 29, 2014 at 1:15 PM, David Montgomery <
davidmontgomery@gmail.com> wrote:

PS...and how is this possible????? I feel so bad I bought the kindle
logstash book:(

I changed to host rather than bind host. I mean..wow..I have ports
open. See? ufw satus ===> 9200:9400/tcp ALLOW
my.logstash.ipaddress

I have ES running
service elasticsearch status

  • elasticsearch is running

/usr/local/share/logstash-1.4.1/bin/logstash -f
/usr/local/share/logstash.indexer.configUsing milestone 2 input plugin
'redis'. This plugin should be stable, but if you see strange behavior,
please let us know! For more information on plugin milestones, see
http://logstash.net/docs/1.4.1/plugin-milestones {:level=>:warn}
Exception in thread ">output"
org.elasticsearch.transport.BindTransportException: Failed to bind to
[9300-9400]
at
org.elasticsearch.transport.netty.NettyTransport.doStart(org/elasticsearch/transport/netty/NettyTransport.java:380)
at
org.elasticsearch.common.component.AbstractLifecycleComponent.start(org/elasticsearch/common/component/AbstractLifecycleComponent.java:85)
at
org.elasticsearch.transport.TransportService.doStart(org/elasticsearch/transport/TransportService.java:92)
at
org.elasticsearch.common.component.AbstractLifecycleComponent.start(org/elasticsearch/common/component/AbstractLifecycleComponent.java:85)
at
org.elasticsearch.node.internal.InternalNode.start(org/elasticsearch/node/internal/InternalNode.java:229)
at
org.elasticsearch.node.NodeBuilder.node(org/elasticsearch/node/NodeBuilder.java:166)
at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:606)
at
RUBY.build_client(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:198)
at
RUBY.client(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:15)
at
RUBY.initialize(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:157)
at
RUBY.register(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch.rb:250)
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
at
RUBY.outputworker(/usr/local/share/logstash-1.4.1/lib/logstash/pipeline.rb:220)
at
RUBY.start_outputs(/usr/local/share/logstash-1.4.1/lib/logstash/pipeline.rb:152)
at java.lang.Thread.run(java/lang/Thread.java:744)

On Thu, May 29, 2014 at 12:43 PM, David Montgomery <
davidmontgomery@gmail.com> wrote:

Hi,

I am, rather concerned that ES is not working with logstash index server.

I start logstash index server like this:

/usr/local/share/logstash-1.4.1/bin/logstash -f
/usr/local/share/logstash.indexer.config

Using milestone 2 input plugin 'redis'. This plugin should be stable,
but if you see strange behavior, please let us know! For more information
on plugin milestones, see
http://logstash.net/docs/1.4.1/plugin-milestones {:level=>:warn}
log4j, [2014-05-29T00:33:12.473] WARN: org.elasticsearch.discovery:
[logstash-do-logstash-sf-development-20140527082230-2162-2010] waited for
30s and no initial state was set by the discovery
Exception in thread ">output"
org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(org/elasticsearch/action/support/master/TransportMasterNodeOperationAction.java:180)
at
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(org/elasticsearch/cluster/service/InternalClusterService.java:492)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor.java:615)
at java.lang.Thread.run(java/lang/Thread.java:744)

Here is my config:

input {
redis {
host => "redis.queue.do.development.sf.test.com"
data_type => "list"
key => "logstash"
codec => json
}
}

output {
stdout { }
elasticsearch {
host => "eslogs.do.development.sf.test.com"
}
}

Yes..ES us running and Yes ports are open for logstash

So..whats the deal?

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CACF%2B8rpwPQuRkxD12GNp5eS_u5%3D%2Bdz-6f5Go1-pQtjnWa76iMg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CACF%2B8rpwPQuRkxD12GNp5eS_u5%3D%2Bdz-6f5Go1-pQtjnWa76iMg%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7A285E35-E405-456B-B739-C47115D37312%40pilato.fr
https://groups.google.com/d/msgid/elasticsearch/7A285E35-E405-456B-B739-C47115D37312%40pilato.fr?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624aEk8nEWphE6yEZQ9LtGGNBfq%3DB1Dm5ZSmjME-HmPRouQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #7