Hadoop hdfs newScalingExecutorService

Elastic Stack Elasticsearch
1

Hi,

I am trying to run ES on hadoop and its hdfs.
I am running hadoop locally for testing in pseudo-distributed
operation. What I get after starting ES with hadoop plugin is an
error, which I cant fix. Hadoop itself works fine. Versions are
matching (both hadoop and plugin are 0.20.204).

My configuration of the gateway is standard:
gateway:
type: hdfs
hdfs:
uri: hdfs://localhost:8020
path: elastic

What I get is:

[INFO ][node] [Gateway] {0.19.0}[28484]: initializing ...
[INFO ][plugins] [Gateway] loaded [hadoop], sites []
[DEBUG][discovery.zen.ping.multicast] [Gateway] using group
[224.2.2.4], with port [54328], ttl [3], and address [null]
[DEBUG][discovery.zen.ping.unicast] [Gateway] using initial hosts [],
with concurrent_connects [10]
][DEBUG][discovery.zen] [Gateway] using ping.timeout [3s]
[DEBUG][discovery.zen.elect] [Gateway] using minimum_master_nodes [-1]
[DEBUG][discovery.zen.fd] [Gateway] [master] uses ping_interval [1s],
ping_timeout [30s], ping_retries [3]
[DEBUG][discovery.zen.fd] [Gateway] [node] uses ping_interval [1s],
ping_timeout [30s], ping_retries [3]
[DEBUG][indices.recovery] [Gateway] using max_size_per_sec[0b],
concurrent_streams [5], file_chunk_size [100kb], translog_size
[100kb], translog_ops [1000], and compress [true]
[ERROR][bootstrap] {0.19.0}: Initialization Failed ...
1)
NoSuchMethodError[org.elasticsearch.common.util.concurrent.EsExecutors.newScalingExecutorService(IIJLjava/
util/concurrent/TimeUnit;Ljava/util/concurrent/ThreadFactory;)Ljava/
util/concurrent/ThreadPoolExecutor;]

Any ideas how to fix it?

2

Which version of the plugin are you using? Make sure to use one that works with 0.19: GitHub - elastic/elasticsearch-hadoop: Elasticsearch real-time search and analytics natively integrated with Hadoop.

On Sunday, March 4, 2012 at 4:54 AM, duze wrote:

Hi,

I am trying to run ES on hadoop and its hdfs.
I am running hadoop locally for testing in pseudo-distributed
operation. What I get after starting ES with hadoop plugin is an
error, which I cant fix. Hadoop itself works fine. Versions are
matching (both hadoop and plugin are 0.20.204).

My configuration of the gateway is standard:
gateway:
type: hdfs
hdfs:
uri: hdfs://localhost:8020
path: elastic

What I get is:

[INFO ][node] [Gateway] {0.19.0}[28484]: initializing ...
[INFO ][plugins] [Gateway] loaded [hadoop], sites
[DEBUG][discovery.zen.ping.multicast] [Gateway] using group
[224.2.2.4], with port [54328], ttl [3], and address [null]
[DEBUG][discovery.zen.ping.unicast] [Gateway] using initial hosts ,
with concurrent_connects [10]
][DEBUG][discovery.zen] [Gateway] using ping.timeout [3s]
[DEBUG][discovery.zen.elect] [Gateway] using minimum_master_nodes [-1]
[DEBUG][discovery.zen.fd] [Gateway] [master] uses ping_interval [1s],
ping_timeout [30s], ping_retries [3]
[DEBUG][discovery.zen.fd] [Gateway] [node] uses ping_interval [1s],
ping_timeout [30s], ping_retries [3]
[DEBUG][indices.recovery] [Gateway] using max_size_per_sec[0b],
concurrent_streams [5], file_chunk_size [100kb], translog_size
[100kb], translog_ops [1000], and compress [true]
[ERROR][bootstrap] {0.19.0}: Initialization Failed ...
1)
NoSuchMethodError[org.elasticsearch.common.util.concurrent.EsExecutors.newScalingExecutorService(IIJLjava/
util/concurrent/TimeUnit;Ljava/util/concurrent/ThreadFactory;)Ljava/
util/concurrent/ThreadPoolExecutor;]

Any ideas how to fix it?

3

Thanks for response,

I checked versions, everything should be correct.
hadoop plugin 1.1.0, Elasticsearch 0.19.0 and embedded hadoop plugin
0.20.204.0, hadoop 0.20.204.0
I tried to reinstall the plugin.. (you never know), but it is the same.
bin/plugin -install elasticsearch/elasticsearch-hadoop/1.1.0

On Sunday, March 4, 2012 10:32:34 AM UTC+1, kimchy wrote:

Which version of the plugin are you using? Make sure to use one that
works with 0.19: GitHub - elastic/elasticsearch-hadoop: Elasticsearch real-time search and analytics natively integrated with Hadoop.

On Sunday, March 4, 2012 at 4:54 AM, duze wrote:

Hi,

I am trying to run ES on hadoop and its hdfs.
I am running hadoop locally for testing in pseudo-distributed
operation. What I get after starting ES with hadoop plugin is an
error, which I cant fix. Hadoop itself works fine. Versions are
matching (both hadoop and plugin are 0.20.204).

My configuration of the gateway is standard:
gateway:
type: hdfs
hdfs:
uri: hdfs://localhost:8020
path: elastic

What I get is:

[INFO ][node] [Gateway] {0.19.0}[28484]: initializing ...
[INFO ][plugins] [Gateway] loaded [hadoop], sites
[DEBUG][discovery.zen.ping.multicast] [Gateway] using group
[224.2.2.4], with port [54328], ttl [3], and address [null]
[DEBUG][discovery.zen.ping.unicast] [Gateway] using initial hosts ,
with concurrent_connects [10]
][DEBUG][discovery.zen] [Gateway] using ping.timeout [3s]
[DEBUG][discovery.zen.elect] [Gateway] using minimum_master_nodes [-1]
[DEBUG][discovery.zen.fd] [Gateway] [master] uses ping_interval [1s],
ping_timeout [30s], ping_retries [3]
[DEBUG][discovery.zen.fd] [Gateway] [node] uses ping_interval [1s],
ping_timeout [30s], ping_retries [3]
[DEBUG][indices.recovery] [Gateway] using max_size_per_sec[0b],
concurrent_streams [5], file_chunk_size [100kb], translog_size
[100kb], translog_ops [1000], and compress [true]
[ERROR][bootstrap] {0.19.0}: Initialization Failed ...
1)

NoSuchMethodError[org.elasticsearch.common.util.concurrent.EsExecutors.newScalingExecutorService(IIJLjava/
util/concurrent/TimeUnit;Ljava/util/concurrent/ThreadFactory;)Ljava/
util/concurrent/ThreadPoolExecutor;]

Any ideas how to fix it?

4

Strange… . I rebuilt against 0.19.0 and published a new version of the plugin (1.2.0), which should work now.

On Sunday, March 4, 2012 at 3:04 PM, duze wrote:

Thanks for response,

I checked versions, everything should be correct.
hadoop plugin 1.1.0, Elasticsearch 0.19.0 and embedded hadoop plugin 0.20.204.0, hadoop 0.20.204.0
I tried to reinstall the plugin.. (you never know), but it is the same.
bin/plugin -install elasticsearch/elasticsearch-hadoop/1.1.0

On Sunday, March 4, 2012 10:32:34 AM UTC+1, kimchy wrote:

Which version of the plugin are you using? Make sure to use one that works with 0.19: GitHub - elastic/elasticsearch-hadoop: Elasticsearch real-time search and analytics natively integrated with Hadoop.

On Sunday, March 4, 2012 at 4:54 AM, duze wrote:

Hi,

I am trying to run ES on hadoop and its hdfs.
I am running hadoop locally for testing in pseudo-distributed
operation. What I get after starting ES with hadoop plugin is an
error, which I cant fix. Hadoop itself works fine. Versions are
matching (both hadoop and plugin are 0.20.204).

My configuration of the gateway is standard:
gateway:
type: hdfs
hdfs:
uri: hdfs://localhost:8020
path: elastic

What I get is:

[INFO ][node] [Gateway] {0.19.0}[28484]: initializing ...
[INFO ][plugins] [Gateway] loaded [hadoop], sites
[DEBUG][discovery.zen.ping.multicast] [Gateway] using group
[224.2.2.4], with port [54328], ttl [3], and address [null]
[DEBUG][discovery.zen.ping.unicast] [Gateway] using initial hosts ,
with concurrent_connects [10]
][DEBUG][discovery.zen] [Gateway] using ping.timeout [3s]
[DEBUG][discovery.zen.elect] [Gateway] using minimum_master_nodes [-1]
[DEBUG][discovery.zen.fd] [Gateway] [master] uses ping_interval [1s],
ping_timeout [30s], ping_retries [3]
[DEBUG][discovery.zen.fd] [Gateway] [node] uses ping_interval [1s],
ping_timeout [30s], ping_retries [3]
[DEBUG][indices.recovery] [Gateway] using max_size_per_sec[0b],
concurrent_streams [5], file_chunk_size [100kb], translog_size
[100kb], translog_ops [1000], and compress [true]
[ERROR][bootstrap] {0.19.0}: Initialization Failed ...
1)
NoSuchMethodError[org.elasticsearch.common.util.concurrent.EsExecutors.newScalingExecutorService(IIJLjava/
util/concurrent/TimeUnit;Ljava/util/concurrent/ThreadFactory;)Ljava/
util/concurrent/ThreadPoolExecutor;]

Any ideas how to fix it?