Missing analyzer plugin causes index creation failure

Elastic Stack Elasticsearch
1

Hi,

I'm using ES 1.0.1 in a 10 node cluster and recently needed to add support
for indexing documents in Japanese. To do that i've installed the Kuromoji
Analysis plugin
https://github.com/elasticsearch/elasticsearch-analysis-kuromoji.
At a later time, when I added a new node to the cluster it joined the
cluster successfully, but I started to see failures when creating new
indexes (any index, in any language) and after the index creation failed
the cluster got into a "red" state where the new indexs' shards remained
Unassigned.
org.elasticsearch.indices.IndexCreationException: [...] failed to create
index
at org.elasticsearch.indices.InternalIndicesService.createIndex(
InternalIndicesService.java:300)
at org.elasticsearch.indices.cluster.IndicesClusterStateService.
applyNewIndices(IndicesClusterStateService.java:306)
at org.elasticsearch.indices.cluster.IndicesClusterStateService.
clusterChanged(IndicesClusterStateService.java:178)
at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(
InternalClusterService.java:424)
at org.elasticsearch.common.util.concurrent.
PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(
PrioritizedEsThreadPoolExecutor.java:134)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: failed
to find analyzer type [kuromoji] or tokenizer for [japanese_snowball]
at org.elasticsearch.index.analysis.AnalysisModule.configure(AnalysisModule
.java:372)
at org.elasticsearch.common.inject.AbstractModule.configure(AbstractModule.
java:60)
at org.elasticsearch.common.inject.spi.Elements$RecordingBinder.install(
Elements.java:204)
at org.elasticsearch.common.inject.spi.Elements.getElements(Elements.java:
85)
at org.elasticsearch.common.inject.InjectorShell$Builder.build(
InjectorShell.java:130)
at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.
java:99)
at org.elasticsearch.common.inject.InjectorImpl.createChildInjector(
InjectorImpl.java:131)
at org.elasticsearch.common.inject.ModulesBuilder.createChildInjector(
ModulesBuilder.java:69)
at org.elasticsearch.indices.InternalIndicesService.createIndex(
InternalIndicesService.java:298)
... 7 more
Caused by: org.elasticsearch.common.settings.NoClassSettingsException: Failed
to load class setting [type] with value [kuromoji]
at org.elasticsearch.common.settings.ImmutableSettings.loadClass(
ImmutableSettings.java:448)
at org.elasticsearch.common.settings.ImmutableSettings.getAsClass(
ImmutableSettings.java:436)
at org.elasticsearch.index.analysis.AnalysisModule.configure(AnalysisModule
.java:356)
... 15 more
Caused by: java.lang.ClassNotFoundException:
org.elasticsearch.index.analysis.kuromoji.KuromojiAnalyzerProvider
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.elasticsearch.common.settings.ImmutableSettings.loadClass(
ImmutableSettings.java:446)
... 17 more

Eventually, I found out that the new node was missing the Kuromoji Analysis
plugin https://github.com/elasticsearch/elasticsearch-analysis-kuromoji.

My question is: why did a missing plugin in one node fail the creation of
any new index? Is that the expected behaviour?

Thanks,
Rafi

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2d1981df-5495-4e4a-930a-2bdd97f213a4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

I think it's expected as long as your index could potentialy being allocated on the node which is missing the plugin.

HTH

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 30 juin 2014 à 10:34, Rafi Aroch rafi.aroch@gmail.com a écrit :

Hi,

I'm using ES 1.0.1 in a 10 node cluster and recently needed to add support for indexing documents in Japanese. To do that i've installed the Kuromoji Analysis plugin.
At a later time, when I added a new node to the cluster it joined the cluster successfully, but I started to see failures when creating new indexes (any index, in any language) and after the index creation failed the cluster got into a "red" state where the new indexs' shards remained Unassigned.
org.elasticsearch.indices.IndexCreationException: [...] failed to create index
at org.elasticsearch.indices.InternalIndicesService.createIndex(InternalIndicesService.java:300)
at org.elasticsearch.indices.cluster.IndicesClusterStateService.applyNewIndices(IndicesClusterStateService.java:306)
at org.elasticsearch.indices.cluster.IndicesClusterStateService.clusterChanged(IndicesClusterStateService.java:178)
at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:424)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:134)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: failed to find analyzer type [kuromoji] or tokenizer for [japanese_snowball]
at org.elasticsearch.index.analysis.AnalysisModule.configure(AnalysisModule.java:372)
at org.elasticsearch.common.inject.AbstractModule.configure(AbstractModule.java:60)
at org.elasticsearch.common.inject.spi.Elements$RecordingBinder.install(Elements.java:204)
at org.elasticsearch.common.inject.spi.Elements.getElements(Elements.java:85)
at org.elasticsearch.common.inject.InjectorShell$Builder.build(InjectorShell.java:130)
at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:99)
at org.elasticsearch.common.inject.InjectorImpl.createChildInjector(InjectorImpl.java:131)
at org.elasticsearch.common.inject.ModulesBuilder.createChildInjector(ModulesBuilder.java:69)
at org.elasticsearch.indices.InternalIndicesService.createIndex(InternalIndicesService.java:298)
... 7 more
Caused by: org.elasticsearch.common.settings.NoClassSettingsException: Failed to load class setting [type] with value [kuromoji]
at org.elasticsearch.common.settings.ImmutableSettings.loadClass(ImmutableSettings.java:448)
at org.elasticsearch.common.settings.ImmutableSettings.getAsClass(ImmutableSettings.java:436)
at org.elasticsearch.index.analysis.AnalysisModule.configure(AnalysisModule.java:356)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.elasticsearch.index.analysis.kuromoji.KuromojiAnalyzerProvider
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.elasticsearch.common.settings.ImmutableSettings.loadClass(ImmutableSettings.java:446)
... 17 more

Eventually, I found out that the new node was missing the Kuromoji Analysis plugin.

My question is: why did a missing plugin in one node fail the creation of any new index? Is that the expected behaviour?

Thanks,
Rafi

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2d1981df-5495-4e4a-930a-2bdd97f213a4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0A6421DE-0251-42D0-AEEB-FD8C1B418018%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

3

Thanks David for your comment.
I was expecting that in case where a shard fails to be allocated on a
specific node, it would try on a different node...

Just an idea, maybe upon an index creation, we know the needed analyzers by
the mapping. We could check which nodes support the index and allocate it's
shards only on supporting nodes and raise a warning instead of failing
altogether.

On Monday, June 30, 2014 11:34:46 AM UTC+3, Rafi Aroch wrote:

Hi,

I'm using ES 1.0.1 in a 10 node cluster and recently needed to add support
for indexing documents in Japanese. To do that i've installed the Kuromoji
Analysis plugin
https://github.com/elasticsearch/elasticsearch-analysis-kuromoji.
At a later time, when I added a new node to the cluster it joined the
cluster successfully, but I started to see failures when creating new
indexes (any index, in any language) and after the index creation failed
the cluster got into a "red" state where the new indexs' shards remained
Unassigned.
org.elasticsearch.indices.IndexCreationException: [...] failed to create
index
at org.elasticsearch.indices.InternalIndicesService.createIndex(
InternalIndicesService.java:300)
at org.elasticsearch.indices.cluster.IndicesClusterStateService.
applyNewIndices(IndicesClusterStateService.java:306)
at org.elasticsearch.indices.cluster.IndicesClusterStateService.
clusterChanged(IndicesClusterStateService.java:178)
at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.
run(InternalClusterService.java:424)
at org.elasticsearch.common.util.concurrent.
PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(
PrioritizedEsThreadPoolExecutor.java:134)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: failed
to find analyzer type [kuromoji] or tokenizer for [japanese_snowball]
at org.elasticsearch.index.analysis.AnalysisModule.configure(
AnalysisModule.java:372)
at org.elasticsearch.common.inject.AbstractModule.configure(
AbstractModule.java:60)
at org.elasticsearch.common.inject.spi.Elements$RecordingBinder.install(
Elements.java:204)
at org.elasticsearch.common.inject.spi.Elements.getElements(Elements.java
:85)
at org.elasticsearch.common.inject.InjectorShell$Builder.build(
InjectorShell.java:130)
at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.
java:99)
at org.elasticsearch.common.inject.InjectorImpl.createChildInjector(
InjectorImpl.java:131)
at org.elasticsearch.common.inject.ModulesBuilder.createChildInjector(
ModulesBuilder.java:69)
at org.elasticsearch.indices.InternalIndicesService.createIndex(
InternalIndicesService.java:298)
... 7 more
Caused by: org.elasticsearch.common.settings.NoClassSettingsException: Failed
to load class setting [type] with value [kuromoji]
at org.elasticsearch.common.settings.ImmutableSettings.loadClass(
ImmutableSettings.java:448)
at org.elasticsearch.common.settings.ImmutableSettings.getAsClass(
ImmutableSettings.java:436)
at org.elasticsearch.index.analysis.AnalysisModule.configure(
AnalysisModule.java:356)
... 15 more
Caused by: java.lang.ClassNotFoundException:
org.elasticsearch.index.analysis.kuromoji.KuromojiAnalyzerProvider
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.elasticsearch.common.settings.ImmutableSettings.loadClass(
ImmutableSettings.java:446)
... 17 more

Eventually, I found out that the new node was missing the Kuromoji
Analysis plugin
https://github.com/elasticsearch/elasticsearch-analysis-kuromoji.

My question is: why did a missing plugin in one node fail the creation of
any new index? Is that the expected behaviour?

Thanks,
Rafi

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8e7a8f44-50e1-4956-a415-87552441a3f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Hi David,
I see ES as a basic component in a production system.
It should be very gentle with "stop working" scenario.
I would expect the cluster to "heal" its self and install the plugin in the
missing nodes.

It is much better then stopping the production.

On Monday, June 30, 2014 11:59:47 AM UTC+3, David Pilato wrote:

I think it's expected as long as your index could potentialy being
allocated on the node which is missing the plugin.

HTH

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 30 juin 2014 à 10:34, Rafi Aroch <rafi....@gmail.com <javascript:>> a
écrit :

Hi,

I'm using ES 1.0.1 in a 10 node cluster and recently needed to add support
for indexing documents in Japanese. To do that i've installed the Kuromoji
Analysis plugin
https://github.com/elasticsearch/elasticsearch-analysis-kuromoji.
At a later time, when I added a new node to the cluster it joined the
cluster successfully, but I started to see failures when creating new
indexes (any index, in any language) and after the index creation failed
the cluster got into a "red" state where the new indexs' shards remained
Unassigned.
org.elasticsearch.indices.IndexCreationException: [...] failed to create
index
at org.elasticsearch.indices.InternalIndicesService.createIndex(
InternalIndicesService.java:300)
at org.elasticsearch.indices.cluster.IndicesClusterStateService.
applyNewIndices(IndicesClusterStateService.java:306)
at org.elasticsearch.indices.cluster.IndicesClusterStateService.
clusterChanged(IndicesClusterStateService.java:178)
at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.
run(InternalClusterService.java:424)
at org.elasticsearch.common.util.concurrent.
PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(
PrioritizedEsThreadPoolExecutor.java:134)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: failed
to find analyzer type [kuromoji] or tokenizer for [japanese_snowball]
at org.elasticsearch.index.analysis.AnalysisModule.configure(
AnalysisModule.java:372)
at org.elasticsearch.common.inject.AbstractModule.configure(
AbstractModule.java:60)
at org.elasticsearch.common.inject.spi.Elements$RecordingBinder.install(
Elements.java:204)
at org.elasticsearch.common.inject.spi.Elements.getElements(Elements.java
:85)
at org.elasticsearch.common.inject.InjectorShell$Builder.build(
InjectorShell.java:130)
at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.
java:99)
at org.elasticsearch.common.inject.InjectorImpl.createChildInjector(
InjectorImpl.java:131)
at org.elasticsearch.common.inject.ModulesBuilder.createChildInjector(
ModulesBuilder.java:69)
at org.elasticsearch.indices.InternalIndicesService.createIndex(
InternalIndicesService.java:298)
... 7 more
Caused by: org.elasticsearch.common.settings.NoClassSettingsException: Failed
to load class setting [type] with value [kuromoji]
at org.elasticsearch.common.settings.ImmutableSettings.loadClass(
ImmutableSettings.java:448)
at org.elasticsearch.common.settings.ImmutableSettings.getAsClass(
ImmutableSettings.java:436)
at org.elasticsearch.index.analysis.AnalysisModule.configure(
AnalysisModule.java:356)
... 15 more
Caused by: java.lang.ClassNotFoundException:
org.elasticsearch.index.analysis.kuromoji.KuromojiAnalyzerProvider
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.elasticsearch.common.settings.ImmutableSettings.loadClass(
ImmutableSettings.java:446)
... 17 more

Eventually, I found out that the new node was missing the <a href="
GitHub - elastic/elasticsearch-analysis-kuromoji: Japanese (kuromoji) Analysis Plugin" st

...

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c807f5d3-1563-4f70-a426-843791fd4554%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.