Dutch hunspell doesn't work in 1.7.1?

Elastic Stack Elasticsearch
1

When I follow the standard procedure of adding a hunspell dictionary for the one for Dutch, I get the following error whenever I start Elasticsearch (for e.g. the en_US dictionary works fine...):

[2015-09-01 15:58:04,959][ERROR][indices.analysis ] [Richard Rider] Could not load hunspell dictionary [nl_NL]
java.lang.IllegalArgumentException: expected only one flag, got: Kc
at org.apache.lucene.analysis.hunspell.Dictionary$FlagParsingStrategy.parseFlag(Dictionary.java:1055)
at org.apache.lucene.analysis.hunspell.Dictionary.readAffixFile(Dictionary.java:338)
at org.apache.lucene.analysis.hunspell.Dictionary.(Dictionary.java:222)
at org.elasticsearch.indices.analysis.HunspellService.loadDictionary(HunspellService.java:185)
at org.elasticsearch.indices.analysis.HunspellService.access$000(HunspellService.java:66)
at org.elasticsearch.indices.analysis.HunspellService$1.load(HunspellService.java:94)
at org.elasticsearch.indices.analysis.HunspellService$1.load(HunspellService.java:89)
at org.elasticsearch.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
at org.elasticsearch.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
at org.elasticsearch.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
at org.elasticsearch.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
at org.elasticsearch.common.cache.LocalCache.get(LocalCache.java:3937)
at org.elasticsearch.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)
at org.elasticsearch.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)
at org.elasticsearch.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4830)
at org.elasticsearch.indices.analysis.HunspellService.scanAndLoadDictionaries(HunspellService.java:130)
at org.elasticsearch.indices.analysis.HunspellService.(HunspellService.java:100)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:54)
at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:86)
at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:98)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:45)
at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:837)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:42)
at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:57)
at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:45)
at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:200)
at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:193)
at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:830)
at org.elasticsearch.common.inject.InjectorBuilder.loadEagerSingletons(InjectorBuilder.java:193)
at org.elasticsearch.common.inject.InjectorBuilder.injectDynamically(InjectorBuilder.java:175)
at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:110)
at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:93)
at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:70)
at org.elasticsearch.common.inject.ModulesBuilder.createInjector(ModulesBuilder.java:59)
at org.elasticsearch.node.internal.InternalNode.(InternalNode.java:210)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:159)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:77)
at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:245)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:32)
[2015-09-01 15:58:04,962][ERROR][indices.analysis ] [Richard Rider] exception while loading dictionary nl_NL

I hope this is the right forum to post this.

Regards,
Jaap

2

Hi, I think you need to find a better/newer copy of the dutch dictionary. I know there was an option for leniency here before, but I removed it: because it made it impossible to improve the hunspell support in lucene when it just silently did the wrong thing.

There are some dictionaries out there with bugs like this, especially old ones, hell I even found one with an executable file embedded inside of it!

My advice is to find the most recent version of the dictionary, usually that works best.

3

Hi rmuir,

thanks for the tip! I'll try that, but I think it is weird that in former version it did work without touching any leniency option. Anyway good you found out that these dictionaries can be buggy. When I find a good one for Dutch I'll post a link here.

J.

4

The problem with the Dutch dictionary is that the use of a two-character flag value:

KEEPCASE Kc

comes before the long flag is enabled in this line:

FLAG long

Solution

Move the FLAG long line up in the nl_NL.aff file, so that it comes before the KEEPCASE Kc line. Then it works.