Question concerning Analyzer

Hi,

we try to do some large text mining with ElasticSearch, but we ran most the
times into some errors. Maybe this is due to some errors in our setup with
the analyzer in the Config file. That is why we posted our config file (in
JSON format) which is send to our cluster on http://pastebin.com/WdtP16pm.
Perhaps somebody (who has some time :slight_smile: ) can look over this file, if it is
in principal correct.

A lot of thanks in advance.

Cheers,

Niko

What type of errors are you hitting?

On Monday, July 4, 2011 at 12:17 PM, Niko Gross wrote:

Hi,

we try to do some large text mining with Elasticsearch, but we ran most the times into some errors. Maybe this is due to some errors in our setup with the analyzer in the Config file. That is why we posted our config file (in JSON format) which is send to our cluster on "index":{ "number_of_shards": 100, "number_of_replicas": 2, "anal - Pastebin.com. Perhaps somebody (who has some time :slight_smile: ) can look over this file, if it is in principal correct.

A lot of thanks in advance.

Cheers,

Niko

One of the error is this one:

org.elasticsearch.common.util.concurrent.UncategorizedExecutionException:
Failed execution
at org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:49)
at com.excerbt.mapreduce.ner.es.ESNER.ner(ESNER.java:64)
at com.excerbt.mapreduce.ner.es.NamedEntityRecognizer$NERMapper.map(NamedEntityRecognizer.java:135)
at com.excerbt.mapreduce.ner.es.NamedEntityRecognizer$NERMapper.map(NamedEntityRecognizer.java:111)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:646)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.util.concurrent.ExecutionException:
java.util.concurrent.RejectedExecutionException
at org.elasticsearch.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:263)
at org.elasticsearch.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:250)
at org.elasticsearch.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:78)
at org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:42)
... 11 more
Caused by: java.util.concurrent.RejectedExecutionException
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1768)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:195)
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:166)
at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteScan(SearchServiceTransportAction.java:350)
at org.elasticsearch.action.search.type.TransportSearchScanAction$AsyncAction.sendExecuteFirstPhase(TransportSearchScanAction.java:71)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:192)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:134)
at org.elasticsearch.action.search.type.TransportSearchScanAction.doExecute(TransportSearchScanAction.java:55)
at org.elasticsearch.action.search.type.TransportSearchScanAction.doExecute(TransportSearchScanAction.java:47)
at org.elasticsearch.action.support.BaseAction.execute(BaseAction.java:61)
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:110)
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:42)
at org.elasticsearch.action.support.BaseAction.execute(BaseAction.java:61)
at org.elasticsearch.client.node.NodeClient.search(NodeClient.java:168)
at org.elasticsearch.client.action.search.SearchRequestBuilder.doExecute(SearchRequestBuilder.java:583)
at org.elasticsearch.client.action.support.BaseRequestBuilder.execute(BaseRequestBuilder.java:56)
at org.elasticsearch.client.action.support.BaseRequestBuilder.execute(BaseRequestBuilder.java:51)
... 11 more

2011/7/4 Shay Banon shay.banon@elasticsearch.com

What type of errors are you hitting?

On Monday, July 4, 2011 at 12:17 PM, Niko Gross wrote:

Hi,

we try to do some large text mining with Elasticsearch, but we ran most the
times into some errors. Maybe this is due to some errors in our setup with
the analyzer in the Config file. That is why we posted our config file (in
JSON format) which is send to our cluster on "index":{ "number_of_shards": 100, "number_of_replicas": 2, "anal - Pastebin.com.
Perhaps somebody (who has some time :slight_smile: ) can look over this file, if it is
in principal correct.

A lot of thanks in advance.

Cheers,

Niko

--

Gruß

Niko Gross

Are you closing the nodes when you do it?

On Tuesday, July 5, 2011 at 10:59 AM, Niko Gross wrote:

One of the error is this one:

org.elasticsearch.common.util.concurrent.UncategorizedExecutionException: Failed execution at org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:49) at com.excerbt.mapreduce.ner.es.ESNER.ner(ESNER.java:64) at com.excerbt.mapreduce.ner.es.NamedEntityRecognizer$NERMapper.map(NamedEntityRecognizer.java:135) at com.excerbt.mapreduce.ner.es.NamedEntityRecognizer$NERMapper.map(NamedEntityRecognizer.java:111) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:646) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115) at org.apache.hadoop.mapred.Child.main(Child.java:262) Caused by: java.util.concurrent.ExecutionException: java.util.concurrent.RejectedExecutionException at org.elasticsearch.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:263) at org.elasticsearch.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:250) at org.elasticsearch.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:78) at org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:42) ... 11 more Caused by: java.util.concurrent.RejectedExecutionException at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1768) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:195) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:166) at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteScan(SearchServiceTransportAction.java:350) at org.elasticsearch.action.search.type.TransportSearchScanAction$AsyncAction.sendExecuteFirstPhase(TransportSearchScanAction.java:71) at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:192) at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:134) at org.elasticsearch.action.search.type.TransportSearchScanAction.doExecute(TransportSearchScanAction.java:55) at org.elasticsearch.action.search.type.TransportSearchScanAction.doExecute(TransportSearchScanAction.java:47) at org.elasticsearch.action.support.BaseAction.execute(BaseAction.java:61) at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:110) at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:42) at org.elasticsearch.action.support.BaseAction.execute(BaseAction.java:61) at org.elasticsearch.client.node.NodeClient.search(NodeClient.java:168) at org.elasticsearch.client.action.search.SearchRequestBuilder.doExecute(SearchRequestBuilder.java:583) at org.elasticsearch.client.action.support.BaseRequestBuilder.execute(BaseRequestBuilder.java:56) at org.elasticsearch.client.action.support.BaseRequestBuilder.execute(BaseRequestBuilder.java:51) ... 11 more
2011/7/4 Shay Banon <shay.banon@elasticsearch.com (mailto:shay.banon@elasticsearch.com)>

What type of errors are you hitting?

On Monday, July 4, 2011 at 12:17 PM, Niko Gross wrote:

Hi,

we try to do some large text mining with Elasticsearch, but we ran most the times into some errors. Maybe this is due to some errors in our setup with the analyzer in the Config file. That is why we posted our config file (in JSON format) which is send to our cluster on "index":{ "number_of_shards": 100, "number_of_replicas": 2, "anal - Pastebin.com. Perhaps somebody (who has some time :slight_smile: ) can look over this file, if it is in principal correct.

A lot of thanks in advance.

Cheers,

Niko

--

Gruß

Niko Gross

I ran into the same stacktrace and the reason was that I was closing the node before using the client.