Problems with memory store


(James Cook) #1

All of my integration tests run fine when I use the niofs storage, but when
I switch to memory storage I see the following exception. THe problem occurs
when I perform a lookup on an id value that does not exist in my index.

org.elasticsearch.transport.RemoteTransportException:
[Hardshell][inet[/10.16.151.129:9310]][indices/get/shard]
Caused by: java.lang.NullPointerException
at
o.e.index.store.memory.ByteBufferFile.numberOfBuffers(ByteBufferFile.java:62)

at
o.e.index.store.memory.ByteBufferIndexInput.switchCurrentBuffer(ByteBufferIndexInput.java:92)

at
o.e.index.store.memory.ByteBufferIndexInput.seek(ByteBufferIndexInput.java:82)

at org.apache.lucene.index.SegmentTermEnum.seek(SegmentTermEnum.java:114)
at
org.apache.lucene.index.TermInfosReader.seekEnum(TermInfosReader.java:172)
at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:231)
at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:179)
at org.apache.lucene.index.SegmentTermDocs.seek(SegmentTermDocs.java:57)
at
org.apache.lucene.index.DirectoryReader$MultiTermDocs.termDocs(DirectoryReader.java:1216)

at
org.apache.lucene.index.DirectoryReader$MultiTermDocs.next(DirectoryReader.java:1148)

at org.elasticseaatrch.common.lucene.Lucene.docId(Lucene.java:65)
at
o.e.action.get.TransportGetAction.shardOperation(TransportGetAction.java:86)

at
o.e.action.get.TransportGetAction.shardOperation(TransportGetAction.java:53)

at
o.e.action.support.single.TransportSingleOperationAction$ShardTransportHandler.messageReceived(TransportSingleOperationAction.java:268)

at
o.e.action.support.single.TransportSingleOperationAction$ShardTransportHandler.messageReceived(TransportSingleOperationAction.java:261)

at
o.e.transport.netty.MessageChannelHandler$3.run(MessageChannelHandler.java:195)

at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:637)

Here is my ES configuration:

path.home = ${es.path.home}
cluster.name = ${cluster.name}
http.enabled = true
http.port = 9311
network.host = non_loopback
node.data = true
node.local = false
node.client = false
transport.tcp.port = 9310
index.store.type = memory
index.store.fs.memory.direct = false
index.analysis.analyzer.lowercase_keyword.type=custom
index.analysis.analyzer.lowercase_keyword.tokenizer=keyword
index.analysis.analyzer.lowercase_keyword.filter.0=lowercase

And the relevant portion of the mapping file:

{
"profiles" : {
"properties" : {
"dataType":{"type": "string", "index": "not_analyzed"},
"username":{"type": "string", "index": "analyzed",
"analyzer":"lowercase_keyword"},
"instantMessenger":{"type":"object",
"properties" : {
"provider":{"type": "string", "index":
"not_analyzed"},
"screenName":{"type": "string", "index":
"not_analyzed"}
}
}, ...


(Shay Banon) #2

Hi,

Someone also reported this on github issues, I need to track it down. If
you have time, it would help to have a simple recreation...

-shay.banon

On Tue, Nov 16, 2010 at 11:56 PM, James Cook jcook@tracermedia.com wrote:

All of my integration tests run fine when I use the niofs storage, but when
I switch to memory storage I see the following exception. THe problem occurs
when I perform a lookup on an id value that does not exist in my index.

org.elasticsearch.transport.RemoteTransportException:
[Hardshell][inet[/10.16.151.129:9310]][indices/get/shard]
Caused by: java.lang.NullPointerException
at
o.e.index.store.memory.ByteBufferFile.numberOfBuffers(ByteBufferFile.java:62)

at
o.e.index.store.memory.ByteBufferIndexInput.switchCurrentBuffer(ByteBufferIndexInput.java:92)

at
o.e.index.store.memory.ByteBufferIndexInput.seek(ByteBufferIndexInput.java:82)

at org.apache.lucene.index.SegmentTermEnum.seek(SegmentTermEnum.java:114)
at
org.apache.lucene.index.TermInfosReader.seekEnum(TermInfosReader.java:172)
at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:231)
at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:179)
at org.apache.lucene.index.SegmentTermDocs.seek(SegmentTermDocs.java:57)
at
org.apache.lucene.index.DirectoryReader$MultiTermDocs.termDocs(DirectoryReader.java:1216)

at
org.apache.lucene.index.DirectoryReader$MultiTermDocs.next(DirectoryReader.java:1148)

at org.elasticseaatrch.common.lucene.Lucene.docId(Lucene.java:65)
at
o.e.action.get.TransportGetAction.shardOperation(TransportGetAction.java:86)

at
o.e.action.get.TransportGetAction.shardOperation(TransportGetAction.java:53)

at
o.e.action.support.single.TransportSingleOperationAction$ShardTransportHandler.messageReceived(TransportSingleOperationAction.java:268)

at
o.e.action.support.single.TransportSingleOperationAction$ShardTransportHandler.messageReceived(TransportSingleOperationAction.java:261)

at
o.e.transport.netty.MessageChannelHandler$3.run(MessageChannelHandler.java:195)

at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:637)

Here is my ES configuration:

path.home = ${es.path.home}
cluster.name = ${cluster.name}
http.enabled = true
http.port = 9311
network.host = non_loopback
node.data = true
node.local = false
node.client = false
transport.tcp.port = 9310
index.store.type = memory
index.store.fs.memory.direct = false
index.analysis.analyzer.lowercase_keyword.type=custom
index.analysis.analyzer.lowercase_keyword.tokenizer=keyword
index.analysis.analyzer.lowercase_keyword.filter.0=lowercase

And the relevant portion of the mapping file:

{
"profiles" : {
"properties" : {
"dataType":{"type": "string", "index": "not_analyzed"},
"username":{"type": "string", "index": "analyzed",
"analyzer":"lowercase_keyword"},
"instantMessenger":{"type":"object",
"properties" : {
"provider":{"type": "string", "index":
"not_analyzed"},
"screenName":{"type": "string", "index":
"not_analyzed"}
}
}, ...


(system) #3