I have a doubt about how things work in ElasticSearch : I've shut down
a node for which a gateway configuration was set. However, when
shutting down the gateway, I saw the following messages in the log.
For me "warn" means that it is not important, although "outofmemory"
generally means "failure" for me. Is it that ElasticSearch will try
another method (no memory mapped files) for storing the index in that
case, or will I loose part of my index ?
[12:10:29,221][WARN ][index.gateway ] [Tana Nile][news4][3]
Failed to snapshot (scheduled)
org.elasticsearch.index.gateway.IndexShardGatewaySnapshotFailedException:
[news4][3] Failed to perform snapshot (index files)
at
org.elasticsearch.index.gateway.fs.FsIndexShardGateway.snapshot(FsIndexShardGateway.java:
185)
at org.elasticsearch.index.gateway.IndexShardGatewayService
$1.snapshot(IndexShardGatewayService.java:179)
at org.elasticsearch.index.gateway.IndexShardGatewayService
$1.snapshot(IndexShardGatewayService.java:175)
at
org.elasticsearch.index.engine.robin.RobinEngine.snapshot(RobinEngine.java:
364)
at
org.elasticsearch.index.shard.service.InternalIndexShard.snapshot(InternalIndexShard.java:
377)
at
org.elasticsearch.index.gateway.IndexShardGatewayService.snapshot(IndexShardGatewayService.java:
175)
at org.elasticsearch.index.gateway.IndexShardGatewayService
$SnapshotRunnable.run(IndexShardGatewayService.java:257)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
441)
at java.util.concurrent.FutureTask
$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor
$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor
$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
at java.util.concurrent.ScheduledThreadPoolExecutor
$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by:
org.elasticsearch.index.gateway.IndexShardGatewaySnapshotFailedException:
[news4][3] Failed to copy to [/home/cedric/DEV/elasticsearch-0.8.0/
work/elasticsearch/gateway/indices/news4/3/index/_1l.cfs], from dir
[org.elasticsearch.index.store.fs.NioFsStore$CustomNioFSDirectory@/
home/cedric/DEV/elasticsearch-0.8.0/work/elasticsearch/indices/
05229c90-e20a-465e-827d-a5b1f2af05a8/news4/3/index] and file [_1l.cfs]
at org.elasticsearch.index.gateway.fs.FsIndexShardGateway
$1.run(FsIndexShardGateway.java:172)
... 3 more
Caused by: java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:758)
at
sun.nio.ch.FileChannelImpl.transferFromFileChannel(FileChannelImpl.java:
537)
at sun.nio.ch.FileChannelImpl.transferFrom(FileChannelImpl.java:600)
at
org.elasticsearch.util.io.FileSystemUtils.copyFile(FileSystemUtils.java:
102)
at
org.elasticsearch.util.lucene.Directories.copyFromDirectory(Directories.java:
88)
at org.elasticsearch.index.gateway.fs.FsIndexShardGateway
$1.run(FsIndexShardGateway.java:170)
... 3 more
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:755)
... 8 more
In the upcoming 0.9 I removed completely the native transfer from
elasticsearch as its so buggy in Java. In 0.8 you can disable it by setting index.gateway.fs.native_copy to false.
I have a doubt about how things work in Elasticsearch : I've shut down
a node for which a gateway configuration was set. However, when
shutting down the gateway, I saw the following messages in the log.
For me "warn" means that it is not important, although "outofmemory"
generally means "failure" for me. Is it that Elasticsearch will try
another method (no memory mapped files) for storing the index in that
case, or will I loose part of my index ?
[12:10:29,221][WARN ][index.gateway ] [Tana Nile][news4][3]
Failed to snapshot (scheduled)
org.elasticsearch.index.gateway.IndexShardGatewaySnapshotFailedException:
[news4][3] Failed to perform snapshot (index files)
at
org.elasticsearch.index.gateway.fs.FsIndexShardGateway.snapshot(FsIndexShardGateway.java:
185)
at org.elasticsearch.index.gateway.IndexShardGatewayService
$1.snapshot(IndexShardGatewayService.java:179)
at org.elasticsearch.index.gateway.IndexShardGatewayService
$1.snapshot(IndexShardGatewayService.java:175)
at
org.elasticsearch.index.engine.robin.RobinEngine.snapshot(RobinEngine.java:
364)
at
org.elasticsearch.index.shard.service.InternalIndexShard.snapshot(InternalIndexShard.java:
377)
at
org.elasticsearch.index.gateway.IndexShardGatewayService.snapshot(IndexShardGatewayService.java:
175)
at org.elasticsearch.index.gateway.IndexShardGatewayService
$SnapshotRunnable.run(IndexShardGatewayService.java:257)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
441)
at java.util.concurrent.FutureTask
$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor
$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor
$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
at java.util.concurrent.ScheduledThreadPoolExecutor
$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by:
org.elasticsearch.index.gateway.IndexShardGatewaySnapshotFailedException:
[news4][3] Failed to copy to [/home/cedric/DEV/elasticsearch-0.8.0/
work/elasticsearch/gateway/indices/news4/3/index/_1l.cfs], from dir
[org.elasticsearch.index.store.fs.NioFsStore$CustomNioFSDirectory@/
home/cedric/DEV/elasticsearch-0.8.0/work/elasticsearch/indices/
05229c90-e20a-465e-827d-a5b1f2af05a8/news4/3/index] and file [_1l.cfs]
at org.elasticsearch.index.gateway.fs.FsIndexShardGateway
$1.run(FsIndexShardGateway.java:172)
... 3 more
Caused by: java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:758)
at
sun.nio.ch.FileChannelImpl.transferFromFileChannel(FileChannelImpl.java:
537)
at sun.nio.ch.FileChannelImpl.transferFrom(FileChannelImpl.java:600)
at
org.elasticsearch.util.io.FileSystemUtils.copyFile(FileSystemUtils.java:
102)
at
org.elasticsearch.util.lucene.Directories.copyFromDirectory(Directories.java:
88)
at org.elasticsearch.index.gateway.fs.FsIndexShardGateway
$1.run(FsIndexShardGateway.java:170)
... 3 more
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:755)
... 8 more
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.