Hello,
I'm having problems starting elasticsearch after an error. I was bulk
indexing a new type when the disk got full and server returned this error:
PHP Fatal error: Uncaught exception
'Elastica\Exception\Bulk\ResponseException' with message 'Error in one or
more bulk request actions:
index: /etoxindex/entity2document/74357083 caused
IndexFailedEngineException[[etoxindex][0] Index failed for
[entity2document#74357083]]; nested: IOException[No space left on device];
index: /etoxindex/entity2document/74410388 caused
IndexFailedEngineException[[etoxindex][0] Index failed for
[entity2document#74410388]]; nested: IOException[No space left on device];
' in
/home/acanada/LIMTOX/etoxSymfony/vendor/ruflin/elastica/lib/Elastica/Bulk.php:372
Stack trace:
#0
/home/acanada/LIMTOX/etoxSymfony/vendor/ruflin/elastica/lib/Elastica/Bulk.php(322):
Elastica\Bulk->_processResponse(Object(Elastica\Response))
#1
/home/acanada/LIMTOX/etoxSymfony/vendor/ruflin/elastica/lib/Elastica/Client.php(260):
Elastica\Bulk->send()
#2
/home/acanada/LIMTOX/etoxSymfony/vendor/ruflin/elastica/lib/Elastica/Index.php(123):
Elastica\Client->addDocuments(Array)
#3 /home/acanada/LIMTOX/etoxSymfony/vendor/ruflin/e in
/home/acanada/LIMTOX/etoxSymfony/vendor/ruflin/elastica/lib/Elastica/Bulk.php
on line 372
After that I stopped the server and copied the elasticsearch files to a
different location. After changing config file and restarting the server
(no matter if I restart with the old or the new) I get this error in log:
[2014-03-04 12:44:43,687][INFO ][node ] [Father Time]
stopping ...
[2014-03-04 12:44:43,806][INFO ][node ] [Father Time]
stopped
[2014-03-04 12:44:43,806][INFO ][node ] [Father Time]
closing ...
[2014-03-04 12:44:43,814][INFO ][node ] [Father Time]
closed
[2014-03-04 12:44:45,489][INFO ][node ] [Fan Boy]
version[0.90.7], pid[24106], build[36897d0/2013-11-13T12:06:54Z]
[2014-03-04 12:44:45,490][INFO ][node ] [Fan Boy]
initializing ...
[2014-03-04 12:44:45,495][INFO ][plugins ] [Fan Boy]
loaded [], sites []
[2014-03-04 12:44:47,411][INFO ][node ] [Fan Boy]
initialized
[2014-03-04 12:44:47,411][INFO ][node ] [Fan Boy]
starting ...
[2014-03-04 12:44:47,568][INFO ][transport ] [Fan Boy]
bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address
{inet[/10.222.112.32:9300]}
[2014-03-04 12:44:50,609][INFO ][cluster.service ] [Fan Boy]
new_master [Fan Boy][gxV9farHQsa4u6e0Akox3g][inet[/10.222.112.32:9300]],
reason: zen-disco-join (elected_as_master)
[2014-03-04 12:44:50,646][INFO ][discovery ] [Fan Boy]
elasticsearch/gxV9farHQsa4u6e0Akox3g
[2014-03-04 12:44:50,697][INFO ][http ] [Fan Boy]
bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address
{inet[/10.222.112.32:9200]}
[2014-03-04 12:44:50,698][INFO ][node ] [Fan Boy]
started
[2014-03-04 12:44:51,300][INFO ][gateway ] [Fan Boy]
recovered [2] indices into cluster_state
[2014-03-04 12:44:54,037][INFO ][index.gateway.local ] [Fan Boy]
[etoxindex][0] ignoring recovery of a corrupt translog entry
org.elasticsearch.index.mapper.MapperParsingException: failed to parse
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:538)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:457)
at
org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:363)
at
org.elasticsearch.index.shard.service.InternalIndexShard.performRecoveryOperation(InternalIndexShard.java:709)
at
org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:224)
at
org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:174)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: org.elasticsearch.common.jackson.core.JsonParseException:
Illegal character ((CTRL-CHAR, code 0)): only regular white space (\r, \n,
\t) is allowed between tokens
at [Source: [B@27f9542c; line: 1, column: 310]
at
org.elasticsearch.common.jackson.core.JsonParser._constructError(JsonParser.java:1369)
at
org.elasticsearch.common.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:532)
at
org.elasticsearch.common.jackson.core.base.ParserMinimalBase._throwInvalidSpace(ParserMinimalBase.java:478)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._skipWS(UTF8StreamJsonParser.java:2485)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:671)
at
org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:50)
at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:473)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:513)
... 8 more
[2014-03-04 12:44:54,040][WARN ][indices.cluster ] [Fan Boy]
[etoxindex][0] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[etoxindex][0] failed to recover shard
at
org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:238)
at
org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:174)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.IllegalArgumentException: No type mapped for [58]
at
org.elasticsearch.index.translog.Translog$Operation$Type.fromId(Translog.java:216)
at
org.elasticsearch.index.translog.TranslogStreams.readTranslogOperation(TranslogStreams.java:34)
at
org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:215)
... 4 more
[2014-03-04 12:44:54,059][WARN ][cluster.action.shard ] [Fan Boy]
[etoxindex][0] sending failed shard for [etoxindex][0],
node[gxV9farHQsa4u6e0Akox3g], [P], s[INITIALIZING], indexUUID
[iruNHoZ-T6i4kfDLov-DVw], reason [Failed to start shard, message
[IndexShardGatewayRecoveryException[[etoxindex][0] failed to recover
shard]; nested: IllegalArgumentException[No type mapped for [58]]; ]]
[2014-03-04 12:44:54,059][WARN ][cluster.action.shard ] [Fan Boy]
[etoxindex][0] received shard failed for [etoxindex][0],
node[gxV9farHQsa4u6e0Akox3g], [P], s[INITIALIZING], indexUUID
[iruNHoZ-T6i4kfDLov-DVw], reason [Failed to start shard, message
[IndexShardGatewayRecoveryException[[etoxindex][0] failed to recover
shard]; nested: IllegalArgumentException[No type mapped for [58]]; ]]
[2014-03-04 12:45:02,614][INFO ][index.gateway.local ] [Fan Boy]
[etoxindex][0] ignoring recovery of a corrupt translog entry
org.elasticsearch.index.mapper.MapperParsingException: failed to parse
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:538)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:457)
at
org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:363)
at
org.elasticsearch.index.shard.service.InternalIndexShard.performRecoveryOperation(InternalIndexShard.java:709)
at
org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:224)
at
org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:174)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: org.elasticsearch.common.jackson.core.JsonParseException:
Illegal character ((CTRL-CHAR, code 0)): only regular white space (\r, \n,
\t) is allowed between tokens
at [Source: [B@4c64f1a9; line: 1, column: 310]
at
org.elasticsearch.common.jackson.core.JsonParser._constructError(JsonParser.java:1369)
at
org.elasticsearch.common.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:532)
at
org.elasticsearch.common.jackson.core.base.ParserMinimalBase._throwInvalidSpace(ParserMinimalBase.java:478)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._skipWS(UTF8StreamJsonParser.java:2485)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:671)
at
org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:50)
at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:473)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:513)
... 8 more
[2014-03-04 12:45:02,617][WARN ][indices.cluster ] [Fan Boy]
[etoxindex][0] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[etoxindex][0] failed to recover shard
at
org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:238)
at
org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:174)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.IllegalArgumentException: No type mapped for [58]
at
org.elasticsearch.index.translog.Translog$Operation$Type.fromId(Translog.java:216)
at
org.elasticsearch.index.translog.TranslogStreams.readTranslogOperation(TranslogStreams.java:34)
at
org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:215)
... 4 more
[2014-03-04 12:45:02,633][WARN ][cluster.action.shard ] [Fan Boy]
[etoxindex][0] sending failed shard for [etoxindex][0],
node[gxV9farHQsa4u6e0Akox3g], [P], s[INITIALIZING], indexUUID
[iruNHoZ-T6i4kfDLov-DVw], reason [Failed to start shard, message
[IndexShardGatewayRecoveryException[[etoxindex][0] failed to recover
shard]; nested: IllegalArgumentException[No type mapped for [58]]; ]]
[2014-03-04 12:45:02,634][WARN ][cluster.action.shard ] [Fan Boy]
[etoxindex][0] received shard failed for [etoxindex][0],
node[gxV9farHQsa4u6e0Akox3g], [P], s[INITIALIZING], indexUUID
[iruNHoZ-T6i4kfDLov-DVw], reason [Failed to start shard, message
[IndexShardGatewayRecoveryException[[etoxindex][0] failed to recover
shard]; nested: IllegalArgumentException[No type mapped for [58]]; ]]
I have already deleted the type that I was indexing when the disk got
full... Any help with this problem?
Thank you very much!!
Andrés
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6154e2cf-3bf5-45e8-890d-5d09b537bbb2%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.