После перезапуска Elasticsearch выдает ошибки

Bob04 · May 19, 2017, 4:42pm

После первого запуска ES в Docker контейнере, ES работает без ошибок:

docker run -d --name elasticsearch -p 9200:9200 -e "bootstrap.system_call_filter=false" \
    -e "xpack.security.enabled=false" -e "xpack.graph.enabled=false" -e "xpack.watcher.enabled=false" \
    -v /opt/elasticsearch/data/:/usr/share/elasticsearch/data/ \
    docker.elastic.co/elasticsearch/elasticsearch:5.3.0

Но после перезапуска ES Docker container, ES выдает ошибки failed to collect data, IOException , NoSuchFileException

Связано ли это с тем, что я монтирую директорию /usr/share/elasticsearch/data/ ?

[2017-05-19T17:17:39,280][ERROR][o.e.x.m.c.i.IndicesStatsCollector] [hY2F_6j] collector [indices-stats-collector] failed to collect data
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
        at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:165) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.action.admin.indices.stats.TransportIndicesStatsAction.checkGlobalBlock(TransportIndicesStatsAction.java:70) ~[elasticsearch-5.3.0.jar:5.3.0]

[2017-05-19T17:17:39,583][ERROR][o.e.x.m.c.i.IndexRecoveryCollector] [hY2F_6j] collector [index-recovery-collector] failed to collect data
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
        at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:165) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.action.admin.indices.recovery.TransportRecoveryAction.checkGlobalBlock(TransportRecoveryAction.java:114) ~[elasticsearch-5.3.0.jar:5.3.0]
		
[2017-05-19T17:17:44,382][WARN ][r.suppressed             ] path: /_xpack/security/_authenticate, params: {index=_xpack, id=_authenticate, type=security}
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
        at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:165) ~[elasticsearch-5.3.0.jar:5.3.0]

[2017-05-19T17:18:05,185][WARN ][o.e.g.GatewayAllocator$InternalPrimaryShardAllocator] [hY2F_6j] [packetbeat-2017.05.19][1]: failed to list shard for shard_started on node [hY2F_6j5SvCEfy_mDR-vhg]
org.elasticsearch.action.FailedNodeException: Failed node [hY2F_6j5SvCEfy_mDR-vhg]
        at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction.onFailure(TransportNodesAction.java:246) [elasticsearch-5.3.0.jar:5.3.0]

Caused by: org.elasticsearch.transport.RemoteTransportException: [hY2F_6j][172.17.0.64:9300][internal:gateway/local/started_shards[n]]
Caused by: org.elasticsearch.ElasticsearchException: failed to load started shards
        at org.elasticsearch.gateway.TransportNodesListGatewayStartedShards.nodeOperation(TransportNodesListGatewayStartedShards.java:171) ~[elasticsearch-5.3.0.jar:5.3.0]

Caused by: org.elasticsearch.ElasticsearchException: java.io.IOException: failed to read [id:11, legacy:false, file:/usr/share/elasticsearch/data/nodes/0/indices/Gj09XHKKRFm8-8NXGmUUJg/_state/state-11.st]
        at org.elasticsearch.ExceptionsHelper.maybeThrowRuntimeAndSuppress(ExceptionsHelper.java:150) ~[elasticsearch-5.3.0.jar:5.3.0]

Caused by: java.io.IOException: failed to read [id:11, legacy:false, file:/usr/share/elasticsearch/data/nodes/0/indices/Gj09XHKKRFm8-8NXGmUUJg/_state/state-11.st]
        at org.elasticsearch.gateway.MetaDataStateFormat.loadLatestState(MetaDataStateFormat.java:327) ~[elasticsearch-5.3.0.jar:5.3.0]

Caused by: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/data/nodes/0/indices/Gj09XHKKRFm8-8NXGmUUJg/_state/state-11.st
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) ~[?:?]

[2017-05-19T17:19:19,726][WARN ][o.e.x.m.MonitoringService] [hY2F_6j] monitoring execution failed
org.elasticsearch.xpack.monitoring.exporter.ExportException: Exception when closing export bulk
        at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$1$1.<init>(ExportBulk.java:106) ~[?:?]

Bob04 · May 19, 2017, 5:45pm

Удалось исправить, изменив права доступа к /opt/elasticsearch/data/ на 777

Igor_Motov · May 19, 2017, 5:56pm

Скорее всего, uid пользователя, под которым elasticsearch запущен во второй раз, не совпадает с uid пользователя, под которым elasticsearch был запущен в первый раз. Поэтому он прочитать свои данные не может.

Bob04 · May 22, 2017, 5:37pm

Снова появляются ошибки при перезапуске ES.
Есть ли способ указать в настройках uid пользователя, под которым Elasticsearch был запущен в первый раз?

[2017-05-22T17:22:50,855][INFO ][o.e.n.Node               ] [hY2F_6j] starting ...
[2017-05-22T17:22:54,334][WARN ][i.n.u.i.MacAddressUtil   ] Failed to find a usable hardware address from the network interfaces; using random bytes: 47:17:2f:ea:27:b8:47:93
[2017-05-22T17:22:57,564][INFO ][o.e.t.TransportService   ] [hY2F_6j] publish_address {...:9300}, bound_addresses {[::]:9300}
[2017-05-22T17:22:58,495][INFO ][o.e.b.BootstrapChecks    ] [hY2F_6j] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-05-22T17:23:04,235][INFO ][o.e.c.s.ClusterService   ] [hY2F_6j] new_master {hY2F_6j}{hY2F_6j5SvCEfy_mDR-vhg}{oUxuE8VtTe2V_TPFgxSxsA}{...}{...:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2017-05-22T17:23:09,877][WARN ][o.e.m.j.JvmGcMonitorService] [hY2F_6j] [gc][young][12][10] duration [4.3s], collections [1]/[4.4s], total[4.3s]/[23.5s], memory [78.8mb]->[65.1mb]/[6.9gb], all_pools {[young] [49.5mb]->[16.5mb]/[66.5mb]}{[survivor] [6.7mb]->[8mb]/[8.3mb]}{[old]
[2017-05-22T17:23:10,622][INFO ][o.e.h.n.Netty4HttpServerTransport] [hY2F_6j] publish_address {...:9200}, bound_addresses {[::]:9200
[2017-05-22T17:23:10,649][INFO ][o.e.n.Node               ] [hY2F_6j] started
[2017-05-22T17:23:12,663][ERROR][o.e.x.m.c.i.IndexStatsCollector] [hY2F_6j] collector [index-stats-collector] failed to collect data
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
        at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:165) ~[elasticsearch-5.3.0.jar:5.3.0]

[2017-05-22T17:26:22,795][WARN ][o.e.x.m.MonitoringService] [hY2F_6j] monitoring execution failed
org.elasticsearch.xpack.monitoring.exporter.ExportException: Exception when closing export bulk
        at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$1$1.<init>(ExportBulk.java:106) ~[?:?]
        at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$1.onFailure(ExportBulk.java:104) ~[?:?]
        at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound$1.onResponse(ExportBulk.java:217) ~[?:?]

Caused by: org.elasticsearch.xpack.monitoring.exporter.ExportException: failed to flush export bulks
        at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound.lambda$null$0(ExportBulk.java:167) ~[?:?]
        ... 25 more
Caused by: org.elasticsearch.xpack.monitoring.exporter.ExportException: bulk [default_local] reports failures when exporting documents
        at org.elasticsearch.xpack.monitoring.exporter.local.LocalBulk.throwExportException(LocalBulk.java:124) ~[?:?]
        ... 23 more

Igor_Motov · May 22, 2017, 6:56pm

Я в докере не силен, но по-моему, это достаточно распространенная проблема. Посмотрите тут - https://denibertovic.com/posts/handling-permissions-with-docker-volumes/, например.

system · June 19, 2017, 6:56pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.