Hi
We recently had an outage on our openshift clusters.
In our Test environment the pods came back up with warnings.
The log file shows the following
Suppressed: org.apache.lucene.index.CorruptIndexException: checksum status indeterminate: remaining=0, please run checkindex for more details (resource=BufferedChecksumIndexInput(MMapIndexInput
(path="/usr/share/elasticsearch/data/nodes/0/indices/2ubIJUqYTHOFOvKofHDl7Q/3/index/_f7l.cfe")))
however when we go that that directory the file is not present.
ls -ltr /usr/share/elasticsearch/data/nodes/0/indices/2ubIJUqYTHOFOvKofHDl7Q/3/index/_f7l.cfe
ls: cannot access /usr/share/elasticsearch/data/nodes/0/indices/2ubIJUqYTHOFOvKofHDl7Q/3/index/_f7l.cfe: No such file or directory
sh-4.2$ /opt/jdk-15.0.1+9/bin/java -cp /usr/share/elasticsearch/lib/lucene-core-7.7.3.jar -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex /usr/share/elasticsearch/data/nodes/0/indices/2ubIJUqYTHOFOvKofHDl7Q/3/index/_f7l.cfe
Opening index @ /usr/share/elasticsearch/data/nodes/0/indices/2ubIJUqYTHOFOvKofHDl7Q/3/index/_f7l.cfe
Exception in thread "main" org.apache.lucene.index.IndexNotFoundException: no segments* file found in MMapDirectory@/usr/share/elasticsearch/data/nodes/0/indices/2ubIJUqYTHOFOvKofHDl7Q/3/index/_f7l.cfe lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d6f64b1: files: [write.lock]
at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:519)
at org.apache.lucene.index.CheckIndex.doCheck(CheckIndex.java:2809)
at org.apache.lucene.index.CheckIndex.doMain(CheckIndex.java:2711)
at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:2637)
It didn't allow me to paste the whole error due to size limits.
Uploading in parts
[2021-11-09T18:52:51,992][WARN ][o.e.g.G.InternalReplicaShardAllocator] [elasticsearch-master-0] [jaeger-span-2021-11-09][3]: failed to list shard for shard_store on node [ZIxGRmm0Tkmxlmxe3VhtNg]
org.elasticsearch.action.FailedNodeException: Failed node [ZIxGRmm0Tkmxlmxe3VhtNg]
at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction.onFailure(TransportNodesAction.java:236) [elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction.access$200(TransportNodesAction.java:151) [elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction$1.handleException(TransportNodesAction.java:210) [elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1126) [elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.transport.TcpTransport.lambda$handleException$24(TcpTransport.java:1011) [elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:193) [elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.transport.TcpTransport.handleException(TcpTransport.java:1009) [elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.transport.TcpTransport.handlerResponseError(TcpTransport.java:1001) [elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:950) [elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:763) [elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:53) [transport-netty4-client-6.8.15.jar:6.8.15]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) [netty-codec-4.1.32.Final.jar:4.1.32.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297) [netty-codec-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241) [netty-handler-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:556) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:510) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) [netty-common-4.1.32.Final.jar:4.1.32.Final]
at java.lang.Thread.run(Thread.java:832) [?:?]
Caused by: org.elasticsearch.transport.RemoteTransportException: [elasticsearch-master-2][10.129.2.25:9300][internal:cluster/nodes/indices/shard/store[n]]
Caused by: org.elasticsearch.ElasticsearchException: Failed to list store metadata for shard [[jaeger-span-2021-11-09][3]]
at org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:116) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:62) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.action.support.nodes.TransportNodesAction.nodeOperation(TransportNodesAction.java:138) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:259) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:255) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:250) ~[?:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:308) ~[?:?]
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1087) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:778) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.8.15.jar:6.8.15]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) ~[?:?]
at java.lang.Thread.run(Thread.java:832) ~[?:?]
Caused by: org.apache.lucene.index.CorruptIndexException: failed engine (reason: [refresh failed source[schedule]]) (resource=preexisting_corruption)
at org.elasticsearch.index.store.Store.failIfCorrupted(Store.java:599) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.index.store.Store.readMetadataSnapshot(Store.java:442) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.listStoreMetaData(TransportNodesListShardStoreMetaData.java:165) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:114) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.nodeOperation(TransportNodesListShardStoreMetaData.java:62) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.action.support.nodes.TransportNodesAction.nodeOperation(TransportNodesAction.java:138) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:259) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:255) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:250) ~[?:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:308) ~[?:?]
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1087) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:778) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.8.15.jar:6.8.15]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) ~[?:?]
at java.lang.Thread.run(Thread.java:832) ~[?:?]
Caused by: java.io.IOException: failed engine (reason: [refresh failed source[schedule]])
at org.elasticsearch.index.engine.Engine.failEngine(Engine.java:1192) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.index.engine.InternalEngine.refresh(InternalEngine.java:1688) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.index.engine.InternalEngine.refresh(InternalEngine.java:1658) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.index.shard.IndexShard.refresh(IndexShard.java:969) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.index.IndexService.maybeRefreshEngine(IndexService.java:745) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.index.IndexService.access$500(IndexService.java:100) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.index.IndexService$AsyncRefreshTask.runInternal(IndexService.java:871) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.common.util.concurrent.AbstractAsyncTask.run(AbstractAsyncTask.java:144) ~[elasticsearch-6.8.15.jar:6.8.15]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:708) ~[elasticsearch-6.8.15.jar:6.8.15]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) ~[?:?]
at java.lang.Thread.run(Thread.java:832) ~[?:?]
I think this indicates that the files that make up this shard were changed by something other than Elasticsearch. However this is only about a replica, the primary should be unaffected and Elasticsearch will rebuild the replica in due course.
You'll need to work out why the data the Elasticsearch read wasn't the data it originally wrote, however. That will eventually cause you a bigger problem.
No. Nothing but Elasticsearch may change Elasticsearch's data. Quoting the same docs as last time:
WARNING : Don’t modify anything within the data directory or run processes that might interfere with its contents. If something other than Elasticsearch modifies the contents of the data directory, then Elasticsearch may fail, reporting corruption or other data inconsistencies, or may appear to work correctly having silently lost some of your data.
Hi David
The below output shows just one index that is in RED.
I would have to restore just that one correct? If we didn't have a snapshot then we should just delete that index?
sh-4.2$ wget -q 'http://localhost:9200/_cat/indices?v' -O-
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open jaeger-service-2021-09-21 8QzFvB4dTjyt-W9RC2K-zg 5 1 37 1 146kb 93.3kb
yellow open jaeger-service-2021-10-07 SbUPpDfdSwKQlK-9JcVNJw 5 1 78 2 161kb 99kb
green open jaeger-service-2021-11-04 5r1idiUcSfS-20JMppPgCA 5 1 59 2 230.6kb 115.3kb
yellow open jaeger-span-2021-08-30 EA5KbnjQQZCQXqOtPtW5kA 5 1 3419896 0 157mb 157mb
yellow open jaeger-span-2021-09-12 9zdLA7SvTDi7dZaEhVgcKA 5 1 3759025 0 278.6mb 174.3mb
yellow open jaeger-span-2021-09-21 Rq1NSpFXSu6ZlHML7fTIew 5 1 3418116 0 253.9mb 158.5mb
yellow open jaeger-service-2021-10-28 78GGztLBSXC7TB855OD_Tg 5 1 59 5 166.9kb 115.8kb
yellow open jaeger-service-2021-10-09 EuUxVC99TfCLAmnKgqp96Q 5 1 33 1 141.5kb 103kb
yellow open jaeger-service-2021-11-03 tUOy47QAR76-Xn1M8XoLOQ 5 1 70 4 97.3kb 97.3kb
yellow open jaeger-service-2021-10-21 2CxwYPtrQCyrASXh3Gdviw 5 1 63 7 213.6kb 129.8kb
yellow open jaeger-span-2021-10-14 WEH_nxnuRfyYC3SYAV4rIg 5 1 3781483 0 275.8mb 172.7mb
yellow open jaeger-service-2021-10-15 gW8nICJiTei643Ex_EIGOw 5 1 36 0 185.1kb 112.9kb
yellow open jaeger-span-2021-08-29 xDUGl0isRZefXgo0Li7JKQ 5 1 3419828 0 158.6mb 158.6mb
yellow open jaeger-service-2021-10-12 OTfD1p1tQ2ae3g9xkoEkCw 5 1 55 5 195.5kb 121.9kb
yellow open jaeger-service-2021-10-24 agIYV6FMRqyF7pFg9u-fQQ 5 1 32 0 167.4kb 109kb
yellow open jaeger-span-2021-10-05 4UkwucmgTkyZDV3kZolo7g 5 1 2940953 0 221.5mb 138.2mb
yellow open jaeger-service-2021-10-27 aIlNXXfnSP6_c8Wn5aW4Cw 5 1 57 7 215kb 141.3kb
yellow open jaeger-span-2021-10-23 m5-G-pNIR_6BADGKCWA22Q 5 1 3518032 0 260.2mb 162.2mb
yellow open jaeger-service-2021-09-27 QouFJ99VSEKzoy11uuGpNQ 5 1 36 0 196.3kb 118.3kb
yellow open jaeger-span-2021-09-13 uN7Er1svQNCIBr1gNWABwA 5 1 3758504 0 274.4mb 171.3mb
yellow open jaeger-service-2021-10-01 yEPyNcmHSrmmMRyekjv1QA 5 1 36 3 190.9kb 115.6kb
yellow open jaeger-service-2021-09-14 3jnX2_iiTOS9zcKBlkpnHg 5 1 56 9 157.9kb 106.2kb
yellow open jaeger-service-2021-11-01 bsJxj5CjTEefE_fFr48yTA 5 1 64 8 157.4kb 100.2kb
yellow open jaeger-service-2021-10-04 oYxeWz-CRFiFWqATcNPwPg 5 1 35 0 186.8kb 111.9kb
yellow open jaeger-span-2021-08-31 5WyEGqnNSY-_wmvmq8WF1g 5 1 3420536 0 158.4mb 158.4mb
yellow open jaeger-span-2021-09-27 Uhnc9w39S9OhAsUZJQpNow 5 1 2827559 0 214.6mb 133.9mb
yellow open jaeger-service-2021-09-24 x_ZBPUBsSg-_Ez48p_5YbQ 5 1 38 1 209.7kb 125kb
yellow open jaeger-service-2021-10-19 -N0a1scwQWWjh4MfR1E60g 5 1 32 0 174.3kb 109kb
yellow open jaeger-service-1970-01-01 3PzIv0_qSRmq_AxAWP_odA 5 1 53 15 103.4kb 103.4kb
green open jaeger-span-2021-11-10 QPGkCp9fQIqwb9m_MOwE9g 5 1 2593750 0 236.8mb 118.8mb
yellow open jaeger-span-2021-10-22 XjtSvOUmSj2taq_P2D0aFQ 5 1 3509767 0 258.9mb 161.9mb
yellow open jaeger-service-2021-09-17 fe4l-1JXTAOQXjSSb-JYZQ 5 1 63 7 190kb 122.6kb
yellow open jaeger-span-2021-09-17 NHt56k4bSAmW_Za_vcNCwQ 5 1 3418743 0 252.9mb 157.9mb
yellow open jaeger-span-2021-09-16 keT3dK5OQq6Ktvmj-oovMA 5 1 3586867 0 230.9mb 164.8mb
yellow open jaeger-span-2021-09-11 frzB6LLmT927OeaSNDlaSg 5 1 3759080 0 278.8mb 173.9mb
yellow open jaeger-service-2021-09-03 otsc3K16R1yLJK05aOvtAQ 5 1 32 0 79.7kb 79.7kb
yellow open jaeger-span-2021-09-14 THoQzjk5SqijHScHgEKzxw 5 1 3758788 0 277.1mb 172.9mb
yellow open jaeger-span-2021-09-19 tQzwpwAhRSqSRtvTWdvn7w 5 1 3417726 0 251.5mb 157mb
yellow open jaeger-service-2021-10-10 AbSKRL0WRmenRv2ciyTUcA 5 1 33 0 178.1kb 109.4kb
yellow open jaeger-span-2021-10-31 pJrE8S1BQFqB20F-vX3FJA 5 1 3815970 0 245.8mb 175.7mb
green open jaeger-span-2021-11-07 CYldGGsQTVivA0gg-rHm3A 5 1 4044528 0 365.9mb 182.9mb
yellow open jaeger-service-2021-08-29 HNdvDMSLSVet68QWd9yZcA 5 1 31 0 72.9kb 72.9kb
yellow open jaeger-span-2021-09-30 BOlg5ccxTzO9CRaz8-JbCQ 5 1 2827235 0 211.5mb 132.4mb
yellow open jaeger-span-2021-09-18 yCe24PoDRAyWg9vzUOk2Nw 5 1 3418253 0 253mb 157.3mb
yellow open jaeger-service-2021-09-26 1GeM_NjlQ4qIYcV31BeAEw 5 1 29 0 150.4kb 95.3kb
yellow open jaeger-span-2021-10-11 xoQOiHoARVerBPmRX32o8w 5 1 3823711 0 277mb 173.4mb
yellow open jaeger-span-2021-10-19 jhkj-ZtkR66z9mY2hV7oGw 5 1 3520879 0 260.9mb 163.1mb
green open jaeger-span-2021-11-04 Rj_zeHOeTHSSA08a_dfUow 5 1 4033151 0 366.1mb 182.7mb
yellow open jaeger-span-2021-11-02 z7iTvsQUQTKchtI3ibrrcQ 5 1 3606838 0 168mb 168mb
yellow open jaeger-service-2021-10-18 wwknDVwaSYyfyTx0NjLKdw 5 1 33 5 162.3kb 103kb
green open jaeger-service-2021-11-05 RaGDypprReuTGkz7VG2ABw 5 1 59 3 224.7kb 109.2kb
yellow open jaeger-service-2021-11-02 LdiC782OQPKuHVXBlIbSLA 5 1 83 20 164.4kb 104.9kb
yellow open jaeger-service-2021-08-31 gDS338OtTjyUFDabSL5hAA 5 1 55 0 123.6kb 123.6kb
yellow open jaeger-service-2021-09-02 ANqXf9kGQrCHFWJamWPzpw 5 1 32 0 79.6kb 79.6kb
yellow open jaeger-service-2021-10-08 KgFcWn_3RnmyN48pS_k1MQ 5 1 79 11 196.3kb 135.9kb
yellow open jaeger-span-2021-09-10 C8ZyIW0sSsiHP8cKjJQp_w 5 1 3759812 0 277.4mb 173.8mb
yellow open jaeger-span-2021-08-27 l_xzfGleSP68vz0QVDuakQ 5 1 2021778 0 94.4mb 94.4mb
yellow open jaeger-service-2021-08-25 juNBB3mlSbe4Q2xcpXD6TQ 5 1 48 1 71.6kb 71.6kb
yellow open jaeger-service-2021-08-27 1RbQII1JQsyvzEVvDrfBsw 5 1 34 4 76.8kb 76.8kb
yellow open jaeger-span-2021-09-04 qDMyR9WkQ5G8P38x9cQ4pg 5 1 3758510 0 173mb 173mb
yellow open jaeger-span-2021-09-06 6Y4zVAoPTU-wpwQaWAnrFg 5 1 3602934 0 167.4mb 167.4mb
yellow open jaeger-span-1970-01-01 6LY4lKgZT1iJAbFA16hdhA 5 1 85404 0 18.9mb 18.9mb
yellow open jaeger-span-2021-09-15 PVtBBX6vQZiddTJjPpipqA 5 1 3754867 0 278.1mb 173.8mb
green open jaeger-span-2021-11-05 AE2C4A79SdyABC83fH0Awg 5 1 4045842 0 367.2mb 183.6mb
yellow open jaeger-service-2021-09-01 QUltamiLRBaxHDG1UXg2uw 5 1 31 0 76.1kb 76.1kb
green open jaeger-service-2021-11-09 vEYHh3waToqhTZ9bIGLseQ 5 1 78 4 209.9kb 108.1kb
yellow open jaeger-span-2021-10-01 UhrUD8mDTI-wetf6WTGj6A 5 1 2783940 0 209.6mb 131.2mb
yellow open jaeger-span-2021-11-01 M8THTlS8Tki_GNTl-eYWkw 5 1 3562843 0 263.7mb 164.6mb
yellow open jaeger-service-2021-09-22 JBpswJb0R2228El-fepXhg 5 1 44 0 162.1kb 103.1kb
yellow open jaeger-span-2021-10-07 zuxa9kEiS8CU2xf3uvRmEg 5 1 3527708 0 267mb 166.4mb
green open jaeger-span-2021-11-08 itNmxk1ORtqohk-0J0NHog 5 1 4045719 0 368.3mb 184.4mb
yellow open jaeger-span-2021-10-04 hJfykD4SSLSI6zvWLSVCsQ 5 1 2828668 0 215.4mb 134.9mb
yellow open jaeger-service-2021-10-14 0Dg97eKCTHGBpPH73Cblmw 5 1 36 2 172.6kb 106.7kb
yellow open jaeger-span-2021-10-26 g-vukOV0QeKVhQ1lW4mxiw 5 1 3658642 0 268.9mb 167.7mb
yellow open jaeger-span-2021-09-29 jo2IEqbJRAGllFfsD6BW2Q 5 1 2826751 0 213.4mb 132.9mb
yellow open jaeger-span-2021-09-24 jwC-lsikRwCe2bN7dySlZA 5 1 2824679 0 212mb 132.6mb
yellow open jaeger-span-2021-10-21 4EbKLYRRSvCam9UMCUN2Kw 5 1 3472462 0 251.8mb 157.9mb
yellow open jaeger-span-2021-10-20 UvZgQWoiSMu62tMuPKMF3g 5 1 3527617 0 260.6mb 163.1mb
yellow open jaeger-service-2021-10-25 4A_ZYcrnQUWqT1D1RGI_GQ 5 1 40 2 146.4kb 100.2kb
yellow open jaeger-service-2021-10-17 cknO_22PTuqPu-_CihnPgw 5 1 32 2 152.3kb 96.5kb
yellow open jaeger-service-2021-09-09 nC-4em23QQWU_skJd5Il3A 5 1 59 5 190.4kb 135.1kb
green open jaeger-service-2021-11-10 5yXuVW_xTgCbteii4Gnvyw 5 1 58 0 272.3kb 134.5kb
yellow open jaeger-service-2021-09-29 x7Tqpt7VTfCBrNaullzaJQ 5 1 37 0 203kb 121.6kb
yellow open jaeger-span-2021-10-16 8hpG9QW3QGW_bopwjtbgSw 5 1 3520005 0 257.2mb 160.9mb
yellow open jaeger-service-2021-10-16 jAlaLt11Rbijuc-M0hc1RA 5 1 32 1 149kb 96.4kb
yellow open jaeger-service-2021-10-26 HFa8dFhiRyu70yDbw2CaVg 5 1 36 4 126.9kb 90.5kb
yellow open jaeger-span-2021-09-25 8davYgblThW145lcNQ11-g 5 1 2825632 0 213.6mb 133.5mb
yellow open jaeger-service-2021-10-31 bQ8RwRMqSY6ACuki-ed1yg 5 1 33 1 155.3kb 103kb
yellow open jaeger-span-2021-09-23 itIiT8GZTUSttz3jr9l-3A 5 1 3056171 0 229.4mb 143.5mb
yellow open jaeger-service-2021-09-20 Nk7ztkUxTm-_CeD6LSyoQA 5 1 32 0 157.5kb 99.1kb
yellow open jaeger-service-2021-08-30 G7BW345eT2CqYcypxxxL7A 5 1 32 0 82.7kb 82.7kb
yellow open jaeger-span-2021-09-03 oRp6kd2OQ5G-5q6m840K-A 5 1 3754278 0 172.3mb 172.3mb
green open jaeger-span-2021-11-06 45IsjZsuS7WY7n-_eg__lg 5 1 4037166 0 366mb 183.1mb
yellow open jaeger-span-2021-08-26 7G79rjUoTE62ww6CHFb5Bg 5 1 3388231 0 153.5mb 153.5mb
yellow open jaeger-service-2021-10-13 vO5clTJhT-6ITMsKk2Iy4g 5 1 37 7 163.8kb 104kb
yellow open jaeger-service-2021-09-10 F9hE3ym2So2tbJhxU4Ka7Q 5 1 57 1 200.4kb 124.5kb
yellow open jaeger-span-2021-10-10 xx9zAY96TVSn5j_GJSuTxw 5 1 3859468 0 285.1mb 177.5mb
yellow open jaeger-service-2021-09-13 BzlAJDBpTqqvngsfElXEhA 5 1 40 3 175.6kb 103.7kb
green open jaeger-service-2021-11-06 Jq1xiYkZRVGzgB09DDXTsA 5 1 34 0 206.1kb 103kb
yellow open jaeger-span-2021-10-25 dG4iPa9wRce92n_QzGRsNg 5 1 3521222 0 261.2mb 162.7mb
red open jaeger-span-2021-11-03 4gd7PS_MQOGv_Y1YUWhWEg 5 1 858966 0 40.3mb 40.3mb
yellow open jaeger-span-2021-10-03 rF8S1oarSmix_9PBQzOELQ 5 1 2828845 0 213.9mb 133.1mb
yellow open jaeger-span-2021-09-08 t6MCQuUwSOK5lkBvpoBwJw 5 1 3412825 0 157.4mb 157.4mb
yellow open jaeger-service-2021-10-06 8UXYbMU7R9ipn0fKqqnd-Q 5 1 79 6 131.6kb 89.3kb
sh-4.2$
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.