Is that a good practice to query ES during a Restore process?

Elastic Stack Elasticsearch
1

hello,
I'm trying to measure the Restore process while it happens by querying ES (a simple get index request, then take the index size from it) basically I want to know how much data is there in my restoring index every 20 seconds. but during those queries I often (but not always) get a 404 exception from ES.
am I doing something wrong by interfering ES on both threads? is that too much for the ES?
or maybe it's because I'm using the same ES client for both of my tasks? (one that's doing the restore and one that's doing the get request)

2

I'm a little surprised you get intermittent 404s here. Can you share the exact API call you're making and the complete response? Is there a corresponding stack trace in the server logs and, if so, can you share that too?

To measure the progress of recovery, I recommend the indices recovery API or the cat recovery API.

3

yes I get this trace: when trying to ask ES if there's an index exists inside it, i get 404 for trying to request something from my ES like it died for some reason...

19/09/15 16:17:57.194 [main] INFO i.s.c.AbstractJestClient: Setting server pool to a list of 1 servers: [http://docker.local:9201]
19/09/15 16:17:57.194 [main] INFO i.s.c.JestClientFactory: Using multi thread/connection supporting pooling connection manager
19/09/15 16:17:57.241 [main] INFO i.s.c.JestClientFactory: Using default GSON instance
19/09/15 16:17:57.241 [main] INFO i.s.c.JestClientFactory: Node Discovery disabled...
19/09/15 16:17:57.241 [main] INFO i.s.c.JestClientFactory: Idle connection reaping disabled...
FlakyTestExtension: Iteration 3 failed.
org.junit.ComparisonFailure: expected:<[200]> but was:<[404]>
at sun.reflect.GeneratedConstructorAccessor48.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at com.logshero.gaia.demeter.itest.migrator.AccountsMigrationITest.assertEsSettings(AccountsMigrationITest.java:676)

4

That looks like a client-side trace that's saying it got a 404 instead of a 200, but unfortunately it doesn't tell us what the exact request was that provoked this response, nor what the whole response was (there will be more detail in the response body) and doesn't include the server-side stack either.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.