Data query issues with elasticsearch 6.8

[sammi (sunny)]

we have 5 node cassandra and 3 node elasticsearch for indexes in a cluster , For smaller count data retrieval is working with queries but when there are large count query is failing with below error.

method [POST], host [http://es01:9200], URI [/_search/scroll], status line [HTTP/1.1 404 Not Found]
{"error":{"root_cause":[{"type":"search_context_missing_exception","reason":"No search context found for id [5]"},{"type":"search_context_missing_exception","reason":"No search context found for id [7]"},{"type":"search_context_missing_exception","reason":"No search context found for id [8]"},{"type":"search_context_missing_exception","reason":"No search context found for id [6]"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":-1,"index":null,"reason":{"type":"search_context_missing_exception","reason":"No search context found for id [5]"}},{"shard":-1,"index":null,"reason":{"type":"search_context_missing_exception","reason":"No search context found for id [7]"}},{"shard":-1,"index":null,"reason":{"type":"search_context_missing_exception","reason":"No search context found for id [8]"}},{"shard":-1,"index":null,"reason":{"type":"search_context_missing_exception","reason":"No search context found for id [6]"}}],"caused_by":{"type":"search_context_missing_exception","reason":"No search context found for id [6]"}},"status":404}

Same query works fine with single node cassandra and elasticsearch, Are we missing any thing in cluster ?

please suggest.

[sammi (sunny)]

Can anyone give some suggestion on this ? Let me know if you need more information.

[NerdSec (Nachiket)]

Hi Sunny,

Please format your code for better readability.

The errors states that the search context is missing. Are you ensuring that the search context is alive while you query the scroll context? https://www.elastic.co/guide/en/elasticsearch/reference/6.8/search-request-scroll.html

Also, any specific reason for using scroll? Try leveraging search after if you need stateless query execution.

[sammi (sunny)]

Hi,

We are getting this error while querying using janusgraph/gremlin query. we are not using scroll here. The data is being indexed in elasticsearch and cassandra as being backend.

Which doesn't make any sense to us is same query working in single elasticsearch but when we moved to multi/3 node elasticsearch it is failing. Is it due to data not being shared/sharding issue ?

We have 3 elasticsearch nodes as master/data/ingester for all 3 nodes. Do we need to have specific configuration when we move to cluster .

Please suggest.

[NerdSec (Nachiket)]

There is no specific setting one has to perform in order to move to a multi-node cluster. Data is broken into shards depending on your index configuration.

The error that you have posted usually comes when the search context is not alive anymore. Is there an option in Janusgraph to maybe increase the search context time duration while making the query?

Without much information about the actual query being run on ES, it is difficult to give any solutions or pointers.