Curator Connection aborted

Elastic Stack Elasticsearch
1

Hi,
I am trying to take snapshots for indices using curator-4.2 but while executing any action, it is always throwing error 'Connection aborted.'. I have almost 850 indices and i believe while initializing meta data, curator is getting timedout.
Where should i make changes to avoid this error, any elasticsearch configurations i should be considering? because i tried with maximum timeout values 86400 also in curator config.yml

PFB more details:

  1. Elasticsearch-2.4.4 with Shiled
  2. Curator-4.2.6
  3. Curator actions tried - close,snapshot
  4. We have total 858 indices (258 active rest are closed)
  5. Same error for cli too:
    curator_cli --config config.yml show_indices
  6. No Info/Error on elasticsearch logs

DEBUG Log:

2017-04-05 11:58:58,350 DEBUG curator.indexlist __build_index_info:80 Building preliminary index metadata for A-log-2017.03.17
2017-04-05 11:58:58,350 DEBUG curator.indexlist __build_index_info:80 Building preliminary index metadata for B-log-2017.03.22
2017-04-05 11:58:58,350 DEBUG curator.indexlist __build_index_info:80 Building preliminary index metadata for C-log-2017.03.30
2017-04-05 11:58:58,351 DEBUG curator.indexlist _get_metadata:145 Getting index metadata
2017-04-05 11:58:58,351 DEBUG curator.indexlist empty_list_check:182 Checking for empty list
2017-04-05 11:59:00,352 DEBUG curator.indexlist _get_index_stats:113 Getting index stats
2017-04-05 11:59:00,352 DEBUG curator.indexlist empty_list_check:182 Checking for empty list
2017-04-05 11:59:00,353 DEBUG curator.indexlist working_list:193 Generating working list of indices
2017-04-05 11:59:00,353 DEBUG curator.indexlist working_list:193 Generating working list of indices
2017-04-05 11:59:00,369 ERROR curator.cli cli:193 Failed to complete action: close. <class 'elasticsearch.exceptions.ConnectionError'>: ConnectionError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))) caused by: ProtocolError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)))

Thanks & Regards,

2

If you're connecting through Shield, does the user performing the close or snapshot action have privileges to perform all necessary actions? It seems related, somehow. Are there any corresponding log entries on the client you're connecting to?

It's unlikely that it's timing out during metadata collection, as that would return a more distinct error (a 5xx or 4xx error). When I look at the timestamps in the log entries, I see things a few milliseconds apart, which doesn't look like a timeout.

You aren't connecting through a load balancer or proxy of any kind, are you? Or is it just a bare Shield-protected node's IP address in your curator.yml file?

3

Shiled user has all permissions, it's actually es-admin.

Yes, i was trying to connect through DNS -> Load balancer. As you suggested i tried connecting to client host and it worked.

Thanks for quick response.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.