Good timeout value for index creation


(ssr) #1

We have 20 node ES 90.2 cluster with about 8mil documents. We use pyes for
talking to elasticsearch. We are using 4 primary 1 replica shard setting
for each index creation.

We have a timeout value of 2 (obviously low) seconds every time we try to
create an index we are getting an IndexAlreadyExistException even though
the index doesn't exist. Looks like it is thrown ES itself with the
following exception
string: '{"error":"RemoteTransportException[[Tempest][inet[/10.3.190.87:9300]][indices/create]];
nested: IndexAlreadyExistsException[[088c058862a240a0b58a883caf0ef69a]
Already exists]; ","status":400}'

And when we set the timeout to something larger like 5 seconds we don't see
this exception.

Any reason why ES throws an cryptic IndexAlreadyExists exception for
transport timeouts?

And what is a reasonable timeout to set for Elastic search operations?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b04a1c3f-264d-4f9b-a996-8a92f8b5eb4e%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Alexander Reelsen) #2

Hey,

can you tell, about which timeout you are speaking? Which one did you
configure? Where did you configure it?

--Alex

On Thu, Mar 6, 2014 at 1:11 PM, ssr sr.sakhamuri@gmail.com wrote:

We have 20 node ES 90.2 cluster with about 8mil documents. We use pyes for
talking to elasticsearch. We are using 4 primary 1 replica shard setting
for each index creation.

We have a timeout value of 2 (obviously low) seconds every time we try to
create an index we are getting an IndexAlreadyExistException even though
the index doesn't exist. Looks like it is thrown ES itself with the
following exception
string: '{"error":"RemoteTransportException[[Tempest][inet[/10.3.190.87:9300]][indices/create]];
nested: IndexAlreadyExistsException[[088c058862a240a0b58a883caf0ef69a]
Already exists]; ","status":400}'

And when we set the timeout to something larger like 5 seconds we don't
see this exception.

Any reason why ES throws an cryptic IndexAlreadyExists exception for
transport timeouts?

And what is a reasonable timeout to set for Elastic search operations?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b04a1c3f-264d-4f9b-a996-8a92f8b5eb4e%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/b04a1c3f-264d-4f9b-a996-8a92f8b5eb4e%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM_THdBkA9zP4zFbq%3DG84W8x%3D6ZohDrJhckgA9sn1iXpKA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(ssr) #3

Hi

I am talking about the Connection timeout to the elasticsearch. We have 20
node ES 90.2 cluster with about 8mil documents

We have observed sometimes the index creation takes several seconds,
understandably so since there is considerable IO involved in creating the
index.

What connection time out value we can reasonably use before timing out a
index creation call. We currently have 8 seconds, and still see timeouts in
the logs but not at all a worrying rate.
Should we continue to use the same or increase it? If so to what value?

Log statment: error creating index
7365dd5fb037287f4099cb9190fe524c07f0186f79b5a64649ddac7ff070d503:
HTTPConnectionPool(host='10.3.45.173', port=9200): Request timed out. (
timeout=8.0)

Srs
On Friday, March 7, 2014 12:39:07 AM UTC-7, Alexander Reelsen wrote:

Hey,

can you tell, about which timeout you are speaking? Which one did you
configure? Where did you configure it?

--Alex

On Thu, Mar 6, 2014 at 1:11 PM, ssr <sr.sak...@gmail.com <javascript:>>wrote:

We have 20 node ES 90.2 cluster with about 8mil documents. We use pyes
for talking to elasticsearch. We are using 4 primary 1 replica shard
setting for each index creation.

We have a timeout value of 2 (obviously low) seconds every time we try to
create an index we are getting an IndexAlreadyExistException even though
the index doesn't exist. Looks like it is thrown ES itself with the
following exception
string: '{"error":"RemoteTransportException[[Tempest][inet[/10.3.190.87:9300]][indices/create]];
nested: IndexAlreadyExistsException[[088c058862a240a0b58a883caf0ef69a]
Already exists]; ","status":400}'

And when we set the timeout to something larger like 5 seconds we don't
see this exception.

Any reason why ES throws an cryptic IndexAlreadyExists exception for
transport timeouts?

And what is a reasonable timeout to set for Elastic search operations?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b04a1c3f-264d-4f9b-a996-8a92f8b5eb4e%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/b04a1c3f-264d-4f9b-a996-8a92f8b5eb4e%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3ee3f561-7a7c-45c7-85ce-84a96751e3be%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #4