RemoteTransportException

Elastic Stack Elasticsearch
1

Hi,

I am importing data from my MySQL database to ES cluster and sometimes have
the following error while updating (already imported) document (Url -
called URL, Request: post data, Error: response error, I removed all
private data from the request):

Url: <url_to_document>/_update
Request: {"script":"ctx._source.chat.duration = duration_value;
ctx._source.chat.messages +=
message_table","params":{"duration_value":19,"message_table":[]}}
Error: RemoteTransportException[[db-es3-sl][inet[/10.61.20.54:9300]][update]];
nested: DocumentMissingException[[users][1] [chat][1057502_MDUPNUG8Y1]:
document missing];
JSON response:
{"error":"RemoteTransportException[[db-es3-sl][inet[/10.61.20.54:9300]][update]];
nested: DocumentMissingException[[users][1] [chat][<doc_id>]: document
missing]; ","status":404}

I use bulk indexing with 10K documents for importing, after this I resume
real time stream to the index that can update already imported documents.

What does this error mean and how to prevent it ?

Thank you.

Best regards.

--

2

How are you importing your data? Have you changed the refresh interval, or
forcing a refresh before doing the updates?

On Monday, December 3, 2012 11:51:41 PM UTC-8, Marcin Dojwa wrote:

Hi,

I am importing data from my MySQL database to ES cluster and sometimes
have the following error while updating (already imported) document (Url -
called URL, Request: post data, Error: response error, I removed all
private data from the request):

Url: <url_to_document>/_update
Request: {"script":"ctx._source.chat.duration = duration_value;
ctx._source.chat.messages +=
message_table","params":{"duration_value":19,"message_table":[]}}
Error:
RemoteTransportException[[db-es3-sl][inet[/10.61.20.54:9300]][update]];
nested: DocumentMissingException[[users][1] [chat][1057502_MDUPNUG8Y1]:
document missing];
JSON response:
{"error":"RemoteTransportException[[db-es3-sl][inet[/10.61.20.54:9300]][update]];
nested: DocumentMissingException[[users][1] [chat][<doc_id>]: document
missing]; ","status":404}

I use bulk indexing with 10K documents for importing, after this I resume
real time stream to the index that can update already imported documents.

What does this error mean and how to prevent it ?

Thank you.

Best regards.

--

3

I did not change the refresh interval and I am not forcing a refresh before
_update. As I can see eg. here

just makes the changes visible for search, but operations using
document id for updating should be able to find the document before the
refresh is done too. Am I wrong ?

2012/12/4 Dan Lecocq dan@seomoz.org

How are you importing your data? Have you changed the refresh interval, or
forcing a refresh before doing the updates?

On Monday, December 3, 2012 11:51:41 PM UTC-8, Marcin Dojwa wrote:

Hi,

I am importing data from my MySQL database to ES cluster and sometimes
have the following error while updating (already imported) document (Url -
called URL, Request: post data, Error: response error, I removed all
private data from the request):

Url: <url_to_document>/_update
Request: {"script":"ctx.source.chat.duration = duration_value;
ctx._source.chat.messages += message_table","params":{"
duration_value":19,"message**table":[]}}
Error: RemoteTransportException[[db-es3-sl][inet[/10.61.20.54:9300]][update]];
nested: DocumentMissingException[[**users][1]

[chat][1057502_MDUPNUG8Y1]: document missing];
JSON response: {"error":"**RemoteTransportException[[db-es3-sl][inet[/
10.61.20.54:9300]][update]]; nested: DocumentMissingException[[**users][1]

[chat][<doc_id>]: document missing]; ","status":404}

I use bulk indexing with 10K documents for importing, after this I resume
real time stream to the index that can update already imported documents.

What does this error mean and how to prevent it ?

Thank you.

Best regards.

--

--

4

I would have expected it to make the document available (for both search
and update), but I could be wrong.

How are you importing your data? Any chance that you're doing bulk inserts
and then not finishing the last bulk request? What client are you using?

On Tuesday, December 4, 2012 1:39:36 PM UTC-8, Marcin Dojwa wrote:

I did not change the refresh interval and I am not forcing a refresh
before _update. As I can see eg. here
Elasticsearch Platform — Find real-time answers at scale | Elastic just makes the changes visible for search, but operations using
document id for updating should be able to find the document before the
refresh is done too. Am I wrong ?

2012/12/4 Dan Lecocq <d...@seomoz.org <javascript:>>

How are you importing your data? Have you changed the refresh interval,
or forcing a refresh before doing the updates?

On Monday, December 3, 2012 11:51:41 PM UTC-8, Marcin Dojwa wrote:

Hi,

I am importing data from my MySQL database to ES cluster and sometimes
have the following error while updating (already imported) document (Url -
called URL, Request: post data, Error: response error, I removed all
private data from the request):

Url: <url_to_document>/_update
Request: {"script":"ctx.source.chat.duration = duration_value;
ctx._source.chat.messages += message_table","params":{"
duration_value":19,"message**table":[]}}
Error: RemoteTransportException[[db-es3-sl][inet[/10.61.20.54:9300]][update]];
nested: DocumentMissingException[[**users][1]

[chat][1057502_MDUPNUG8Y1]: document missing];
JSON response: {"error":"**RemoteTransportException[[db-es3-sl][inet[/
10.61.20.54:9300]][update]]; nested: DocumentMissingException[[**users][1]

[chat][<doc_id>]: document missing]; ","status":404}

I use bulk indexing with 10K documents for importing, after this I
resume real time stream to the index that can update already imported
documents.

What does this error mean and how to prevent it ?

Thank you.

Best regards.

--

--

5

Hi,

I do the bulk inserts using curl from command line. I wait untill this curl
requests finishes and then I resume indexing from my service. I do the bulk
like this:
curl -s -XPOST localhost:9200/_bulk --data-binary @<bulk_data_file> >
/dev/null
when the command above finishes I resume real time indexing and then
sometimes (very rare) I get this error.

Best regards.

2012/12/4 Dan Lecocq dan@seomoz.org

I would have expected it to make the document available (for both search
and update), but I could be wrong.

How are you importing your data? Any chance that you're doing bulk inserts
and then not finishing the last bulk request? What client are you using?

On Tuesday, December 4, 2012 1:39:36 PM UTC-8, Marcin Dojwa wrote:

I did not change the refresh interval and I am not forcing a refresh
before _update. As I can see eg. here http://www.elasticsearch.**
org/guide/reference/api/admin-**indices-refresh.htmlhttp://www.elasticsearch.org/guide/reference/api/admin-indices-refresh.htmlrefreshing just makes the changes visible for search, but operations using
document id for updating should be able to find the document before the
refresh is done too. Am I wrong ?

2012/12/4 Dan Lecocq d...@seomoz.org

How are you importing your data? Have you changed the refresh interval,

or forcing a refresh before doing the updates?

On Monday, December 3, 2012 11:51:41 PM UTC-8, Marcin Dojwa wrote:

Hi,

I am importing data from my MySQL database to ES cluster and sometimes
have the following error while updating (already imported) document (Url -
called URL, Request: post data, Error: response error, I removed all
private data from the request):

Url: <url_to_document>/_update
Request: {"script":"ctx._source.chat.duration = duration_value;
ctx.source.chat.messages += message_table","params":{"dura
tion_value":19,"messagetable":[]}}
Error: RemoteTransportException[[db-es3-sl][inet[/10.61.20.54:9300*
*]**][update]]; nested: DocumentMissingException[[users][1]

[chat][1057502_MDUPNUG8Y1]: document missing];
JSON response: {"error":"RemoteTransportException[[db-**
es3-sl][inet[/10.61.20.54:9300 http://10.61.20.54:9300]][update]];
nested: DocumentMissingException[[users][1] [chat][<doc_id>]:
document missing]; ","status":404}

I use bulk indexing with 10K documents for importing, after this I
resume real time stream to the index that can update already imported
documents.

What does this error mean and how to prevent it ?

Thank you.

Best regards.

--

--

--

6

OK, it's not rare, if that happens for a specified alias (single client)
after importing its data to ES (using bulk) there are about 10-20 such
errors for this single alias within about 30 minutes. So it's NOT only
right after the bulk indexing is done.

2012/12/6 Marcin Dojwa m.dojwa@livechatinc.com

Hi,

I do the bulk inserts using curl from command line. I wait untill this
curl requests finishes and then I resume indexing from my service. I do the
bulk like this:
curl -s -XPOST localhost:9200/_bulk --data-binary @<bulk_data_file> >
/dev/null
when the command above finishes I resume real time indexing and then
sometimes (very rare) I get this error.

Best regards.

2012/12/4 Dan Lecocq dan@seomoz.org

I would have expected it to make the document available (for both search
and update), but I could be wrong.

How are you importing your data? Any chance that you're doing bulk
inserts and then not finishing the last bulk request? What client are you
using?

On Tuesday, December 4, 2012 1:39:36 PM UTC-8, Marcin Dojwa wrote:

I did not change the refresh interval and I am not forcing a refresh
before _update. As I can see eg. here http://www.elasticsearch.**
org/guide/reference/api/admin-**indices-refresh.htmlhttp://www.elasticsearch.org/guide/reference/api/admin-indices-refresh.htmlrefreshing just makes the changes visible for search, but operations using
document id for updating should be able to find the document before the
refresh is done too. Am I wrong ?

2012/12/4 Dan Lecocq d...@seomoz.org

How are you importing your data? Have you changed the refresh interval,

or forcing a refresh before doing the updates?

On Monday, December 3, 2012 11:51:41 PM UTC-8, Marcin Dojwa wrote:

Hi,

I am importing data from my MySQL database to ES cluster and sometimes
have the following error while updating (already imported) document (Url -
called URL, Request: post data, Error: response error, I removed all
private data from the request):

Url: <url_to_document>/_update
Request: {"script":"ctx._source.chat.duration = duration_value;
ctx.source.chat.messages += message_table","params":{"dura
tion_value":19,"messagetable":[]}}
Error: RemoteTransportException[[db-es3-sl][inet[/10.61.20.54:9300
]][update]]; nested: DocumentMissingException[[users][1]

[chat][1057502_MDUPNUG8Y1]: document missing];
JSON response: {"error":"RemoteTransportException[[db-**
es3-sl][inet[/10.61.20.54:9300 http://10.61.20.54:9300]][update]];
nested: DocumentMissingException[[users][1] [chat][<doc_id>]:
document missing]; ","status":404}

I use bulk indexing with 10K documents for importing, after this I
resume real time stream to the index that can update already imported
documents.

What does this error mean and how to prevent it ?

Thank you.

Best regards.

--

--

--

7

Blerg :-/ That's odd. I'd expect the implicit refresh to be called at that
point. Could you post a gist of the response you're getting from the bulk
request?

On Thursday, December 6, 2012 3:08:17 AM UTC-8, Marcin Dojwa wrote:

OK, it's not rare, if that happens for a specified alias (single client)
after importing its data to ES (using bulk) there are about 10-20 such
errors for this single alias within about 30 minutes. So it's NOT only
right after the bulk indexing is done.

2012/12/6 Marcin Dojwa <m.d...@livechatinc.com <javascript:>>

Hi,

I do the bulk inserts using curl from command line. I wait untill this
curl requests finishes and then I resume indexing from my service. I do the
bulk like this:
curl -s -XPOST localhost:9200/_bulk --data-binary @<bulk_data_file> >
/dev/null
when the command above finishes I resume real time indexing and then
sometimes (very rare) I get this error.

Best regards.

2012/12/4 Dan Lecocq <d...@seomoz.org <javascript:>>

I would have expected it to make the document available (for both search
and update), but I could be wrong.

How are you importing your data? Any chance that you're doing bulk
inserts and then not finishing the last bulk request? What client are you
using?

On Tuesday, December 4, 2012 1:39:36 PM UTC-8, Marcin Dojwa wrote:

I did not change the refresh interval and I am not forcing a refresh
before _update. As I can see eg. here http://www.elasticsearch.**
org/guide/reference/api/admin-**indices-refresh.htmlhttp://www.elasticsearch.org/guide/reference/api/admin-indices-refresh.htmlrefreshing just makes the changes visible for search, but operations using
document id for updating should be able to find the document before the
refresh is done too. Am I wrong ?

2012/12/4 Dan Lecocq d...@seomoz.org

How are you importing your data? Have you changed the refresh interval,

or forcing a refresh before doing the updates?

On Monday, December 3, 2012 11:51:41 PM UTC-8, Marcin Dojwa wrote:

Hi,

I am importing data from my MySQL database to ES cluster and
sometimes have the following error while updating (already imported)
document (Url - called URL, Request: post data, Error: response error, I
removed all private data from the request):

Url: <url_to_document>/_update
Request: {"script":"ctx._source.chat.duration = duration_value;
ctx.source.chat.messages += message_table","params":{"dura
tion_value":19,"messagetable":[]}}
Error: RemoteTransportException[[db-es3-sl][inet[/
10.61.20.54:9300**]][update]]; nested: DocumentMissingException[[
users][1] [chat][1057502_MDUPNUG8Y1]: document missing];
JSON response: {"error":"RemoteTransportException[[db-
es3-sl][inet[/10.61.20.54:9300 http://10.61.20.54:9300]][update]];
nested: DocumentMissingException[[users][1] [chat][<doc_id>]:
document missing]; ","status":404}

I use bulk indexing with 10K documents for importing, after this I
resume real time stream to the index that can update already imported
documents.

What does this error mean and how to prevent it ?

Thank you.

Best regards.

--

--

--

8

Unfortunatelly I do not have the responses from bulk indexing (I do not log
them). But this can be helpful :slight_smile: I will do the response logging to be able
to check the logs for the aliases that fails.

2012/12/6 Dan Lecocq dan@seomoz.org

Blerg :-/ That's odd. I'd expect the implicit refresh to be called at that
point. Could you post a gist of the response you're getting from the bulk
request?

On Thursday, December 6, 2012 3:08:17 AM UTC-8, Marcin Dojwa wrote:

OK, it's not rare, if that happens for a specified alias (single client)
after importing its data to ES (using bulk) there are about 10-20 such
errors for this single alias within about 30 minutes. So it's NOT only
right after the bulk indexing is done.

2012/12/6 Marcin Dojwa m.d...@livechatinc.com

Hi,

I do the bulk inserts using curl from command line. I wait untill this
curl requests finishes and then I resume indexing from my service. I do the
bulk like this:
curl -s -XPOST localhost:9200/_bulk --data-binary @<bulk_data_file> >
/dev/null
when the command above finishes I resume real time indexing and then
sometimes (very rare) I get this error.

Best regards.

2012/12/4 Dan Lecocq d...@seomoz.org

I would have expected it to make the document available (for both search

and update), but I could be wrong.

How are you importing your data? Any chance that you're doing bulk
inserts and then not finishing the last bulk request? What client are you
using?

On Tuesday, December 4, 2012 1:39:36 PM UTC-8, Marcin Dojwa wrote:

I did not change the refresh interval and I am not forcing a refresh
before _update. As I can see eg. here http://www.elasticsearch.****
org/guide/reference/api/admin-****indices-refresh.htmlhttp://www.elasticsearch.org/guide/reference/api/admin-indices-refresh.htmlrefreshing just makes the changes visible for search, but operations using
document id for updating should be able to find the document before the
refresh is done too. Am I wrong ?

2012/12/4 Dan Lecocq d...@seomoz.org

How are you importing your data? Have you changed the refresh

interval, or forcing a refresh before doing the updates?

On Monday, December 3, 2012 11:51:41 PM UTC-8, Marcin Dojwa wrote:

Hi,

I am importing data from my MySQL database to ES cluster and
sometimes have the following error while updating (already imported)
document (Url - called URL, Request: post data, Error: response error, I
removed all private data from the request):

Url: <url_to_document>/_update
Request: {"script":"ctx.source.chat.**duration =
duration_value; ctx._source.chat.messages += message_table","params":{"
**duration_value":19,"messagetable":[]}}
Error: RemoteTransportException[[db-**es3-sl][inet[/
10.61.20.54:9300]][update]]; nested:
DocumentMissingException[[**users][1]

[chat][1057502_MDUPNUG8Y1]: document missing];
JSON response: {"error":"RemoteTransportExcep****tion[[db-
es3-sl][inet[/10.61.20.54:9300 http://10.61.20.54:9300**]][update]];
nested: DocumentMissingException[[**user****s][1] [chat][<doc_id>]:
document missing]; ","status":404}

I use bulk indexing with 10K documents for importing, after this I
resume real time stream to the index that can update already imported
documents.

What does this error mean and how to prevent it ?

Thank you.

Best regards.

--

--

--

--