100% processor usage

Marcin_Dojwa · October 3, 2012, 8:30pm

But executing a single delete by query I use 1 alias per query.

W dniu środa, 3 października 2012 użytkownik Marcin Dojwa napisał:

More, I have 1 index and about 50000 aliases to this index.

W dniu środa, 3 października 2012 użytkownik Martijn v Groningen napisał:

Hi Marcin,

Are you using one or more aliases when executing the delete by query?

Martijn

On 3 October 2012 10:50, Marcin Dojwa m.dojwa@livechatinc.com wrote:

Hi,

The exact mapping is here: ES 100% processor usage · GitHub
I guess that this is about chat document type because this is the only
one with nested mapping. I can not give you exact data that could possibly
cause such a situation because this is our clients' private data. Are you
able to try reproduce the problem with this information about mapping only?
If not I will try to prepare fake example data.

Best regards.

2012/10/2 Shay Banon kimchy@gmail.com

It seems like its doing the delete by query with nested documents. Can
you share your mapping, and sample data that you index, and the
delete_by_query that you are using?

On Oct 2, 2012, at 4:16 AM, Marcin Dojwa m.dojwa@livechatinc.com
wrote:

Hi,

I attached 4 different (I think) snapshots of curl
localhost:9200/_nodes/hot_threads here:
ES 100% processor usage · GitHub

Best regards.
Marcin Dojwa

2012/9/29 Marcin Dojwa m.dojwa@livechatinc.com

Sure, I will try to reproduce this case on Monday. I updated ES to
0.19.9 so I will check if this happens with this version too.

Best regards.

2012/9/28 Shay Banon kimchy@gmail.com

Can you issue: curl localhost:9200/_nodes/hot_threads and gist the
response?

On Sep 27, 2012, at 1:19 PM, Marcin Dojwa m.dojwa@livechatinc.com
wrote:

One one thing, I am not sure if it hangs on bulk indexing or on 'delete
by query' after bulk indexing.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

Fix for 1.: each bulk has 10000 inserts

Best regards.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

Oh, and there is nothing in gc log.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

I am using ES 0.19.8, I want to upgrade to 0.19.9 now but I am not sure
if that will help.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

Hi,

I still need help in this I have the following information:

2 nodes (es1 and es2)

1 index Users with 30 shards and 1 replica

I do the following:

On es2 I run consecutively many bulk index operations (each bulk has
1000 inserts) and I do about 1000 such bulk inserts.

At some point of inserting es1 uses about 300% of processor and bulk

--

--
Met vriendelijke groet,

Martijn van Groningen

--

--

mvg · October 3, 2012, 9:45pm

I found out that the delete by query basically hangs when executed
with an alias that has a filter associated with it.
If you run the delete by query on the concrete index, it should work.
I'll open an issue for this.

Martijn

On 3 October 2012 22:30, Marcin Dojwa m.dojwa@livechatinc.com wrote:

But executing a single delete by query I use 1 alias per query.

W dniu środa, 3 października 2012 użytkownik Marcin Dojwa napisał:

More, I have 1 index and about 50000 aliases to this index.

W dniu środa, 3 października 2012 użytkownik Martijn v Groningen napisał:

Hi Marcin,

Are you using one or more aliases when executing the delete by query?

Martijn

On 3 October 2012 10:50, Marcin Dojwa m.dojwa@livechatinc.com wrote:

Hi,

The exact mapping is here: ES 100% processor usage · GitHub
I guess that this is about chat document type because this is the only
one with nested mapping. I can not give you exact data that could possibly
cause such a situation because this is our clients' private data. Are you
able to try reproduce the problem with this information about mapping only?
If not I will try to prepare fake example data.

Best regards.

2012/10/2 Shay Banon kimchy@gmail.com

It seems like its doing the delete by query with nested documents. Can
you share your mapping, and sample data that you index, and the
delete_by_query that you are using?

On Oct 2, 2012, at 4:16 AM, Marcin Dojwa m.dojwa@livechatinc.com
wrote:

Hi,

I attached 4 different (I think) snapshots of curl
localhost:9200/_nodes/hot_threads here:
ES 100% processor usage · GitHub

Best regards.
Marcin Dojwa

2012/9/29 Marcin Dojwa m.dojwa@livechatinc.com

Sure, I will try to reproduce this case on Monday. I updated ES to
0.19.9 so I will check if this happens with this version too.

Best regards.

2012/9/28 Shay Banon kimchy@gmail.com

Can you issue: curl localhost:9200/_nodes/hot_threads and gist the
response?

On Sep 27, 2012, at 1:19 PM, Marcin Dojwa m.dojwa@livechatinc.com
wrote:

One one thing, I am not sure if it hangs on bulk indexing or on 'delete
by query' after bulk indexing.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

Fix for 1.: each bulk has 10000 inserts

Best regards.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

Oh, and there is nothing in gc log.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

I am using ES 0.19.8, I want to upgrade to 0.19.9 now but I am not sure
if that will help.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

Hi,

I still need help in this I have the following information:

2 nodes (es1 and es2)

1 index Users with 30 shards and 1 replica

I do the following:

On es2 I run consecutively many bulk index operations (each bulk has
1000 inserts) and I do about 1000 such bulk inserts.

At some point of inserting es1 uses about 300% of processor and bulk

--

--
Met vriendelijke groet,

Martijn van Groningen

--

--

--
Met vriendelijke groet,

Martijn van Groningen

--

Marcin_Dojwa · October 4, 2012, 4:53am

That would solve my problem, I will check this out today. Thanks.

W dniu środa, 3 października 2012 użytkownik Martijn v Groningen napisał:

I found out that the delete by query basically hangs when executed
with an alias that has a filter associated with it.
If you run the delete by query on the concrete index, it should work.
I'll open an issue for this.

Martijn

On 3 October 2012 22:30, Marcin Dojwa m.dojwa@livechatinc.com wrote:

But executing a single delete by query I use 1 alias per query.

W dniu środa, 3 października 2012 użytkownik Marcin Dojwa napisał:

More, I have 1 index and about 50000 aliases to this index.

W dniu środa, 3 października 2012 użytkownik Martijn v Groningen
napisał:

Hi Marcin,

Are you using one or more aliases when executing the delete by query?

Martijn

On 3 October 2012 10:50, Marcin Dojwa m.dojwa@livechatinc.com wrote:

Hi,

The exact mapping is here:
ES 100% processor usage · GitHub
I guess that this is about chat document type because this is the only
one with nested mapping. I can not give you exact data that could
possibly
cause such a situation because this is our clients' private data. Are
you
able to try reproduce the problem with this information about mapping
only?
If not I will try to prepare fake example data.

Best regards.

2012/10/2 Shay Banon kimchy@gmail.com

It seems like its doing the delete by query with nested documents. Can
you share your mapping, and sample data that you index, and the
delete_by_query that you are using?

On Oct 2, 2012, at 4:16 AM, Marcin Dojwa m.dojwa@livechatinc.com
wrote:

Hi,

I attached 4 different (I think) snapshots of curl
localhost:9200/_nodes/hot_threads here:
ES 100% processor usage · GitHub

Best regards.
Marcin Dojwa

2012/9/29 Marcin Dojwa m.dojwa@livechatinc.com

Sure, I will try to reproduce this case on Monday. I updated ES to
0.19.9 so I will check if this happens with this version too.

Best regards.

2012/9/28 Shay Banon kimchy@gmail.com

Can you issue: curl localhost:9200/_nodes/hot_threads and gist the
response?

On Sep 27, 2012, at 1:19 PM, Marcin Dojwa m.dojwa@livechatinc.com
wrote:

One one thing, I am not sure if it hangs on bulk indexing or on
'delete
by query' after bulk indexing.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

Fix for 1.: each bulk has 10000 inserts

Best regards.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

Oh, and there is nothing in gc log.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

I am using ES 0.19.8, I want to upgrade to 0.19.9 now but I am not
sure
if that will help.

2012/9/27 Marcin Dojwa <

--

Marcin_Dojwa · October 4, 2012, 10:25am

You were right Martijn, it works fine since I replace an alias with exact
index name. Thank you.

Best regards.

2012/10/4 Marcin Dojwa m.dojwa@livechatinc.com

That would solve my problem, I will check this out today. Thanks.

W dniu środa, 3 października 2012 użytkownik Martijn v Groningen napisał:

I found out that the delete by query basically hangs when executed
with an alias that has a filter associated with it.
If you run the delete by query on the concrete index, it should work.
I'll open an issue for this.

Martijn

On 3 October 2012 22:30, Marcin Dojwa m.dojwa@livechatinc.com wrote:

But executing a single delete by query I use 1 alias per query.

W dniu środa, 3 października 2012 użytkownik Marcin Dojwa napisał:

More, I have 1 index and about 50000 aliases to this index.

W dniu środa, 3 października 2012 użytkownik Martijn v Groningen
napisał:

Hi Marcin,

Are you using one or more aliases when executing the delete by query?

Martijn

On 3 October 2012 10:50, Marcin Dojwa m.dojwa@livechatinc.com
wrote:

Hi,

The exact mapping is here:
ES 100% processor usage · GitHub
I guess that this is about chat document type because this is the
only
one with nested mapping. I can not give you exact data that could
possibly
cause such a situation because this is our clients' private data.
Are you
able to try reproduce the problem with this information about
mapping only?
If not I will try to prepare fake example data.

Best regards.

2012/10/2 Shay Banon kimchy@gmail.com

It seems like its doing the delete by query with nested documents.
Can
you share your mapping, and sample data that you index, and the
delete_by_query that you are using?

On Oct 2, 2012, at 4:16 AM, Marcin Dojwa m.dojwa@livechatinc.com
wrote:

Hi,

I attached 4 different (I think) snapshots of curl
localhost:9200/_nodes/hot_threads here:
ES 100% processor usage · GitHub

Best regards.
Marcin Dojwa

2012/9/29 Marcin Dojwa m.dojwa@livechatinc.com

Sure, I will try to reproduce this case on Monday. I updated ES to
0.19.9 so I will check if this happens with this version too.

Best regards.

2012/9/28 Shay Banon kimchy@gmail.com

Can you issue: curl localhost:9200/_nodes/hot_threads and gist the
response?

On Sep 27, 2012, at 1:19 PM, Marcin Dojwa m.dojwa@livechatinc.com
wrote:

One one thing, I am not sure if it hangs on bulk indexing or on
'delete
by query' after bulk indexing.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

Fix for 1.: each bulk has 10000 inserts

Best regards.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

Oh, and there is nothing in gc log.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

I am using ES 0.19.8, I want to upgrade to 0.19.9 now but I am not
sure
if that will help.

2012/9/27 Marcin Dojwa <

--

mvg · October 4, 2012, 11:18am

Thanks for verifying this Marcin.

Martijn

On 4 October 2012 12:25, Marcin Dojwa m.dojwa@livechatinc.com wrote:

You were right Martijn, it works fine since I replace an alias with exact
index name. Thank you.

Best regards.

2012/10/4 Marcin Dojwa m.dojwa@livechatinc.com

That would solve my problem, I will check this out today. Thanks.

W dniu środa, 3 października 2012 użytkownik Martijn v Groningen napisał:

I found out that the delete by query basically hangs when executed
with an alias that has a filter associated with it.
If you run the delete by query on the concrete index, it should work.
I'll open an issue for this.

Martijn

On 3 October 2012 22:30, Marcin Dojwa m.dojwa@livechatinc.com wrote:

But executing a single delete by query I use 1 alias per query.

W dniu środa, 3 października 2012 użytkownik Marcin Dojwa napisał:

More, I have 1 index and about 50000 aliases to this index.

W dniu środa, 3 października 2012 użytkownik Martijn v Groningen
napisał:

Hi Marcin,

Are you using one or more aliases when executing the delete by query?

Martijn

On 3 October 2012 10:50, Marcin Dojwa m.dojwa@livechatinc.com
wrote:

Hi,

The exact mapping is here:
ES 100% processor usage · GitHub
I guess that this is about chat document type because this is the
only
one with nested mapping. I can not give you exact data that could
possibly
cause such a situation because this is our clients' private data.
Are you
able to try reproduce the problem with this information about
mapping only?
If not I will try to prepare fake example data.

Best regards.

2012/10/2 Shay Banon kimchy@gmail.com

It seems like its doing the delete by query with nested documents.
Can
you share your mapping, and sample data that you index, and the
delete_by_query that you are using?

On Oct 2, 2012, at 4:16 AM, Marcin Dojwa m.dojwa@livechatinc.com
wrote:

Hi,

I attached 4 different (I think) snapshots of curl
localhost:9200/_nodes/hot_threads here:
ES 100% processor usage · GitHub

Best regards.
Marcin Dojwa

2012/9/29 Marcin Dojwa m.dojwa@livechatinc.com

Sure, I will try to reproduce this case on Monday. I updated ES to
0.19.9 so I will check if this happens with this version too.

Best regards.

2012/9/28 Shay Banon kimchy@gmail.com

Can you issue: curl localhost:9200/_nodes/hot_threads and gist the
response?

On Sep 27, 2012, at 1:19 PM, Marcin Dojwa m.dojwa@livechatinc.com
wrote:

One one thing, I am not sure if it hangs on bulk indexing or on
'delete
by query' after bulk indexing.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

Fix for 1.: each bulk has 10000 inserts

Best regards.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

Oh, and there is nothing in gc log.

2012/9/27 Marcin Dojwa m.dojwa@livechatinc.com

I am using ES 0.19.8, I want to upgrade to 0.19.9 now but I am not
sure
if that will help.

2012/9/27 Marcin Dojwa <

--

--
Met vriendelijke groet,

Martijn van Groningen

--