Why to cause OOM when searching with query and facet?

Elastic Stack Elasticsearch
1

Hi folks.

I am a newbie to ES.
My goal is to compute recommendation music list for a user.
The commendation music is simply musics which the user`s friends listened
to.

For that, I have added about 10,000,000 documents as following mapping.

properties: {
follows: {
type: "string"
},
id: {
type: "string"
},
name: {
type: "string"
},
plays: {
properties: {
count: {
type: "long"
},
lastPlayed: {
type: "string"
},
musicId: {
type: "string"
}}}
}

Then, I have searched as following.

{"size":0,"query":{"field":{"follows":"2324"}},"facets":{"plays":{"terms":{"size":10,"script":"doc['plays.musicId'].values"}}}}

I expected search cost as "O(log n * friend count of '2344')" and then
facet computing cost on searched friends docs.
But when I run above query, OutOfMemory occurs as following.

reason:
RemoteTransportException[[m4][inet[/10.99.195.57:9300]][search/phase/query]];
nested: QueryPhaseExecutionException[[music][1]:
query[id:2324],from[0],size[0]: Query Failed [Failed to execute main
query]]; nested: ElasticSearchException[java.lang.OutOfMemoryError: Java
heap space]; nested: ExecutionError[java.lang.OutOfMemoryError: Java heap
space]; nested: OutOfMemoryError[Java heap space];

Who can I help me? Please give me some advice.
Thanks for reading.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Sorry. 'follows' is List. 'plays' is also List.

2013년 10월 18일 금요일 오전 11시 1분 23초 UTC+9, Min Cha 님의 말:

Hi folks.

I am a newbie to ES.
My goal is to compute recommendation music list for a user.
The commendation music is simply musics which the user`s friends listened
to.

For that, I have added about 10,000,000 documents as following mapping.

properties: {
follows: {
type: "string"
},
id: {
type: "string"
},
name: {
type: "string"
},
plays: {
properties: {
count: {
type: "long"
},
lastPlayed: {
type: "string"
},
musicId: {
type: "string"
}}}
}

Then, I have searched as following.

{"size":0,"query":{"field":{"follows":"2324"}},"facets":{"plays":{"terms":{"size":10,"script":"doc['plays.musicId'].values"}}}}

I expected search cost as "O(log n * friend count of '2344')" and then
facet computing cost on searched friends docs.
But when I run above query, OutOfMemory occurs as following.

reason:
RemoteTransportException[[m4][inet[/10.99.195.57:9300]][search/phase/query]];
nested: QueryPhaseExecutionException[[music][1]:
query[id:2324],from[0],size[0]: Query Failed [Failed to execute main
query]]; nested: ElasticSearchException[java.lang.OutOfMemoryError: Java
heap space]; nested: ExecutionError[java.lang.OutOfMemoryError: Java heap
space]; nested: OutOfMemoryError[Java heap space];

Who can I help me? Please give me some advice.
Thanks for reading.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hey,

two notes: First, by default only 1 GB of memory is used by the JVM when
starting up. You can change this using the ES_HEAP_SIZE parameter. See

Second, you should not facet on an analyzed field, but set the field you
want to facet on to not_analyzed. If you need to search on that field as
well, consider using a multi field, see

The first link also explains a bit, why you should not facet on an analyzed
field.

As a last hint, you should monitor the size of your fielddata in order to
prevent out of memory exceptions, see

Hope this helps... When in doubt, always ask :slight_smile:

--Alex

On Fri, Oct 18, 2013 at 4:04 AM, Min Cha minslovey@gmail.com wrote:

Sorry. 'follows' is List. 'plays' is also List.

2013년 10월 18일 금요일 오전 11시 1분 23초 UTC+9, Min Cha 님의 말:

Hi folks.

I am a newbie to ES.
My goal is to compute recommendation music list for a user.
The commendation music is simply musics which the user`s friends listened
to.

For that, I have added about 10,000,000 documents as following mapping.

properties: {
follows: {
type: "string"
},
id: {
type: "string"
},
name: {
type: "string"
},
plays: {
properties: {
count: {
type: "long"
},
lastPlayed: {
type: "string"
},
musicId: {
type: "string"
}}}
}

Then, I have searched as following.

{"size":0,"query":{"field":{"follows":"2324"}},"facets":{"
plays":{"terms":{"size":10,"**script":"doc['plays.musicId'].**values"}}}}

I expected search cost as "O(log n * friend count of '2344')" and then
facet computing cost on searched friends docs.
But when I run above query, OutOfMemory occurs as following.

reason: RemoteTransportException[[m4][**inet[/10.99.195.57:9300]][**search/phase/query]];
nested: QueryPhaseExecutionException[[music][1]:
query[id:2324],from[0],size[0]: Query Failed [Failed to execute main
query]]; nested: ElasticSearchException[java.**lang.OutOfMemoryError:
Java heap space]; nested: ExecutionError[java.lang.**OutOfMemoryError:
Java heap space]; nested: OutOfMemoryError[Java heap space];

Who can I help me? Please give me some advice.
Thanks for reading.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.