Searching from array with OR

Elastic Stack Elasticsearch
1

hello there, im trying to make a query as simple and obviously as fast as
possible.

i have a bunch of items indexed in ES with field sid,

i have a an array of sids -> [1111,2222,3333] and so i want to find all
items in ES where sid==1111 or sid==22222 or sid==3333 etc.

Previously I have only used query string, but my array of sids is pretty
massive (500+) so dont want to use query string, what would be the best way
to find matching items in ES?

Hope that makes sense.
Cheers
Lu

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

am looking at this

so would this be the right way to go?

{
"constant_score" : {
"filter" : {
"terms" : {
"sid" : ["11111", "22222", "3333"],
"execution" : "bool",
"_cache": true
}
}
}
}

On Friday, 1 March 2013 15:18:23 UTC+11, moocow wrote:

hello there, im trying to make a query as simple and obviously as fast as
possible.

i have a bunch of items indexed in ES with field sid,

i have a an array of sids -> [1111,2222,3333] and so i want to find all
items in ES where sid==1111 or sid==22222 or sid==3333 etc.

Previously I have only used query string, but my array of sids is pretty
massive (500+) so dont want to use query string, what would be the best way
to find matching items in ES?

Hope that makes sense.
Cheers
Lu

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Yes, this will be the fastest approach. You should probably use the default
execution mode (bitset) and a cache key (if possible) but regardless, this
approach will handle large arrays very easily.

-Eric

On Thursday, February 28, 2013 11:22:35 PM UTC-5, moocow wrote:

am looking at this

Elasticsearch Platform — Find real-time answers at scale | Elastic

so would this be the right way to go?

{
"constant_score" : {
"filter" : {
"terms" : {
"sid" : ["11111", "22222", "3333"],
"execution" : "bool",
"_cache": true
}
}
}
}

On Friday, 1 March 2013 15:18:23 UTC+11, moocow wrote:

hello there, im trying to make a query as simple and obviously as fast as
possible.

i have a bunch of items indexed in ES with field sid,

i have a an array of sids -> [1111,2222,3333] and so i want to find all
items in ES where sid==1111 or sid==22222 or sid==3333 etc.

Previously I have only used query string, but my array of sids is pretty
massive (500+) so dont want to use query string, what would be the best way
to find matching items in ES?

Hope that makes sense.
Cheers
Lu

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Thanks for replying so quickly!

I got the following query working

curl -XGET localhost:9200/users/user/_search -d '{"query" :
{
"terms" : {
"fb_id" : [ "11111", "2222" ],
"execution" : "bool",
"_cache_key" : "user_2_friends"
}
}}'

for some reason when I put in the line

"_cache": true, it gives me a parse error.

I also tried "cache":"true" but still no luck.

With the _cache_key set, will it default to cache=true?

Cheers
lu

On Friday, 1 March 2013 15:18:23 UTC+11, moocow wrote:

hello there, im trying to make a query as simple and obviously as fast as
possible.

i have a bunch of items indexed in ES with field sid,

i have a an array of sids -> [1111,2222,3333] and so i want to find all
items in ES where sid==1111 or sid==22222 or sid==3333 etc.

Previously I have only used query string, but my array of sids is pretty
massive (500+) so dont want to use query string, what would be the best way
to find matching items in ES?

Hope that makes sense.
Cheers
Lu

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

With a mode of bool, each term filter is cached so you wouldn't specify a
key. If you use the default mode, the whole filter is cached as a single
entity. Without the cache_key, the key becomes the filter (so 500 terms in
your case). When you specify a cache_key you essentially provide a smaller
key which saves some memory.

-Eric

On Thursday, February 28, 2013 11:44:24 PM UTC-5, moocow wrote:

Thanks for replying so quickly!

I got the following query working

curl -XGET localhost:9200/users/user/_search -d '{"query" :
{
"terms" : {
"fb_id" : [ "11111", "2222" ],
"execution" : "bool",
"_cache_key" : "user_2_friends"
}
}}'

for some reason when I put in the line

"_cache": true, it gives me a parse error.

I also tried "cache":"true" but still no luck.

With the _cache_key set, will it default to cache=true?

Cheers
lu

On Friday, 1 March 2013 15:18:23 UTC+11, moocow wrote:

hello there, im trying to make a query as simple and obviously as fast as
possible.

i have a bunch of items indexed in ES with field sid,

i have a an array of sids -> [1111,2222,3333] and so i want to find all
items in ES where sid==1111 or sid==22222 or sid==3333 etc.

Previously I have only used query string, but my array of sids is pretty
massive (500+) so dont want to use query string, what would be the best way
to find matching items in ES?

Hope that makes sense.
Cheers
Lu

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

curl -XGET localhost:9200/users/user/_search -d '{"query" :
{
"terms" : {
"fb_id" : [ "11111", "2222" ],
"execution" : "bool",
"_cache_key" : "user_2_friends"
}
}}'

for some reason when I put in the line
"_cache": true, it gives me a parse error.

You can only cache filters, not queries.

You're using the "terms" query, not the "terms" filter. It would be much
more efficient to use the terms filter, as you pasted in your original
email:

{
"constant_score" : {
"filter" : {
"terms" : {
"sid" : ["11111", "22222", "3333"],
"execution" : "bool",
"_cache": true
}
}
}
}

clint

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

7

but when i run this

curl -XGET localhost:9200/users/user/_search -d '{
"constant_score" : {
"filter" : {
"terms" : {
"fb_id" : ["111", "22222", "3333"],
"execution" : "bitset",
"_cache": true,
"_cache_key":'user_2_friends'
}
}
}
}'

I get

{"error":"SearchPhaseExecutionException[Failed to execute phase [query],
total failure; shardFailures .....: SearchParseException[[users][3]:
from[-1],size[-1]: Parse Failure [Failed to parse source [{\n
"constant_score" : {\n "filter" : {\n "terms" :
{\n "fb_id" : ["111", "22222",
"3333"],\n "execution" : "bool",
\n "_cache_key":user_2_friends\n }\n }\

any ideas?

Cheers
Lu

On Friday, 1 March 2013 23:00:03 UTC+11, Clinton Gormley wrote:

curl -XGET localhost:9200/users/user/_search -d '{"query" :
{
"terms" : {
"fb_id" : [ "11111", "2222" ],
"execution" : "bool",
"_cache_key" : "user_2_friends"
}
}}'

for some reason when I put in the line
"_cache": true, it gives me a parse error.

You can only cache filters, not queries.

You're using the "terms" query, not the "terms" filter. It would be much
more efficient to use the terms filter, as you pasted in your original
email:

{
"constant_score" : {
"filter" : {
"terms" : {
"sid" : ["11111", "22222", "3333"],
"execution" : "bool",
"_cache": true
}
}
}
}

clint

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

8

okay i think i have it,

it was complaining about \u so I retyped the entire thing and put the query
at the start ( didnt realise you have to always have query as root?)

curl -XGET localhost:9200/users/user/_search?pretty=true -d '{"query":
{"constant_score":{ "filter": {"terms":{"fb_id":["1111"]}} }}}'

On Saturday, 2 March 2013 13:06:35 UTC+11, moocow wrote:

but when i run this

curl -XGET localhost:9200/users/user/_search -d '{
"constant_score" : {
"filter" : {
"terms" : {
"fb_id" : ["111", "22222", "3333"],
"execution" : "bitset",
"_cache": true,
"_cache_key":'user_2_friends'
}
}
}
}'

I get

{"error":"SearchPhaseExecutionException[Failed to execute phase [query],
total failure; shardFailures .....: SearchParseException[[users][3]:
from[-1],size[-1]: Parse Failure [Failed to parse source [{\n
"constant_score" : {\n "filter" : {\n "terms" :
{\n "fb_id" : ["111", "22222",
"3333"],\n "execution" : "bool",
\n "_cache_key":user_2_friends\n }\n }\

any ideas?

Cheers
Lu

On Friday, 1 March 2013 23:00:03 UTC+11, Clinton Gormley wrote:

curl -XGET localhost:9200/users/user/_search -d '{"query" :
{
"terms" : {
"fb_id" : [ "11111", "2222" ],
"execution" : "bool",
"_cache_key" : "user_2_friends"
}
}}'

for some reason when I put in the line
"_cache": true, it gives me a parse error.

You can only cache filters, not queries.

You're using the "terms" query, not the "terms" filter. It would be much
more efficient to use the terms filter, as you pasted in your original
email:

{
"constant_score" : {
"filter" : {
"terms" : {
"sid" : ["11111", "22222", "3333"],
"execution" : "bool",
"_cache": true
}
}
}
}

clint

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.