The difference of query and filter


(J. Schulz) #1

Hello together,

does anyone know what is the exact difference between the following 2
requests? Both requests returns the same result.

I read the elasticsearch docs about filter and query, but I didn't
understand the exact difference.

What is faster? What internally happends? What is the right way?

From the docs about filters:

"As a general rule, filters should be used instead of queries ... for
queries on exact values"

But I found tons of examples that uses "query -> term -> key -> value". Why
is it so?

curl "http://localhost:9200/myapp/event/_search?pretty=true" -d '{
"filter" : {
"and" : [
{
"term" : {
"user_id" : "2"
}
},{
"range" : {
"time" : {
"from" : 1385655592000,
"to" : 1386260392000
}
}
}
]
},
"sort" : {
"time" : { "order" : "desc" }
}
}'

curl "http://localhost:9200/myapp/event/_search?pretty=true" -d '{
"query" : {
"filtered" : {
"query" : {
"term" : { "user_id" : "2" }
},
"filter" : {
"range" : {
"time" : {
"from" : 1385655592000,
"to" : 1386260392000
}
}
}
}
},
"sort" : {
"time" : { "order" : "desc" }
}
}'

Cheers
Jonny

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/df6ed558-1b8c-45cb-905c-152963870819%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Ivan Brusic) #2

Two key differences stick out:

  1. Filters do not contribute to the scoring. In your first example, since
    there is no query (essentially a match all behind the scenes), all the
    documents will have a score of 1.0.

  2. Filters can be cached. Internally, filters are represented with bitsets.
    A filter check simply sees if the bit that represents that document/field
    is checked. This difference means filters can be faster than queries.

In general, filters work best on fields where a full-text search will not
be executed (such as the ones your provided).

Cheers,

Ivan

On Thu, Dec 5, 2013 at 12:36 PM, J. Schulz js.bloonix@gmail.com wrote:

Hello together,

does anyone know what is the exact difference between the following 2
requests? Both requests returns the same result.

I read the elasticsearch docs about filter and query, but I didn't
understand the exact difference.

What is faster? What internally happends? What is the right way?

From the docs about filters:

"As a general rule, filters should be used instead of queries ... for
queries on exact values"

But I found tons of examples that uses "query -> term -> key -> value".
Why is it so?

curl "http://localhost:9200/myapp/event/_search?pretty=true" -d '{
"filter" : {
"and" : [
{
"term" : {
"user_id" : "2"
}
},{
"range" : {
"time" : {
"from" : 1385655592000,
"to" : 1386260392000
}
}
}
]
},
"sort" : {
"time" : { "order" : "desc" }
}
}'

curl "http://localhost:9200/myapp/event/_search?pretty=true" -d '{
"query" : {
"filtered" : {
"query" : {
"term" : { "user_id" : "2" }
},
"filter" : {
"range" : {
"time" : {
"from" : 1385655592000,
"to" : 1386260392000
}
}
}
}
},
"sort" : {
"time" : { "order" : "desc" }
}
}'

Cheers
Jonny

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/df6ed558-1b8c-45cb-905c-152963870819%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDwHX89_q7rTN39z%2BNvMXcd1DwhP02ykFfZ5RQF97-m8w%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #3

Using only filters is faster as filters are cached. That means that if you execute the same query (could be with different parameters), you will see that it's faster.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 5 déc. 2013 à 21:36, "J. Schulz" js.bloonix@gmail.com a écrit :

Hello together,

does anyone know what is the exact difference between the following 2 requests? Both requests returns the same result.

I read the elasticsearch docs about filter and query, but I didn't understand the exact difference.

What is faster? What internally happends? What is the right way?

From the docs about filters:

"As a general rule, filters should be used instead of queries ... for queries on exact values"

But I found tons of examples that uses "query -> term -> key -> value". Why is it so?

curl "http://localhost:9200/myapp/event/_search?pretty=true" -d '{
"filter" : {
"and" : [
{
"term" : {
"user_id" : "2"
}
},{
"range" : {
"time" : {
"from" : 1385655592000,
"to" : 1386260392000
}
}
}
]
},
"sort" : {
"time" : { "order" : "desc" }
}
}'

curl "http://localhost:9200/myapp/event/_search?pretty=true" -d '{
"query" : {
"filtered" : {
"query" : {
"term" : { "user_id" : "2" }
},
"filter" : {
"range" : {
"time" : {
"from" : 1385655592000,
"to" : 1386260392000
}
}
}
}
},
"sort" : {
"time" : { "order" : "desc" }
}
}'

Cheers
Jonny

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/df6ed558-1b8c-45cb-905c-152963870819%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/10A38EB3-D32D-4736-8248-A2C27A6E4DE1%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #4

filters are faster. Ivan is as well faster than me! :wink:

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 5 déc. 2013 à 21:50, David Pilato david@pilato.fr a écrit :

Using only filters is faster as filters are cached. That means that if you execute the same query (could be with different parameters), you will see that it's faster.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 5 déc. 2013 à 21:36, "J. Schulz" js.bloonix@gmail.com a écrit :

Hello together,

does anyone know what is the exact difference between the following 2 requests? Both requests returns the same result.

I read the elasticsearch docs about filter and query, but I didn't understand the exact difference.

What is faster? What internally happends? What is the right way?

From the docs about filters:

"As a general rule, filters should be used instead of queries ... for queries on exact values"

But I found tons of examples that uses "query -> term -> key -> value". Why is it so?

curl "http://localhost:9200/myapp/event/_search?pretty=true" -d '{
"filter" : {
"and" : [
{
"term" : {
"user_id" : "2"
}
},{
"range" : {
"time" : {
"from" : 1385655592000,
"to" : 1386260392000
}
}
}
]
},
"sort" : {
"time" : { "order" : "desc" }
}
}'

curl "http://localhost:9200/myapp/event/_search?pretty=true" -d '{
"query" : {
"filtered" : {
"query" : {
"term" : { "user_id" : "2" }
},
"filter" : {
"range" : {
"time" : {
"from" : 1385655592000,
"to" : 1386260392000
}
}
}
}
},
"sort" : {
"time" : { "order" : "desc" }
}
}'

Cheers
Jonny

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/df6ed558-1b8c-45cb-905c-152963870819%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/10A38EB3-D32D-4736-8248-A2C27A6E4DE1%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/EAA013AD-AAB1-43A0-8CDB-FEB7495D7EA9%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.


(J. Schulz) #5

Great! Thank you for your explantation! :slight_smile:

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5dd814cf-6b63-4d1c-871b-b35f9bb42f36%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Hannes Korte) #6

Hi,

in addition to the mentioned differences, filters don't influence
facets. So, if you want to compute facets only on the the filtered
search results, you have to use a filtered query. Otherwise the facets
are computed on the unfiltered results.

Best regards
Hannes

On 05.12.2013 21:36, J. Schulz wrote:

Hello together,

does anyone know what is the exact difference between the following 2
requests? Both requests returns the same result.

I read the elasticsearch docs about filter and query, but I didn't
understand the exact difference.

What is faster? What internally happends? What is the right way?

From the docs about filters:

"As a general rule, filters should be used instead of queries ... for
queries on exact values"

But I found tons of examples that uses "query -> term -> key -> value". Why
is it so?

curl "http://localhost:9200/myapp/event/_search?pretty=true" -d '{
"filter" : {
"and" : [
{
"term" : {
"user_id" : "2"
}
},{
"range" : {
"time" : {
"from" : 1385655592000,
"to" : 1386260392000
}
}
}
]
},
"sort" : {
"time" : { "order" : "desc" }
}
}'

curl "http://localhost:9200/myapp/event/_search?pretty=true" -d '{
"query" : {
"filtered" : {
"query" : {
"term" : { "user_id" : "2" }
},
"filter" : {
"range" : {
"time" : {
"from" : 1385655592000,
"to" : 1386260392000
}
}
}
}
},
"sort" : {
"time" : { "order" : "desc" }
}
}'

Cheers
Jonny

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/52A1AC8F.9000904%40hkorte.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #7