# Script filter slow query execution
**URL:** https://discuss.elastic.co/t/script-filter-slow-query-execution/15455
**Category:** Elasticsearch
**Created:** [January 28, 2014, 7:27pm UTC](https://discuss.elastic.co/t/script-filter-slow-query-execution/15455 "2014-01-28T19:27:50Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@vinamar](https://discuss.elastic.co/u/vinamar)
#### Post date: [January 28, 2014, 7:27pm UTC](https://discuss.elastic.co/t/script-filter-slow-query-execution/15455/1 "2014-01-28T19:27:50Z")
I'm trying to use script filter with filtered query as given below. The query execution takes around 20 to 30 secs for 70K matching facet results. How to speed up the query execution ?
{
"timeout": 30000,
"highlight": {
"pre\_tags": [
""
],
"post\_tags": [
""
],
"fields": {
"srId": {
"number\_of\_fragments": 0
},
"emails.emailBody": {
"number\_of\_fragments": 0
},
"chatTextArray.text": {
"number\_of\_fragments": 0
}
}
},
"query": {
"custom\_filters\_score": {
"query": {
"filtered": {
"filter": {
"and": [
{
"term": {
"domains.L1.domain.id": "1"
}
},
{
"range": {
"closeDt": {
"from": 1325404800000,
"to": 1390982400000
}
}
}
]
},
"query": {
"query\_string": {
"query": "payment button",
"default\_operator": "AND",
"fields": [
[
"emails.emailBody",
"srId",
"chatTextArray.text"
]
]
}
}
}
},
"filters": [
{
"filter": {
"exists": {
"field": "domains.L1.domain.id"
}
},
"script": "int docscore=doc['domains.L1.domain.1.prob'].value_1.5;int expscore=pow(docscore,7);\_score=\_score_\_source.\_boost\*expscore"
}
]
}
},
"size": 10
}
---
### Author:  [@Binh\_Ly](https://discuss.elastic.co/u/Binh_Ly)
#### Post date: [January 28, 2014, 11:05pm UTC](https://discuss.elastic.co/t/script-filter-slow-query-execution/15455/2 "2014-01-28T23:05:29Z")
Vinoth,
I'd try to eliminate that "\_source.\_boost" part from your script and see if
that makes any difference. If it does, store your doc boost in a field and
access it like:
doc["myboostvalue"].value
and see if that helps.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/5fe43c96-d3e8-4cfa-8004-a7e7c0b665f8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/5fe43c96-d3e8-4cfa-8004-a7e7c0b665f8%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@vinamar](https://discuss.elastic.co/u/vinamar)
#### Post date: [January 29, 2014, 6:58pm UTC](https://discuss.elastic.co/t/script-filter-slow-query-execution/15455/3 "2014-01-29T18:58:03Z")
I tried removing \_source.boost the query performance is significantly improved 30x
Is there a reason why \_source.boost would hit on the performance ?
During index time we compute the document boost and store it in the boost field itself.
Do we need to also store it in a separate field ?
Thanks, Vinoth.
---
### Author:  [@Binh\_Ly](https://discuss.elastic.co/u/Binh_Ly)
#### Post date: [January 29, 2014, 7:22pm UTC](https://discuss.elastic.co/t/script-filter-slow-query-execution/15455/4 "2014-01-29T19:22:08Z")
Vinoth,
When you access \_source.X in a script, the \_source field is loaded per doc,
parsed, and then provided to the script. Depending on how many documents
you are hitting, it can be slow if there are a lot.
The document \_boost should already be factored into the score, but if you
need to extract a numeric boost value and make it part of your script, try
using doc["boostfield"].value, instead of \_source.boostfield. That way, the
values are loaded into memory and would perform better. Just be aware that
it will take up some memory.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/2788e9f4-4532-46dc-971b-77ec54585faa%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/2788e9f4-4532-46dc-971b-77ec54585faa%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@vinamar](https://discuss.elastic.co/u/vinamar)
#### Post date: [January 30, 2014, 8:22pm UTC](https://discuss.elastic.co/t/script-filter-slow-query-execution/15455/5 "2014-01-30T20:22:05Z")
Hi Binh,
We are setting \_boost at index time per document. If that is included as a part of \_score then we are fine.
Can you confirm if the document \_boost is included in the \_score. I don't see it in the query explain.
for doc \_boost: 1.41
"value" : 0.14836232,
"description" : "score(doc=140,freq=22.0 = termFreq=22.0\n), product of:",
"details" : [ {
"value" : 0.15365347,
"description" : "queryWeight, product of:",
"details" : [ {
"value" : 3.2937443,
"description" : "idf(docFreq=142048, maxDocs=1407987)"
}, {
"value" : 0.046650093,
"description" : "queryNorm"
} ]
}, {
"value" : 0.9655644,
"description" : "fieldWeight in 140, product of:",
"details" : [ {
"value" : 4.690416,
"description" : "tf(freq=22.0), with freq of:",
"details" : [ {
"value" : 22.0,
"description" : "termFreq=22.0"
} ]
}, {
"value" : 3.2937443,
"description" : "idf(docFreq=142048, maxDocs=1407987)"
}, {
"value" : 0.0625,
"description" : "fieldNorm(doc=140)"
---
### Author:  [@Binh\_Ly](https://discuss.elastic.co/u/Binh_Ly)
#### Post date: [January 30, 2014, 8:41pm UTC](https://discuss.elastic.co/t/script-filter-slow-query-execution/15455/6 "2014-01-30T20:41:19Z")
Vinoth, I just got word that doc \_boost will be deprecated in ES 1.0. It is
recommended that you start using the function\_score query instead and just
store your doc "boost" as a field and extract it using doc["boost"].value
method moving forward:
[http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/mapping-boost-field.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/mapping-boost-field.html)
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/dd72fbbd-4a0a-4084-9b56-0f6a91df6584%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/dd72fbbd-4a0a-4084-9b56-0f6a91df6584%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@vinamar](https://discuss.elastic.co/u/vinamar)
#### Post date: [January 30, 2014, 10:05pm UTC](https://discuss.elastic.co/t/script-filter-slow-query-execution/15455/7 "2014-01-30T22:05:59Z")
Hi Binh,
Thanks we will take a look at using function\_score query at later point of time.
To switch to use function\_score\_query to access this new boost field will require us to re-process and re-index all documents.
We will migrate to using function\_score query when we plan to re-index documents at later point of time (at that time we will have this new numeric field representing the boost)
Does \_score include the product of \_boost factor as of now ? is there any other way to include the \_boost value without any overhead in the script function ?
fyi. we are on ES 0.90.7
Thanks, Vinoth.
---
### Author:  [@Binh\_Ly](https://discuss.elastic.co/u/Binh_Ly)
#### Post date: [January 30, 2014, 10:25pm UTC](https://discuss.elastic.co/t/script-filter-slow-query-execution/15455/8 "2014-01-30T22:25:20Z")
Vinoth, If you did the \_boost according to this link
([http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-boost-field.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-boost-field.html)),
then yes it is precomputed into the \_score already.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0be266f0-3ada-43e7-836c-d98d54a966b3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0be266f0-3ada-43e7-836c-d98d54a966b3%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [January 31, 2014, 4:01pm UTC](https://discuss.elastic.co/t/script-filter-slow-query-execution/15455/9 "2014-01-31T16:01:09Z")
Judging by the commits, the boost functionality was not removed, only
deprecated. That said, you really should move to query time boosting.
Index-time boosts are encoded inside the field norms. You should see a
difference inside each field norm. If you have omitted norms, then you will
not have any boosts on that field. The field norm is also lossly since it
uses only 1-byte. Some of the many reasons to switch from document time
boosts.
Cheers,
Ivan
On Thu, Jan 30, 2014 at 2:25 PM, Binh Ly [binh@hibalo.com](mailto:binh@hibalo.com) wrote:
> Vinoth, If you did the \_boost according to this link (
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-boost-field.html)),
> then yes it is precomputed into the \_score already.
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> To view this discussion on the web visit
> [https://groups.google.com/d/msgid/elasticsearch/0be266f0-3ada-43e7-836c-d98d54a966b3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0be266f0-3ada-43e7-836c-d98d54a966b3%40googlegroups.com)
> .
>
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCLDBP4qmg0y8Ay%3De%2BPwWjKDduT1Nm%3Dn%2B7mttefA142uQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCLDBP4qmg0y8Ay%3De%2BPwWjKDduT1Nm%3Dn%2B7mttefA142uQ%40mail.gmail.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:53am UTC](https://discuss.elastic.co/t/script-filter-slow-query-execution/15455/10 "2017-07-06T01:53:39Z")