Hi, I notice that using a script_fields that returns true or false values is going much faster then
using the same script but with filter script declaration (so it will filter the docs returning false).
I was sure that the filter script is taking so long because I'm using the source().get(...) method, but turns out that when using the same script, only with script_fields instead, I'm receiving the performance I need. the only problem here is that I want to filter the docs that now have "MessageReverted" = false.
1.Any way I can filter the docs containing "MessageReverted" = false ?(some wrapper query?)
2. Any idea way the filter script is taking much longer then the script field(8000 mill against 250 mill)?
both ways are retrieving the source() for the script logic so it can't be a matter of source fetching as far as I understand.
what kind of query are you executing? Using script fields results in the
scipt only being executed for each search hit, whereas executing it as a
script filter it might need to execute for each document in your index (you
can try to cache the script filter so it might be faster for subsequent
requests).
Hope this helps as a start for optimization, if not, please provide some
more information.
Hi, I notice that using a script_fields that returns true or false values
is
going much faster then
using the same script but with filter script declaration (so it will filter
the docs returning false).
I was sure that the filter script is taking so long because I'm using the
source().get(...) method, but turns out that when using the same script,
only with script_fields instead, I'm receiving the performance I need. the
only problem here is that I want to filter the docs that now have
"MessageReverted" = false.
1.Any way I can filter the docs containing "MessageReverted" = false
?(some
wrapper query?)
2. Any idea way the filter script is taking much longer then the script
field(8000 mill against 250 mill)?
both ways are retrieving the source() for the script logic so it can't be a
matter of source fetching as far as I understand.
Hi Alex,
1.I checked the cache solution but its taking 15 times more then my starting
time (10s against 150s), so that will be a problem since my filter has
dynamic params.
It does go fast once it's stored though. Do you know if it's possible to
do some kind of caching for all source documents for future queries?
2.From what I understand ,both the filter script and the script_field are
suppose to go over each document that results from the prior query.
The only thing I can think of that makes the difference is that the
script_filter actually needs to filter the false documents (for the hit
count) while the script_field only
needs to add the field for the first 10 document returning by default.
I'm trying to figure out how I can speed the response when using source()
on native java script.
I'm assuming the bottle neck is somewhere within creating the response. I
read that using source has some overhead because elasticsearch has to
parse the json source,
but if that was the case here, then I should have received the same big
overhead for both script_field and filter script runs.
All I actually need is the hit count so if I'm correct about the response
parsing and that can be excluded I'll be really glad.
what kind of query are you executing? Using script fields results in the
scipt only being executed for each search hit, whereas executing it as a
script filter it might need to execute for each document in your index (you
can try to cache the script filter so it might be faster for subsequent
requests).
Hope this helps as a start for optimization, if not, please provide some
more information.
Hi, I notice that using a script_fields that returns true or false values
is
going much faster then
using the same script but with filter script declaration (so it will
filter
the docs returning false).
I was sure that the filter script is taking so long because I'm using the
source().get(...) method, but turns out that when using the same script,
only with script_fields instead, I'm receiving the performance I need.
the
only problem here is that I want to filter the docs that now have
"MessageReverted" = false.
1.Any way I can filter the docs containing "MessageReverted" = false
?(some
wrapper query?)
2. Any idea way the filter script is taking much longer then the script
field(8000 mill against 250 mill)?
both ways are retrieving the source() for the script logic so it can't be
a
matter of source fetching as far as I understand.
Can you actually post your solution If you figured out.
I am having similar issue, I need to filter search result based on
script_field. I don't want to use filter_script though because I am using
facets and I want my records to filter out for facets too.
Do you know if can extends any class or any any plugin or anything to
filter my records based on the script field.
On Sunday, July 7, 2013 1:21:38 PM UTC-4, Oreno wrote:
Hi Alex,
1.I checked the cash solution but its taking 15 times more then my
starting time (10s against 150s), so that will be a problem since my filter
has dynamic params.
It does go fast once it's stored though. Do you know if it's possible to
do some kind of cashing for all source documents for future queries?
2.From what I understand ,both the filter script and the script_field are
suppose to go over each document that results from the prior query.
The only thing I can think of that makes the difference is that the
script_filter actually needs to filter the false documents (for the hit
count) while the script_field only
needs to add the field for the first 10 document returning by default.
I'm trying to figure out how I can speed the response when using source()
on native java script.
I'm assuming the bottle neck is somewhere within creating the response. I
read that using source has some overhead because elasticsearch has to
parse the json source,
but if that was the case here, then I should have received the same big
overhead for both script_field and filter script runs.
All I actually need is the hit count so if I'm correct about the response
parsing and that can be excluded I'll be really glad.
what kind of query are you executing? Using script fields results in the
scipt only being executed for each search hit, whereas executing it as a
script filter it might need to execute for each document in your index (you
can try to cache the script filter so it might be faster for subsequent
requests).
Hope this helps as a start for optimization, if not, please provide some
more information.
Hi, I notice that using a script_fields that returns true or false
values is
going much faster then
using the same script but with filter script declaration (so it will
filter
the docs returning false).
I was sure that the filter script is taking so long because I'm using
the
source().get(...) method, but turns out that when using the same script,
only with script_fields instead, I'm receiving the performance I need.
the
only problem here is that I want to filter the docs that now have
"MessageReverted" = false.
1.Any way I can filter the docs containing "MessageReverted" = false
?(some
wrapper query?)
2. Any idea way the filter script is taking much longer then the script
field(8000 mill against 250 mill)?
both ways are retrieving the source() for the script logic so it can't
be a
matter of source fetching as far as I understand.
I'm trying to filter on a calculated script field as well.
Have you figured this out Kajal?
On Tuesday, May 27, 2014 10:49:35 AM UTC-6, Kajal Patel wrote:
Hey,
Can you actually post your solution If you figured out.
I am having similar issue, I need to filter search result based on
script_field. I don't want to use filter_script though because I am using
facets and I want my records to filter out for facets too.
Do you know if can extends any class or any any plugin or anything to
filter my records based on the script field.
On Sunday, July 7, 2013 1:21:38 PM UTC-4, Oreno wrote:
Hi Alex,
1.I checked the cash solution but its taking 15 times more then my
starting time (10s against 150s), so that will be a problem since my filter
has dynamic params.
It does go fast once it's stored though. Do you know if it's possible to
do some kind of cashing for all source documents for future queries?
2.From what I understand ,both the filter script and the script_field are
suppose to go over each document that results from the prior query.
The only thing I can think of that makes the difference is that the
script_filter actually needs to filter the false documents (for the hit
count) while the script_field only
needs to add the field for the first 10 document returning by default.
I'm trying to figure out how I can speed the response when using source()
on native java script.
I'm assuming the bottle neck is somewhere within creating the response. I
read that using source has some overhead because elasticsearch has to
parse the json source,
but if that was the case here, then I should have received the same big
overhead for both script_field and filter script runs.
All I actually need is the hit count so if I'm correct about the response
parsing and that can be excluded I'll be really glad.
what kind of query are you executing? Using script fields results in the
scipt only being executed for each search hit, whereas executing it as a
script filter it might need to execute for each document in your index (you
can try to cache the script filter so it might be faster for subsequent
requests).
Hope this helps as a start for optimization, if not, please provide some
more information.
Hi, I notice that using a script_fields that returns true or false
values is
going much faster then
using the same script but with filter script declaration (so it will
filter
the docs returning false).
I was sure that the filter script is taking so long because I'm using
the
source().get(...) method, but turns out that when using the same script,
only with script_fields instead, I'm receiving the performance I need.
the
only problem here is that I want to filter the docs that now have
"MessageReverted" = false.
1.Any way I can filter the docs containing "MessageReverted" = false
?(some
wrapper query?)
2. Any idea way the filter script is taking much longer then the script
field(8000 mill against 250 mill)?
both ways are retrieving the source() for the script logic so it can't
be a
matter of source fetching as far as I understand.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
