How to improve search performance in ES?


(dark_shadow) #1

Hi,

I'm working on improving the search response of ES but not able to do
anything. My scenario is something like this:

I'm using 3 ES queries to get relevant results for my autocompleter.

  1. A function score query with a match query ( To get a correct match if
    user typed query is available in documents based on popularity)

  2. A multi match query (To handle those scenarios in which a user types
    some text which is present in different fields in a document since my
    documents are multi fields like name, address, city, state, country )

  3. A query string (In order to ensure if I missed user query by the above
    type I'll be able to search using more powerful but less accurate query
    string)

Along with all the 3 queries, I'm using 4 filters (clubbed using AND
filter).

My performance is really bad and I want to improve it along with delivering
relevat results in my autocompleter.

Can anyone help me how can I improve this ? Any way I can club the queries
for better performance ?

I have read that I BOOL filters should be used instead of AND filter since
they use bitset which are cached internally. I think this makes one
improvement because if in the first query ES stores the information of
filters in bitset, it can reuse it in other two queries. That will make the
thigs a little fast but based on queries, I'm not able to do any
improvement ?

Is there any way by which I can combine match and multi-match queries ( 1
and 2) into a single effective query.

Also, in place of query_string should I use some other query for faster
execution.

Any suggestions are welcome.
Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5d99495b-20ef-46b6-a069-365574fdc0a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #2

For autocompletion, you should use the completion suggester

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-suggesters-completion.html

or edge ngram tokenizer

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-edgengram-tokenizer.html

Jörg

On Fri, Jul 11, 2014 at 8:11 PM, coder mukulnitkkr@gmail.com wrote:

Hi,

I'm working on improving the search response of ES but not able to do
anything. My scenario is something like this:

I'm using 3 ES queries to get relevant results for my autocompleter.

  1. A function score query with a match query ( To get a correct match if
    user typed query is available in documents based on popularity)

  2. A multi match query (To handle those scenarios in which a user types
    some text which is present in different fields in a document since my
    documents are multi fields like name, address, city, state, country )

  3. A query string (In order to ensure if I missed user query by the above
    type I'll be able to search using more powerful but less accurate query
    string)

Along with all the 3 queries, I'm using 4 filters (clubbed using AND
filter).

My performance is really bad and I want to improve it along with
delivering relevat results in my autocompleter.

Can anyone help me how can I improve this ? Any way I can club the queries
for better performance ?

I have read that I BOOL filters should be used instead of AND filter since
they use bitset which are cached internally. I think this makes one
improvement because if in the first query ES stores the information of
filters in bitset, it can reuse it in other two queries. That will make the
thigs a little fast but based on queries, I'm not able to do any
improvement ?

Is there any way by which I can combine match and multi-match queries ( 1
and 2) into a single effective query.

Also, in place of query_string should I use some other query for faster
execution.

Any suggestions are welcome.
Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5d99495b-20ef-46b6-a069-365574fdc0a9%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/5d99495b-20ef-46b6-a069-365574fdc0a9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEBchn0AKB_heFFjr%2B%3Df1X_CzfJBGFnFQf_rEpgAiHUvA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(dark_shadow) #3

Hi Jörg,

I have seen these links. I'm using ngram tokenizer. Issue which I'm facing
is slow response time. For that I need some suggestions, how can I improve
it ? Is there anyway by which I can query in a better way ? Also, I'm using
a match query in a field in one of my filters but I have read that term
filters are more effective. Can you give me some insight how can I use term
filter in this case even if the field on which I want to apply the filter
is not present in all the documents.

Thanks

On Saturday, 12 July 2014 00:09:50 UTC+5:30, Jörg Prante wrote:

For autocompletion, you should use the completion suggester

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-suggesters-completion.html

or edge ngram tokenizer

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-edgengram-tokenizer.html

Jörg

On Fri, Jul 11, 2014 at 8:11 PM, coder <mukul...@gmail.com <javascript:>>
wrote:

Hi,

I'm working on improving the search response of ES but not able to do
anything. My scenario is something like this:

I'm using 3 ES queries to get relevant results for my autocompleter.

  1. A function score query with a match query ( To get a correct match if
    user typed query is available in documents based on popularity)

  2. A multi match query (To handle those scenarios in which a user types
    some text which is present in different fields in a document since my
    documents are multi fields like name, address, city, state, country )

  3. A query string (In order to ensure if I missed user query by the above
    type I'll be able to search using more powerful but less accurate query
    string)

Along with all the 3 queries, I'm using 4 filters (clubbed using AND
filter).

My performance is really bad and I want to improve it along with
delivering relevat results in my autocompleter.

Can anyone help me how can I improve this ? Any way I can club the
queries for better performance ?

I have read that I BOOL filters should be used instead of AND filter
since they use bitset which are cached internally. I think this makes one
improvement because if in the first query ES stores the information of
filters in bitset, it can reuse it in other two queries. That will make the
thigs a little fast but based on queries, I'm not able to do any
improvement ?

Is there any way by which I can combine match and multi-match queries ( 1
and 2) into a single effective query.

Also, in place of query_string should I use some other query for faster
execution.

Any suggestions are welcome.
Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5d99495b-20ef-46b6-a069-365574fdc0a9%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/5d99495b-20ef-46b6-a069-365574fdc0a9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/741a7bc5-ffd7-4ba7-9296-ff6fff8f559f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Mark Walkom) #4

Can you elaborate on how you're measuring and comparing these response
times and why you feel they are slow?
It might also help if you can put a sample query and document into
a gist/pastebin

Also, is your cluster under load when you run these queries? What metrics
are you gathering around that side?

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 12 July 2014 05:01, coder mukulnitkkr@gmail.com wrote:

Hi Jörg,

I have seen these links. I'm using ngram tokenizer. Issue which I'm facing
is slow response time. For that I need some suggestions, how can I improve
it ? Is there anyway by which I can query in a better way ? Also, I'm using
a match query in a field in one of my filters but I have read that term
filters are more effective. Can you give me some insight how can I use term
filter in this case even if the field on which I want to apply the filter
is not present in all the documents.

Thanks

On Saturday, 12 July 2014 00:09:50 UTC+5:30, Jörg Prante wrote:

For autocompletion, you should use the completion suggester

http://www.elasticsearch.org/guide/en/elasticsearch/
reference/current/search-suggesters-completion.html

or edge ngram tokenizer

http://www.elasticsearch.org/guide/en/elasticsearch/
reference/current/analysis-edgengram-tokenizer.html

Jörg

On Fri, Jul 11, 2014 at 8:11 PM, coder mukul...@gmail.com wrote:

Hi,

I'm working on improving the search response of ES but not able to do
anything. My scenario is something like this:

I'm using 3 ES queries to get relevant results for my autocompleter.

  1. A function score query with a match query ( To get a correct match
    if user typed query is available in documents based on popularity)

  2. A multi match query (To handle those scenarios in which a user types
    some text which is present in different fields in a document since my
    documents are multi fields like name, address, city, state, country )

  3. A query string (In order to ensure if I missed user query by the
    above type I'll be able to search using more powerful but less accurate
    query string)

Along with all the 3 queries, I'm using 4 filters (clubbed using AND
filter).

My performance is really bad and I want to improve it along with
delivering relevat results in my autocompleter.

Can anyone help me how can I improve this ? Any way I can club the
queries for better performance ?

I have read that I BOOL filters should be used instead of AND filter
since they use bitset which are cached internally. I think this makes one
improvement because if in the first query ES stores the information of
filters in bitset, it can reuse it in other two queries. That will make the
thigs a little fast but based on queries, I'm not able to do any
improvement ?

Is there any way by which I can combine match and multi-match queries (
1 and 2) into a single effective query.

Also, in place of query_string should I use some other query for faster
execution.

Any suggestions are welcome.
Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/5d99495b-20ef-46b6-a069-365574fdc0a9%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/5d99495b-20ef-46b6-a069-365574fdc0a9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/741a7bc5-ffd7-4ba7-9296-ff6fff8f559f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/741a7bc5-ffd7-4ba7-9296-ff6fff8f559f%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624ZJUXjDVyXFtQSB0-0Mghf5FHC%2Bmm%2BiJUiSCjKJtPDCbg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #5