# Optimise fuzzy search **URL:** \ **Category:** Elasticsearch\ **Created:** [June 19, 2013, 2:50am UTC](https://discuss.elastic.co/t/optimise-fuzzy-search/12471 "2013-06-19T02:50:35Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Arjit\_Gupta](https://avatars.discourse-cdn.com/v4/letter/a/e9bcb4/32.png) [@Arjit\_Gupta](https://discuss.elastic.co/u/Arjit_Gupta)\ **Post date:** [June 19, 2013, 2:50am UTC](https://discuss.elastic.co/t/optimise-fuzzy-search/12471/1 "2013-06-19T02:50:35Z")
I have about 20 million documents with book titles. I want to find similar titles by applying edit distance on the tittles. By the document of fuzzy-query does the same. [http://www.elasticsearch.org/guide/reference/query-dsl/fuzzy-query/](http://www.elasticsearch.org/guide/reference/query-dsl/fuzzy-query/) The problem now is search result are very slow. It takes about 3sec to search 5-6 letter title in Es with fuzzy-query. Is there any other query type I can use which can make it faster and give the same/similar results ? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 19, 2013, 3:05am UTC](https://discuss.elastic.co/t/optimise-fuzzy-search/12471/2 "2013-06-19T03:05:38Z")
What are your memory settings? How many nodes do you have? How many shards? -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 19 juin 2013 à 04:50, Arjit Gupta [arjit292@gmail.com](mailto:arjit292@gmail.com) a écrit : ## I have about 20 million documents with book titles. I want to find similar titles by applying edit distance on the tittles. By the document of fuzzy-query does the same. [http://www.elasticsearch.org/guide/reference/query-dsl/fuzzy-query/](http://www.elasticsearch.org/guide/reference/query-dsl/fuzzy-query/) The problem now is search result are very slow. It takes about 3sec to search 5-6 letter title in Es with fuzzy-query. Is there any other query type I can use which can make it faster and give the same/similar results ? You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Arjit\_Gupta](https://avatars.discourse-cdn.com/v4/letter/a/e9bcb4/32.png) [@Arjit\_Gupta](https://discuss.elastic.co/u/Arjit_Gupta)\ **Post date:** [June 19, 2013, 3:20am UTC](https://discuss.elastic.co/t/optimise-fuzzy-search/12471/3 "2013-06-19T03:20:05Z")
What are your memory settings? How many nodes do you have? How many shards? I have given about 16G of ram in 4 node cluster. I have 5 shards. My total document size is about 10G. So I am sure everything is in memory. Thanks , Arjit On Wed, Jun 19, 2013 at 8:35 AM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > What are your memory settings? > How many nodes do you have? > How many shards? > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 19 juin 2013 à 04:50, Arjit Gupta [arjit292@gmail.com](mailto:arjit292@gmail.com) a écrit : > > I have about 20 million documents with book titles. I want to find similar > titles by applying edit distance on the tittles. > By the document of fuzzy-query does the same. > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/fuzzy-query/) > The problem now is search result are very slow. It takes about 3sec to > search 5-6 letter title in Es with fuzzy-query. > Is there any other query type I can use which can make it faster and give > the same/similar results ? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/Q2vC2N1Gcrg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/Q2vC2N1Gcrg/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 19, 2013, 6:46am UTC](https://discuss.elastic.co/t/optimise-fuzzy-search/12471/4 "2013-06-19T06:46:28Z")
16Gb of 32 Gb available per box, right? Do you see anything in logs? -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 19 juin 2013 à 05:20, Arjit Gupta [arjit292@gmail.com](mailto:arjit292@gmail.com) a écrit : What are your memory settings? How many nodes do you have? How many shards? I have given about 16G of ram in 4 node cluster. I have 5 shards. My total document size is about 10G. So I am sure everything is in memory. Thanks , Arjit On Wed, Jun 19, 2013 at 8:35 AM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > What are your memory settings? > How many nodes do you have? > How many shards? > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 19 juin 2013 à 04:50, Arjit Gupta [arjit292@gmail.com](mailto:arjit292@gmail.com) a écrit : > > ## I have about 20 million documents with book titles. I want to find similar titles by applying edit distance on the tittles. By the document of fuzzy-query does the same. [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/fuzzy-query/) The problem now is search result are very slow. It takes about 3sec to search 5-6 letter title in Es with fuzzy-query. Is there any other query type I can use which can make it faster and give the same/similar results ? > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/Q2vC2N1Gcrg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/Q2vC2N1Gcrg/unsubscribe). > To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Arjit\_Gupta](https://avatars.discourse-cdn.com/v4/letter/a/e9bcb4/32.png) [@Arjit\_Gupta](https://discuss.elastic.co/u/Arjit_Gupta)\ **Post date:** [June 19, 2013, 9:07am UTC](https://discuss.elastic.co/t/optimise-fuzzy-search/12471/5 "2013-06-19T09:07:51Z")
Hi, Yes 16Gb of 32Gb is available on the box. I cant see anything in the logs. Their are no exception in the logs. And I think 2-3 sec is lot of time. Thanks, Arjit Thanks , Arjit On Wed, Jun 19, 2013 at 12:16 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > 16Gb of 32 Gb available per box, right? > Do you see anything in logs? > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 19 juin 2013 à 05:20, Arjit Gupta [arjit292@gmail.com](mailto:arjit292@gmail.com) a écrit : > > What are your memory settings? > > How many nodes do you have? > How many shards? > > I have given about 16G of ram in 4 node cluster. I have 5 shards. My total > document size is about 10G. So I am sure everything is in memory. > > Thanks , > Arjit > > On Wed, Jun 19, 2013 at 8:35 AM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > > > What are your memory settings? > > How many nodes do you have? > > How many shards? > > > > -- > > David 😉 > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > Le 19 juin 2013 à 04:50, Arjit Gupta [arjit292@gmail.com](mailto:arjit292@gmail.com) a écrit : > > > > I have about 20 million documents with book titles. I want to find > > similar titles by applying edit distance on the tittles. > > By the document of fuzzy-query does the same. > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/fuzzy-query/) > > The problem now is search result are very slow. It takes about 3sec to > > search 5-6 letter title in Es with fuzzy-query. > > Is there any other query type I can use which can make it faster and give > > the same/similar results ? > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/Q2vC2N1Gcrg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/Q2vC2N1Gcrg/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/Q2vC2N1Gcrg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/Q2vC2N1Gcrg/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 19, 2013, 10:47am UTC](https://discuss.elastic.co/t/optimise-fuzzy-search/12471/6 "2013-06-19T10:47:44Z")
What does your query look like? Can you reduce a bit the dataset to query on with your use case? I mean filter first on dates for example. Depends on your use case. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 19 juin 2013 à 11:07, Arjit Gupta [arjit292@gmail.com](mailto:arjit292@gmail.com) a écrit : Hi, Yes 16Gb of 32Gb is available on the box. I cant see anything in the logs. Their are no exception in the logs. And I think 2-3 sec is lot of time. Thanks, Arjit Thanks , Arjit On Wed, Jun 19, 2013 at 12:16 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > 16Gb of 32 Gb available per box, right? > Do you see anything in logs? > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 19 juin 2013 à 05:20, Arjit Gupta [arjit292@gmail.com](mailto:arjit292@gmail.com) a écrit : > > What are your memory settings? > > How many nodes do you have? > How many shards? > > I have given about 16G of ram in 4 node cluster. I have 5 shards. My total document size is about 10G. So I am sure everything is in memory. > > Thanks , > Arjit > > On Wed, Jun 19, 2013 at 8:35 AM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > > > What are your memory settings? > > How many nodes do you have? > > How many shards? > > > > -- > > David 😉 > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > Le 19 juin 2013 à 04:50, Arjit Gupta [arjit292@gmail.com](mailto:arjit292@gmail.com) a écrit : > > > > ## I have about 20 million documents with book titles. I want to find similar titles by applying edit distance on the tittles. By the document of fuzzy-query does the same. [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/fuzzy-query/) The problem now is search result are very slow. It takes about 3sec to search 5-6 letter title in Es with fuzzy-query. Is there any other query type I can use which can make it faster and give the same/similar results ? > > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/Q2vC2N1Gcrg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/Q2vC2N1Gcrg/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/Q2vC2N1Gcrg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/Q2vC2N1Gcrg/unsubscribe). > To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:30am UTC](https://discuss.elastic.co/t/optimise-fuzzy-search/12471/7 "2017-07-06T02:30:23Z")