# Search for similar documents **URL:** \ **Category:** Elasticsearch\ **Created:** [July 5, 2011, 12:47pm UTC](https://discuss.elastic.co/t/search-for-similar-documents/4778 "2011-07-05T12:47:36Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![konstantin](https://avatars.discourse-cdn.com/v4/letter/k/858c86/32.png) [@konstantin](https://discuss.elastic.co/u/konstantin)\ **Post date:** [July 5, 2011, 12:47pm UTC](https://discuss.elastic.co/t/search-for-similar-documents/4778/1 "2011-07-05T12:47:36Z")
Hi guys, I have a task. Given document I have to find the set of similar documents. The document has title and content fields. I'd like to use some kind of cosine similarity. My current approach is to represent input document as boolean query constructed for each term with OR conjunction. So there are some shortcomings 1. query is too large (thus I expect reduce in performance) 2. I have to use query\_string query type so I need to use my own query parser (I merge all terms from all the fields in one query and boost the terms that belongs to the title) My questions are what is the best way to solve this task? Is the elastic search/lucene good for that kind of searching? I would be much obliged, -Kostya ps what is "MoreLikeThis" function? Are there any description how it works? ---
**Author:** ![Stefan\_Matheis](https://avatars.discourse-cdn.com/v4/letter/s/977dab/32.png) [@Stefan\_Matheis](https://discuss.elastic.co/u/Stefan_Matheis)\ **Post date:** [July 5, 2011, 12:52pm UTC](https://discuss.elastic.co/t/search-for-similar-documents/4778/2 "2011-07-05T12:52:05Z")
Kostya, what about this > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. and this [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/mlt-query.html) ? Regards Stefan On Tue, Jul 5, 2011 at 2:47 PM, konstantin [konstantin.selivanov@gmail.com](mailto:konstantin.selivanov@gmail.com) wrote: > Hi guys, > > I have a task. Given document I have to find the set of similar > documents. > The document has title and content fields. I'd like to use some kind > of cosine similarity. > > My current approach is to represent input document as boolean query > constructed for each term with OR conjunction. > > So there are some shortcomings > > 1. query is too large (thus I expect reduce in performance) > 2. I have to use query\_string query type so I need to use my own query > parser (I merge all terms from all the fields in one query and boost > the terms that belongs to the title) > > My questions are what is the best way to solve this task? Is the > Elasticsearch/lucene good for that kind of searching? > > I would be much obliged, > -Kostya > > ps what is "MoreLikeThis" function? Are there any description how it > works? ---
**Author:** ![Karussell1](https://avatars.discourse-cdn.com/v4/letter/k/50afbb/32.png) [@Karussell1](https://discuss.elastic.co/u/Karussell1)\ **Post date:** [July 5, 2011, 4:42pm UTC](https://discuss.elastic.co/t/search-for-similar-documents/4778/3 "2011-07-05T16:42:00Z")
also have a look into flt: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. > **[Duplicates Detection with ElasticSearch » Andrei Zmievski](http://zmievski.org/2011/03/duplicates-detection-with-elasticsearch/)** > > Life, technology, and other good things and > On Jul 5, 2:52 pm, Stefan Matheis [matheis.ste...@googlemail.com](mailto:matheis.ste...@googlemail.com) wrote: > Kostya, > > what about thishttp://www.elasticsearch.org/guide/reference/api/more-like-this.html > and thishttp://www.elasticsearch.org/guide/reference/query-dsl/mlt-query.html > ? > > Regards > Stefan > > On Tue, Jul 5, 2011 at 2:47 PM, konstantin > > [konstantin.seliva...@gmail.com](mailto:konstantin.seliva...@gmail.com) wrote: > > > Hi guys, > > > I have a task. Given document I have to find the set of similar > > documents. > > The document has title and content fields. I'd like to use some kind > > of cosine similarity. > > > My current approach is to represent input document as boolean query > > constructed for each term with OR conjunction. > > > So there are some shortcomings > > > > 1. query is too large (thus I expect reduce in performance) > > 2. I have to use query\_string query type so I need to use my own query > > parser (I merge all terms from all the fields in one query and boost > > the terms that belongs to the title) > > > My questions are what is the best way to solve this task? Is the > > Elasticsearch/lucene good for that kind of searching? > > > I would be much obliged, > > -Kostya > > > ps what is "MoreLikeThis" function? Are there any description how it > > works? ---
**Author:** ![konstantin](https://avatars.discourse-cdn.com/v4/letter/k/858c86/32.png) [@konstantin](https://discuss.elastic.co/u/konstantin)\ **Post date:** [July 6, 2011, 11:19am UTC](https://discuss.elastic.co/t/search-for-similar-documents/4778/4 "2011-07-06T11:19:37Z")
Thanks for the links! btw I found the article that describes how mlt works: [http://cephas.net/blog/2008/03/30/how-morelikethis-works-in-lucene/](http://cephas.net/blog/2008/03/30/how-morelikethis-works-in-lucene/) So, I guess mlt is a possible solution for me. I could use fuzzy-like-this as well but it seems that I will get a lot of false positives. Thanks. On Jul 5, 8:42 pm, Karussell [tableyourt...@googlemail.com](mailto:tableyourt...@googlemail.com) wrote: > also have a look into flt: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/flt-field-quer)... > > [Duplicates Detection with ElasticSearch » Andrei Zmievski](http://zmievski.org/2011/03/duplicates-detection-with-elasticsearch) > > and > > [https://github.com/karussell/Jetwick/blob/master/src/main/java/de/jet](https://github.com/karussell/Jetwick/blob/master/src/main/java/de/jet)... > > On Jul 5, 2:52 pm, Stefan Matheis [matheis.ste...@googlemail.com](mailto:matheis.ste...@googlemail.com) > wrote: > > > Kostya, > > > what about thishttp://www.elasticsearch.org/guide/reference/api/more-like-this.html > > and thishttp://www.elasticsearch.org/guide/reference/query-dsl/mlt-query.html > > ? > > > Regards > > Stefan > > > On Tue, Jul 5, 2011 at 2:47 PM, konstantin > > > [konstantin.seliva...@gmail.com](mailto:konstantin.seliva...@gmail.com) wrote: > > > > > Hi guys, > > > > I have a task. Given document I have to find the set of similar > > > documents. > > > The document has title and content fields. I'd like to use some kind > > > of cosine similarity. > > > > My current approach is to represent input document as boolean query > > > constructed for each term with OR conjunction. > > > > So there are some shortcomings > > > > > > 1. query is too large (thus I expect reduce in performance) > > > 2. I have to use query\_string query type so I need to use my own query > > > parser (I merge all terms from all the fields in one query and boost > > > the terms that belongs to the title) > > > > My questions are what is the best way to solve this task? Is the > > > Elasticsearch/lucene good for that kind of searching? > > > > I would be much obliged, > > > -Kostya > > > > ps what is "MoreLikeThis" function? Are there any description how it > > > works? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:01am UTC](https://discuss.elastic.co/t/search-for-similar-documents/4778/5 "2017-07-06T04:01:38Z")