# Similar documentation detection System **URL:** \ **Category:** Elasticsearch\ **Created:** [September 29, 2011, 6:31am UTC](https://discuss.elastic.co/t/similar-documentation-detection-system/5473 "2011-09-29T06:31:49Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![vineeth\_mohan](https://avatars.discourse-cdn.com/v4/letter/v/bc79bd/32.png) [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\ **Post date:** [September 29, 2011, 6:31am UTC](https://discuss.elastic.co/t/similar-documentation-detection-system/5473/1 "2011-09-29T06:31:49Z")
Hi , Is there a feature in ES where when i give a document to elasticSearch , it tells me which all documents in the ES are similar to the document i inserted. Like instance let there be a score for similarity checking between documents (between 1 to 100). Whenever i add a document to ES , it should tell me which all documents have similarity of score more than 70. Also i want ES to look into similarity of last N days only (Not the whole lot if it takes too much of processing power). Thanks Vineeth ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [September 29, 2011, 7:25am UTC](https://discuss.elastic.co/t/similar-documentation-detection-system/5473/2 "2011-09-29T07:25:47Z")
Hi Vineeth > Is there a feature in ES where when i give a document to > elasticSearch , it tells me which all documents in the ES are similar > to the document i inserted. > Like instance let there be a score for similarity checking between > documents (between 1 to 100). Look at: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. > Whenever i add a document to ES , it should tell me which all > documents have similarity of score more than 70. > Also i want ES to look into similarity of last N days only (Not the > whole lot if it takes too much of processing power). > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. And combing queries with one or more filters: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. clint ---
**Author:** ![Karussell1](https://avatars.discourse-cdn.com/v4/letter/k/50afbb/32.png) [@Karussell1](https://discuss.elastic.co/u/Karussell1)\ **Post date:** [September 29, 2011, 7:26am UTC](https://discuss.elastic.co/t/similar-documentation-detection-system/5473/3 "2011-09-29T07:26:59Z")
You should have a look into the more/fuzzy like this query: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. and also [http://www.phpriot.com/news/duplicates-detection-with-elasticsearch](http://www.phpriot.com/news/duplicates-detection-with-elasticsearch) But to speed up these think you probably would have to use one of the local sensitive hashing algorithm On 29 Sep., 08:31, Vineeth Mohan [vineethmo...@algotree.com](mailto:vineethmo...@algotree.com) wrote: > Hi , > > Is there a feature in ES where when i give a document to elasticSearch , it > tells me which all documents in the ES are similar to the document i > inserted. > Like instance let there be a score for similarity checking between documents > (between 1 to 100). > > Whenever i add a document to ES , it should tell me which all documents have > similarity of score more than 70. > Also i want ES to look into similarity of last N days only (Not the whole > lot if it takes too much of processing power). > > Thanks > Vineeth ---
**Author:** ![vineeth\_mohan](https://avatars.discourse-cdn.com/v4/letter/v/bc79bd/32.png) [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\ **Post date:** [September 29, 2011, 9:32am UTC](https://discuss.elastic.co/t/similar-documentation-detection-system/5473/4 "2011-09-29T09:32:02Z")
Thanks guyz ... Elastic search simply rockz..... Thanks Vineeth On Thu, Sep 29, 2011 at 12:56 PM, Karussell [tableyourtime@googlemail.com](mailto:tableyourtime@googlemail.com)wrote: > You should have a look into the more/fuzzy like this query: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/mlt-query.html) > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/flt-query.html) > > and also > > [phpBBservice.nl - Forumoverzicht](http://www.phpriot.com/news/duplicates-detection-with-elasticsearch) > > But to speed up these think you probably would have to use one of the > local sensitive hashing algorithm > > On 29 Sep., 08:31, Vineeth Mohan [vineethmo...@algotree.com](mailto:vineethmo...@algotree.com) wrote: > > > Hi , > > > > Is there a feature in ES where when i give a document to elasticSearch , > > it > > tells me which all documents in the ES are similar to the document i > > inserted. > > Like instance let there be a score for similarity checking between > > documents > > (between 1 to 100). > > > > Whenever i add a document to ES , it should tell me which all documents > > have > > similarity of score more than 70. > > Also i want ES to look into similarity of last N days only (Not the whole > > lot if it takes too much of processing power). > > > > Thanks > > Vineeth ---
**Author:** ![vineeth\_mohan](https://avatars.discourse-cdn.com/v4/letter/v/bc79bd/32.png) [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\ **Post date:** [September 29, 2011, 9:33am UTC](https://discuss.elastic.co/t/similar-documentation-detection-system/5473/5 "2011-09-29T09:33:34Z")
Just one more questions. Is it possible to do the same without actually inserting the document ? I need to decided if the document needs to be inserted looking at if there are duplicates for it. Thanks Vineeth On Thu, Sep 29, 2011 at 3:02 PM, Vineeth Mohan [vineethmohan@algotree.com](mailto:vineethmohan@algotree.com)wrote: > Thanks guyz ... > Elastic search simply rockz..... > > Thanks > Vineeth > > On Thu, Sep 29, 2011 at 12:56 PM, Karussell [tableyourtime@googlemail.com](mailto:tableyourtime@googlemail.com)wrote: > > > You should have a look into the more/fuzzy like this query: > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/mlt-query.html) > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/flt-query.html) > > > > and also > > > > [phpBBservice.nl - Forumoverzicht](http://www.phpriot.com/news/duplicates-detection-with-elasticsearch) > > > > But to speed up these think you probably would have to use one of the > > local sensitive hashing algorithm > > > > On 29 Sep., 08:31, Vineeth Mohan [vineethmo...@algotree.com](mailto:vineethmo...@algotree.com) wrote: > > > > > Hi , > > > > > > Is there a feature in ES where when i give a document to elasticSearch , > > > it > > > tells me which all documents in the ES are similar to the document i > > > inserted. > > > Like instance let there be a score for similarity checking between > > > documents > > > (between 1 to 100). > > > > > > Whenever i add a document to ES , it should tell me which all documents > > > have > > > similarity of score more than 70. > > > Also i want ES to look into similarity of last N days only (Not the > > > whole > > > lot if it takes too much of processing power). > > > > > > Thanks > > > Vineeth ---
**Author:** ![Karussell1](https://avatars.discourse-cdn.com/v4/letter/k/50afbb/32.png) [@Karussell1](https://discuss.elastic.co/u/Karussell1)\ **Post date:** [September 29, 2011, 1:31pm UTC](https://discuss.elastic.co/t/similar-documentation-detection-system/5473/6 "2011-09-29T13:31:16Z")
yes, just query for it via mlt or flt this is not indexing ... On 29 Sep., 11:33, Vineeth Mohan [vineethmo...@algotree.com](mailto:vineethmo...@algotree.com) wrote: > Just one more questions. > > Is it possible to do the same without actually inserting the document ? > I need to decided if the document needs to be inserted looking at if there > are duplicates for it. > > Thanks > Vineeth > > On Thu, Sep 29, 2011 at 3:02 PM, Vineeth Mohan [vineethmo...@algotree.com](mailto:vineethmo...@algotree.com)wrote: > > > Thanks guyz ... > > Elastic search simply rockz..... > > > Thanks > > Vineeth > > > On Thu, Sep 29, 2011 at 12:56 PM, Karussell [tableyourt...@googlemail.com](mailto:tableyourt...@googlemail.com)wrote: > > > > You should have a look into the more/fuzzy like this query: > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/mlt-query.html) > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/flt-query.html) > > > > and also > > > > [phpBBservice.nl - Forumoverzicht](http://www.phpriot.com/news/duplicates-detection-with-elasticsearch) > > > > But to speed up these think you probably would have to use one of the > > > local sensitive hashing algorithm > > > > On 29 Sep., 08:31, Vineeth Mohan [vineethmo...@algotree.com](mailto:vineethmo...@algotree.com) wrote: > > > > > > > Hi , > > > > > Is there a feature in ES where when i give a document to elasticSearch , > > > > it > > > > tells me which all documents in the ES are similar to the document i > > > > inserted. > > > > Like instance let there be a score for similarity checking between > > > > documents > > > > (between 1 to 100). > > > > > Whenever i add a document to ES , it should tell me which all documents > > > > have > > > > similarity of score more than 70. > > > > Also i want ES to look into similarity of last N days only (Not the > > > > whole > > > > lot if it takes too much of processing power). > > > > > Thanks > > > > Vineeth ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:53am UTC](https://discuss.elastic.co/t/similar-documentation-detection-system/5473/7 "2017-07-06T03:53:21Z")