# Document Clustering **URL:** \ **Category:** Elasticsearch\ **Created:** [November 21, 2011, 10:14pm UTC](https://discuss.elastic.co/t/document-clustering/5934 "2011-11-21T22:14:59Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Michael\_Shapiro](https://avatars.discourse-cdn.com/v4/letter/m/6bbea6/32.png) [@Michael\_Shapiro](https://discuss.elastic.co/u/Michael_Shapiro)\ **Post date:** [November 21, 2011, 10:14pm UTC](https://discuss.elastic.co/t/document-clustering/5934/1 "2011-11-21T22:14:59Z")
Hi Folks, I've been mulling over ES' docs in order to determine if it'd fit my document clustering needs. It doesn't particularly look like it, but I just wanted to throw out the question in case I'm missing something. I've got a large number of documents, many of them are quite similar and I'd like to come up with a list of documents that are effectively "unique". It looks like I could possibly do this with MLT queries, but I'm not sure if I'd be trying to stuff a round peg into a square hole. Any thoughts would be greatly appreciated! --Mike ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [November 22, 2011, 5:27am UTC](https://discuss.elastic.co/t/document-clustering/5934/2 "2011-11-22T05:27:59Z")
Hi Mike, For clustering of search results, see Carrot2 project. You may also want to see how Carrot2 is integrated with Solr. For off-line, batch document clustering, see Apache Mahout. If you have document that are very, very similar, off by a few characters or words, and want to dedupe them, you could have a look at how it's done in Solr: [http://wiki.apache.org/solr/Deduplication](http://wiki.apache.org/solr/Deduplication) ## Otis Sematext is hiring -- [Jobs](http://sematext.com/about/jobs.html) On Nov 21, 5:14 pm, Michael Shapiro [koude...@gmail.com](mailto:koude...@gmail.com) wrote: > Hi Folks, > > I've been mulling over ES' docs in order to determine if it'd fit my > document clustering needs. It doesn't particularly look like it, but I > just wanted to throw out the question in case I'm missing something. > > I've got a large number of documents, many of them are quite similar > and I'd like to come up with a list of documents that are effectively > "unique". It looks like I could possibly do this with MLT queries, but > I'm not sure if I'd be trying to stuff a round peg into a square hole. > > Any thoughts would be greatly appreciated! > > --Mike ---
**Author:** ![Karussell1](https://avatars.discourse-cdn.com/v4/letter/k/50afbb/32.png) [@Karussell1](https://discuss.elastic.co/u/Karussell1)\ **Post date:** [November 23, 2011, 8:02am UTC](https://discuss.elastic.co/t/document-clustering/5934/3 "2011-11-23T08:02:51Z")
yes, mlt or "fuzzy like this" could be an option (but I'm using a customized one\*\*). also have a look at the project otis mentioned. Peter. \*\* > On 21 Nov., 23:14, Michael Shapiro [koude...@gmail.com](mailto:koude...@gmail.com) wrote: > Hi Folks, > > I've been mulling over ES' docs in order to determine if it'd fit my > document clustering needs. It doesn't particularly look like it, but I > just wanted to throw out the question in case I'm missing something. > > I've got a large number of documents, many of them are quite similar > and I'd like to come up with a list of documents that are effectively > "unique". It looks like I could possibly do this with MLT queries, but > I'm not sure if I'd be trying to stuff a round peg into a square hole. > > Any thoughts would be greatly appreciated! > > --Mike ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:47am UTC](https://discuss.elastic.co/t/document-clustering/5934/4 "2017-07-06T03:47:43Z")