# Best way to index documents for retrieving clustered result **URL:** \ **Category:** Elasticsearch\ **Created:** [April 30, 2012, 9:25am UTC](https://discuss.elastic.co/t/best-way-to-index-documents-for-retrieving-clustered-result/7511 "2012-04-30T09:25:31Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![mp2893](https://avatars.discourse-cdn.com/v4/letter/m/d9b06d/32.png) [@mp2893](https://discuss.elastic.co/u/mp2893)\ **Post date:** [April 30, 2012, 9:25am UTC](https://discuss.elastic.co/t/best-way-to-index-documents-for-retrieving-clustered-result/7511/1 "2012-04-30T09:25:31Z")
Hi, I have millions of news articles. They are clustered based on the similarity of their topic/content All news documents are clustered on daily basis. So each cluster has news documents from the same day. . Each news document has a field named "newsid" that represents to which news cluster it belongs to. So basically I have one index, named "news", and one mapping named "document". What I want to do is retrieve news clusters with given text queries and date range queries. For example, "Get news clusters that are related to 'Barack Obama' from 2012-04-01 to 2012-04-30. The clusters must have more than 5 documents as the member of the cluster (namely, the cluster size should be bigger than 5)" Note that I do not want the documents themselves. I want the clusters, namely "newsid". As far as I know, there are a couple of ways to achieve this goal. I know that parent-child mapping can do this. Faceting seems to be another option. I heard that nested mapping also helps but I'm not sure about this. Currently my "document" mapping has the following fields: docid, date, title content, newsid, count, url "count" represents how many members the "newsid" cluster has. I have additional mapping named "cluster" which is the parent mapping of "document". "cluster" mapping has the following fields: count, date, label "count" field is the same as above. 'label" field is the label of the cluster. I currently do a "has\_child" query to achieve my goal. It works fine, but doesn't seem be quick enough (it takes several seconds to retrieve entries from "cluster" mapping) Is my configuration the most optimized one? Should I be using faceted search instead of using two mappings? (the "label" field of "cluster" mapping can be integrated into "document" mapping so that won't be a problem) I would like some assessment plz. Thanks in advance. Ed ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [April 30, 2012, 10:28am UTC](https://discuss.elastic.co/t/best-way-to-index-documents-for-retrieving-clustered-result/7511/2 "2012-04-30T10:28:45Z")
Hiya > So basically I have one index, named "news", and one mapping named > "document". > What I want to do is retrieve news clusters with given text queries > and date range queries. > For example, > "Get news clusters that are related to 'Barack Obama' from 2012-04-01 > to 2012-04-30. The clusters must have more than 5 documents as the > member of the cluster (namely, the cluster size should be bigger than > 5)" > Note that I do not want the documents themselves. I want the clusters, > namely "newsid". It sounds like facets are your best bet, at least for the query you describe above. Try this: curl -XGET '[http://127.0.0.1:9200/news/document/\_search?pretty=1](http://127.0.0.1:9200/news/document/_search?pretty=1)' -d ' { "query" : { "filtered" : { "filter" : { "range" : { "published" : { "lte" : "2012-04-30", "gte" : "2012-04-01" } } }, "query" : { "text" : { "content" : "barack obama" } } } }, "facets" : { "clusters" : { "terms" : { "field" : "newsid" } } }, "size" : 0 } ' clint ---
**Author:** ![mp2893](https://avatars.discourse-cdn.com/v4/letter/m/d9b06d/32.png) [@mp2893](https://discuss.elastic.co/u/mp2893)\ **Post date:** [May 1, 2012, 5:41am UTC](https://discuss.elastic.co/t/best-way-to-index-documents-for-retrieving-clustered-result/7511/3 "2012-05-01T05:41:38Z")
Thanks for the reply Clinton!! I'd try your query format and post results on this thread. Ed 2012/4/30 Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com) > Hiya > > > So basically I have one index, named "news", and one mapping named > > "document". > > What I want to do is retrieve news clusters with given text queries > > and date range queries. > > For example, > > "Get news clusters that are related to 'Barack Obama' from 2012-04-01 > > to 2012-04-30. The clusters must have more than 5 documents as the > > member of the cluster (namely, the cluster size should be bigger than > > 5)" > > Note that I do not want the documents themselves. I want the clusters, > > namely "newsid". > > It sounds like facets are your best bet, at least for the query you > describe above. > > Try this: > > curl -XGET '[http://127.0.0.1:9200/news/document/\_search?pretty=1](http://127.0.0.1:9200/news/document/_search?pretty=1)' -d ' > { > "query" : { > "filtered" : { > "filter" : { > "range" : { > "published" : { > "lte" : "2012-04-30", > "gte" : "2012-04-01" > } > } > }, > "query" : { > "text" : { > "content" : "barack obama" > } > } > } > }, > "facets" : { > "clusters" : { > "terms" : { > "field" : "newsid" > } > } > }, > "size" : 0 > } > ' > > clint ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:30am UTC](https://discuss.elastic.co/t/best-way-to-index-documents-for-retrieving-clustered-result/7511/4 "2017-07-06T03:30:30Z")
Closed on July 6, 2017, 3:30am UTC