Elastic search and data clustering/grouping

noor · September 5, 2012, 6:16am

Hi,

Me and my team have started a eCommerce project. We are using elasticsearch
to provide quick search for products. Apart from normal searching we also
want our products to be grouped or clustered based on the similarity of
there features. e.g. If a user search for 'Apple' then search result should
show different types of products from apple where each search result record
represents a specific type like iphone, or mac book etc. (Note: the product
features varies from product to product).

My question is that can it be achived from elasticsearch only or we have to
use other projects like mahout or weka for clustering the similar products
from search result. If so then please suggest me how we can integrate
mahout/weka with ealsticsearch.

It would be great for us if anyone who had experience in this area can give
us suggestion to address this problem.

Thanks,

Noor

--

Ivan · September 5, 2012, 5:04pm

Noor,

You can aggregate results using facets, which provides similar
functionality to using GROUP BY in SQL.

For true grouping, also called field collapsing, is [yet] not
supported in Elasticsearch (even Lucene support is not consistent):

github.com/elastic/elasticsearch

Field Collapsing/Combining

opened 05:02PM - 13 Jul 10 UTC

closed 02:08PM - 23 May 14 UTC

ppearcy

Ability to collapse on a field. For example, I want the most relevant result fro…m all different report types. Or similarly, the most recent result of each report type. Or maybe, I want to de-dup on headline. So, the sort order would dictate which one from the group is returned. Similar to what is discussed here: http://blog.jteam.nl/2009/10/20/result-grouping-field-collapsing-with-solr/ From my understanding, it seems that in order for field collapsing to be efficient, the result set must be relatively small. This is also referred to as "Combine" on some other search products.

--

Ivan

On Tue, Sep 4, 2012 at 11:16 PM, Noor nooremailbox@gmail.com wrote:

Hi,

Me and my team have started a eCommerce project. We are using elasticsearch
to provide quick search for products. Apart from normal searching we also
want our products to be grouped or clustered based on the similarity of
there features. e.g. If a user search for 'Apple' then search result should
show different types of products from apple where each search result record
represents a specific type like iphone, or mac book etc. (Note: the product
features varies from product to product).

My question is that can it be achived from elasticsearch only or we have to
use other projects like mahout or weka for clustering the similar products
from search result. If so then please suggest me how we can integrate
mahout/weka with ealsticsearch.

It would be great for us if anyone who had experience in this area can give
us suggestion to address this problem.

Thanks,

Noor

--

--