Relevance sorting / JOIN

[Tom_Miller]

I've been using Elasticsearch for a while, but mainly for indexing logging style data.

I'm looking to build a shopping style system, and I have a lot of sales data, and products, and I'd like to sort the products using the underlying sales data.

To give an example, both the sales data and the product data have a brand. i.e - if I buy Nike Sneakers and Adidas sneakers, I'd like the products to be listed based on how much I've spent with each brand - so Nike and Adidas are at the top, and Reebok are at the bottom.

I could do with relatively easily with joins/SQL, but I'm unsure how this could be accomplished using Elasicsearch.

Before I spend days trying to work out the best way to implement this, can someone tell me if this is possible and if so, how it can be achieved?

For reference, I'm using Elastic Cloud, so ES 6.5.

Thanks,

Tom.

[softwaredoug (Doug Turnbull)]

You definitely want to try your best to create just a flat document structure on ingestion from a primary source of truth (like a RDMS). There's nested documents in Elasticsearch, but they're a bit awkward to work with. You definitely want to avoid trying to implement an RDMS in Elastic

Your goal is to try to provide a search-specific view to your content in a flat document structure with the attributes that might matter in relevance ranking.

Maybe this blog article helps https://opensourceconnections.com/blog/2015/05/15/relevance-data-modeling/

-Doug

[Tom_Miller]

Thanks doug. I only mentioned RDMS because I could do a cheap/cheerful that way relatively easily - but it wouldn't scale particularly well or be that fast.

By relevance, I meant relevance to the user - less about search per-se, more about listings/sorting by category.

After doing some research, it doesn't look like i'll be able to do this entirely in ES - perhaps I'll need to work with Spark/Hadoop or something like that to get training data, to return what is essentially recommendation based sorting.

Thanks again.

Tom.