Re: Full text search vs map reduce

Elastic Stack Elasticsearch
1

Facets are sort of built in custom map reduce operations. With the ability
to specify scripts when you execute them, it gives you quite a flexibility.

A complete generic map reduce is possible, though the benefit of facets is
that they are highly optimized and fast. Currently, I prefer to add more
facets options than to create a more complex (usability wise) generic map
reduce.

On Wed, Aug 10, 2011 at 2:44 PM, ajsie johnny.weng.luu@gmail.com wrote:

I'm coming from CouchDB and wanna get how I would do the same thing with
ES.

It seems that ES's full text search could be compared to CouchDB's map
function since their purpose is to retrieve documents.

However, is there an equivalence to the reduce function in ES?

With CouchDB if I wanna make a calculation based on some record values,
the reduce function allows me to get the value from the database directly.
The database is doing the calculations.

How does document values calculations work in ES? Do I have to retrieve all
records to my server and do the calculation there?

2

I think map/reduce would add a tremendous value, especially if you could register it like you can with percolate, and use it to generate another index. I think both mongodb and couchdb support this.

Some calculations are just too specific for a facet, unless you made the facet so generic that you'd end up pretty close to map/reduce anyways.

As far as the usability is concerned, I feel like it's become a popular enough pattern that most developers working with big data have already used it.

From an implementation standpoint I definitely understand this would be a huge feature... but man the things I could do with it :slight_smile:

-javier

From: elasticsearch@googlegroups.com [elasticsearch@googlegroups.com] on behalf of Shay Banon [kimchy@gmail.com]
Sent: Wednesday, August 10, 2011 7:57 AM
To: elasticsearch@googlegroups.com
Subject: Re: Full text search vs map reduce

Facets are sort of built in custom map reduce operations. With the ability to specify scripts when you execute them, it gives you quite a flexibility.

A complete generic map reduce is possible, though the benefit of facets is that they are highly optimized and fast. Currently, I prefer to add more facets options than to create a more complex (usability wise) generic map reduce.

On Wed, Aug 10, 2011 at 2:44 PM, ajsie <johnny.weng.luu@gmail.commailto:johnny.weng.luu@gmail.com> wrote:
I'm coming from CouchDB and wanna get how I would do the same thing with ES.

It seems that ES's full text search could be compared to CouchDB's map function since their purpose is to retrieve documents.

However, is there an equivalence to the reduce function in ES?

With CouchDB if I wanna make a calculation based on some record values, the reduce function allows me to get the value from the database directly. The database is doing the calculations.

How does document values calculations work in ES? Do I have to retrieve all records to my server and do the calculation there?

3

Agreed, a completely custom "map reduce" facet would be cool. Just by its
nature, and the abstraction for it, it would take longer to execute (i.e. a
custom map reduce that does something similar to terms facets will be
slower), but, it will open up a lot of options...

On Wed, Aug 10, 2011 at 8:51 PM, Javier Muniz javier@granicus.com wrote:

I think map/reduce would add a tremendous value, especially if you could
register it like you can with percolate, and use it to generate another
index. I think both mongodb and couchdb support this.

Some calculations are just too specific for a facet, unless you made the
facet so generic that you'd end up pretty close to map/reduce anyways.

As far as the usability is concerned, I feel like it's become a popular
enough pattern that most developers working with big data have already used
it.

From an implementation standpoint I definitely understand this would be a
huge feature... but man the things I could do with it :slight_smile:

-javier

From: elasticsearch@googlegroups.com [elasticsearch@googlegroups.com] on
behalf of Shay Banon [kimchy@gmail.com]
Sent: Wednesday, August 10, 2011 7:57 AM
To: elasticsearch@googlegroups.com
Subject: Re: Full text search vs map reduce

Facets are sort of built in custom map reduce operations. With the
ability to specify scripts when you execute them, it gives you quite
a flexibility.

A complete generic map reduce is possible, though the benefit of facets
is that they are highly optimized and fast. Currently, I prefer to add more
facets options than to create a more complex (usability wise) generic map
reduce.

On Wed, Aug 10, 2011 at 2:44 PM, ajsie johnny.weng.luu@gmail.com wrote:

I'm coming from CouchDB and wanna get how I would do the same thing with
ES.

It seems that ES's full text search could be compared to CouchDB's map
function since their purpose is to retrieve documents.

However, is there an equivalence to the reduce function in ES?

With CouchDB if I wanna make a calculation based on some record values,
the reduce function allows me to get the value from the database directly.
The database is doing the calculations.

How does document values calculations work in ES? Do I have to retrieve
all records to my server and do the calculation there?