ElasticSearch as a searchable cache


(Franz Allan Valencia See) #1

Good day,

I'm planning to use ElasticSearch as a horizontally scalable cache that I
can search data from (because querying data from our RDBMs even with
indexing and pagination is too slow). That is, I will query data from our
RDBMs, (optionally do some processing on those data,) then store it in
ElasticSearch. So that when I do a search, I just search against
ElasticSearch. And when I do an update, I update both RDBMs and
ElasticSearch.

Is my plan on how to use ElasticSearch correct, or I am totally wrong on
ElasticSearch's usage? ..or is there a better tool for that?

Note: Our application is on java.

Thanks,

--
Franz Allan Valencia See | Java Software Engineer
franz.see@gmail.com
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see


(Shay Banon) #2

Sounds good to me, especially since you get some features out of
elasticsearch that you can't use in the database.

-shay.banon

On Mon, Jun 28, 2010 at 11:52 AM, Franz Allan Valencia See <
franz.see@gmail.com> wrote:

Good day,

I'm planning to use ElasticSearch as a horizontally scalable cache that I
can search data from (because querying data from our RDBMs even with
indexing and pagination is too slow). That is, I will query data from our
RDBMs, (optionally do some processing on those data,) then store it in
ElasticSearch. So that when I do a search, I just search against
ElasticSearch. And when I do an update, I update both RDBMs and
ElasticSearch.

Is my plan on how to use ElasticSearch correct, or I am totally wrong on
ElasticSearch's usage? ..or is there a better tool for that?

Note: Our application is on java.

Thanks,

--
Franz Allan Valencia See | Java Software Engineer
franz.see@gmail.com
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see


(Franz Allan Valencia See) #3

Thanks :slight_smile:

Follow-up question:
From an RDBMs, to retrieve/search data from multiple table, we use a join.
How about for elastic search? Are these data 'denormalized' to documents or
can ElasticSearch's index + type mimic joining a table?

Pardon for the very noob question.

Thanks,

Franz Allan Valencia See | Java Software Engineer
franz.see@gmail.com
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see

On Mon, Jun 28, 2010 at 6:43 PM, Shay Banon shay.banon@elasticsearch.comwrote:

Sounds good to me, especially since you get some features out of
elasticsearch that you can't use in the database.

-shay.banon

On Mon, Jun 28, 2010 at 11:52 AM, Franz Allan Valencia See <
franz.see@gmail.com> wrote:

Good day,

I'm planning to use ElasticSearch as a horizontally scalable cache that I
can search data from (because querying data from our RDBMs even with
indexing and pagination is too slow). That is, I will query data from our
RDBMs, (optionally do some processing on those data,) then store it in
ElasticSearch. So that when I do a search, I just search against
ElasticSearch. And when I do an update, I update both RDBMs and
ElasticSearch.

Is my plan on how to use ElasticSearch correct, or I am totally wrong on
ElasticSearch's usage? ..or is there a better tool for that?

Note: Our application is on java.

Thanks,

--
Franz Allan Valencia See | Java Software Engineer
franz.see@gmail.com
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see


(Shay Banon) #4

You will have to denormalize you data into self contained/searchable
documents. Though there is some interesting work being done at creating
parent / child doc relationship, but its at a very very early stage.

-shay.banon

On Tue, Jun 29, 2010 at 2:06 AM, Franz Allan Valencia See <
franz.see@gmail.com> wrote:

Thanks :slight_smile:

Follow-up question:
From an RDBMs, to retrieve/search data from multiple table, we use a join.
How about for elastic search? Are these data 'denormalized' to documents or
can ElasticSearch's index + type mimic joining a table?

Pardon for the very noob question.

Thanks,

Franz Allan Valencia See | Java Software Engineer
franz.see@gmail.com
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see

On Mon, Jun 28, 2010 at 6:43 PM, Shay Banon shay.banon@elasticsearch.comwrote:

Sounds good to me, especially since you get some features out of
elasticsearch that you can't use in the database.

-shay.banon

On Mon, Jun 28, 2010 at 11:52 AM, Franz Allan Valencia See <
franz.see@gmail.com> wrote:

Good day,

I'm planning to use ElasticSearch as a horizontally scalable cache that I
can search data from (because querying data from our RDBMs even with
indexing and pagination is too slow). That is, I will query data from our
RDBMs, (optionally do some processing on those data,) then store it in
ElasticSearch. So that when I do a search, I just search against
ElasticSearch. And when I do an update, I update both RDBMs and
ElasticSearch.

Is my plan on how to use ElasticSearch correct, or I am totally wrong on
ElasticSearch's usage? ..or is there a better tool for that?

Note: Our application is on java.

Thanks,

--
Franz Allan Valencia See | Java Software Engineer
franz.see@gmail.com
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see


(Franz Allan Valencia See) #5

K, thanks :slight_smile:

On Tue, Jun 29, 2010 at 4:46 PM, Shay Banon shay.banon@elasticsearch.comwrote:

You will have to denormalize you data into self contained/searchable
documents. Though there is some interesting work being done at creating
parent / child doc relationship, but its at a very very early stage.

-shay.banon

On Tue, Jun 29, 2010 at 2:06 AM, Franz Allan Valencia See <
franz.see@gmail.com> wrote:

Thanks :slight_smile:

Follow-up question:
From an RDBMs, to retrieve/search data from multiple table, we use a join.
How about for elastic search? Are these data 'denormalized' to documents or
can ElasticSearch's index + type mimic joining a table?

Pardon for the very noob question.

Thanks,

Franz Allan Valencia See | Java Software Engineer
franz.see@gmail.com
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see

On Mon, Jun 28, 2010 at 6:43 PM, Shay Banon <shay.banon@elasticsearch.com

wrote:

Sounds good to me, especially since you get some features out of
elasticsearch that you can't use in the database.

-shay.banon

On Mon, Jun 28, 2010 at 11:52 AM, Franz Allan Valencia See <
franz.see@gmail.com> wrote:

Good day,

I'm planning to use ElasticSearch as a horizontally scalable cache that
I can search data from (because querying data from our RDBMs even with
indexing and pagination is too slow). That is, I will query data from our
RDBMs, (optionally do some processing on those data,) then store it in
ElasticSearch. So that when I do a search, I just search against
ElasticSearch. And when I do an update, I update both RDBMs and
ElasticSearch.

Is my plan on how to use ElasticSearch correct, or I am totally wrong on
ElasticSearch's usage? ..or is there a better tool for that?

Note: Our application is on java.

Thanks,

--
Franz Allan Valencia See | Java Software Engineer
franz.see@gmail.com
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see

--
Franz Allan Valencia See | Java Software Engineer
franz.see@gmail.com
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see


(Martijn Laarman) #6

@Shay could you elaborate some more on the work in progress on parent /
child relationships ?

Especially in relation to the ticket I created here:
http://github.com/elasticsearch/elasticsearch/issues#issue/211 ?

Thanks in advance!

On Tue, Jun 29, 2010 at 9:46 AM, Shay Banon shay.banon@elasticsearch.comwrote:

You will have to denormalize you data into self contained/searchable
documents. Though there is some interesting work being done at creating
parent / child doc relationship, but its at a very very early stage.

-shay.banon

On Tue, Jun 29, 2010 at 2:06 AM, Franz Allan Valencia See <
franz.see@gmail.com> wrote:

Thanks :slight_smile:

Follow-up question:
From an RDBMs, to retrieve/search data from multiple table, we use a join.
How about for elastic search? Are these data 'denormalized' to documents or
can ElasticSearch's index + type mimic joining a table?

Pardon for the very noob question.

Thanks,

Franz Allan Valencia See | Java Software Engineer
franz.see@gmail.com
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see

On Mon, Jun 28, 2010 at 6:43 PM, Shay Banon <shay.banon@elasticsearch.com

wrote:

Sounds good to me, especially since you get some features out of
elasticsearch that you can't use in the database.

-shay.banon

On Mon, Jun 28, 2010 at 11:52 AM, Franz Allan Valencia See <
franz.see@gmail.com> wrote:

Good day,

I'm planning to use ElasticSearch as a horizontally scalable cache that
I can search data from (because querying data from our RDBMs even with
indexing and pagination is too slow). That is, I will query data from our
RDBMs, (optionally do some processing on those data,) then store it in
ElasticSearch. So that when I do a search, I just search against
ElasticSearch. And when I do an update, I update both RDBMs and
ElasticSearch.

Is my plan on how to use ElasticSearch correct, or I am totally wrong on
ElasticSearch's usage? ..or is there a better tool for that?

Note: Our application is on java.

Thanks,

--
Franz Allan Valencia See | Java Software Engineer
franz.see@gmail.com
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see


(Shay Banon) #7

Same answer I gave in the issue, there is some work in Lucene to support
this, but its far from something that can be used properly. I am not
tackling it on elasticsearch level (with or without native Lucene support)
anytime soon.

-shay.banon

On Wed, Jun 30, 2010 at 11:28 AM, Martijn Laarman m.laarman@datheon.comwrote:

@Shay could you elaborate some more on the work in progress on parent /
child relationships ?

Especially in relation to the ticket I created here:
http://github.com/elasticsearch/elasticsearch/issues#issue/211 ?

Thanks in advance!

On Tue, Jun 29, 2010 at 9:46 AM, Shay Banon shay.banon@elasticsearch.comwrote:

You will have to denormalize you data into self contained/searchable
documents. Though there is some interesting work being done at creating
parent / child doc relationship, but its at a very very early stage.

-shay.banon

On Tue, Jun 29, 2010 at 2:06 AM, Franz Allan Valencia See <
franz.see@gmail.com> wrote:

Thanks :slight_smile:

Follow-up question:
From an RDBMs, to retrieve/search data from multiple table, we use a
join. How about for elastic search? Are these data 'denormalized' to
documents or can ElasticSearch's index + type mimic joining a table?

Pardon for the very noob question.

Thanks,

Franz Allan Valencia See | Java Software Engineer
franz.see@gmail.com
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see

On Mon, Jun 28, 2010 at 6:43 PM, Shay Banon <
shay.banon@elasticsearch.com> wrote:

Sounds good to me, especially since you get some features out of
elasticsearch that you can't use in the database.

-shay.banon

On Mon, Jun 28, 2010 at 11:52 AM, Franz Allan Valencia See <
franz.see@gmail.com> wrote:

Good day,

I'm planning to use ElasticSearch as a horizontally scalable cache that
I can search data from (because querying data from our RDBMs even with
indexing and pagination is too slow). That is, I will query data from our
RDBMs, (optionally do some processing on those data,) then store it in
ElasticSearch. So that when I do a search, I just search against
ElasticSearch. And when I do an update, I update both RDBMs and
ElasticSearch.

Is my plan on how to use ElasticSearch correct, or I am totally wrong
on ElasticSearch's usage? ..or is there a better tool for that?

Note: Our application is on java.

Thanks,

--
Franz Allan Valencia See | Java Software Engineer
franz.see@gmail.com
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see


(system) #8