Relational Data Modelling in Elasticsearch

Elastic Stack Elasticsearch
1

I'm indexing data from a relational database to Elasticsearch which
requires
several joins across multiple tables.

Now, am thinking if using Nested object to model my relationship would be a
good idea.

Basically, I want to create separate batch jobs for each related tables to
push data
from the database to ES using bulk UPSERT.

i.e if the document id exist, partially update the child (nested) document
otherwise, creates a new document, set the id and partially update
the child (nested) document.

Would there be a big performance hit if I have 5 jobs doing partial updates
on each document (x 1+ million documents) on my index?

My data rarely change once it's indexed and so far I've not seen any
disadvantage of using nested objects.

Please advise.

Thanks,

  • Phil

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fae0f08c-511a-4274-95f2-a58fe2df63e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Just cluster it if you need to have a better performance out of your index
update bulk upsert runs...

On Thursday, November 13, 2014 2:34:09 PM UTC-5, Philip K. Adetiloye wrote:

I'm indexing data from a relational database to Elasticsearch which
requires
several joins across multiple tables.

Now, am thinking if using Nested object to model my relationship would be
a good idea.

Basically, I want to create separate batch jobs for each related tables to
push data
from the database to ES using bulk UPSERT.

i.e if the document id exist, partially update the child (nested)
document otherwise, creates a new document, set the id and partially
update
the child (nested) document.

Would there be a big performance hit if I have 5 jobs doing partial
updates on each document (x 1+ million documents) on my index?

My data rarely change once it's indexed and so far I've not seen any
disadvantage of using nested objects.

Please advise.

Thanks,

  • Phil

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f2afba1c-d14a-4591-b750-f0e1ece7b814%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.