Let's suppose that I have two tables in my project's DB (today it's SQL
Server) - Documents and Tasks. It was decided to store every data inserted
into the DB in ElasticSearch.
I am beginning to learn ES and I'm wondering how many indices should I have
one for all data or one index for every DB entity (Documents index and
Tasks index).
Hi Igor,
the data design phase is really important. And it's mainly driven by how
you are going to use your data. For instance, are you going to get
documents by id? Or run full-text searches? Where are the fields that you
want to search on, in the documents or in the tasks? And how do you want to
get results back? Is there a relation between the two tables?
Those are some of the questions that you'd need to ask yourself to decide
how to structure your data. If you can add some details we'll try and help
you with this.
Cheers
Luca
On Tuesday, September 17, 2013 12:18:51 PM UTC+2, igo...@tomax.co.il wrote:
Let's suppose that I have two tables in my project's DB (today it's SQL
Server) - Documents and Tasks. It was decided to store every data inserted
into the DB in Elasticsearch.
I am beginning to learn ES and I'm wondering how many indices should I
have - one for all data or one index for every DB entity (Documents index
and Tasks index).
Are you going to get documents by id? "no" (today, but who knows what
we'll want tomorrow?)
Run full-text searches? Of course!
Where are the fields that you want to search on, in the documents or in
the tasks? (In both)
How do you want to get results back? I'd prefer to get them as list of
objects defined in our project.
Is there a relation between the two tables? Sure. Every task has one or
more documents.
Thank you in advance,
Igor
On Tuesday, September 17, 2013 1:47:18 PM UTC+3, Luca Cavanna wrote:
Hi Igor,
the data design phase is really important. And it's mainly driven by how
you are going to use your data. For instance, are you going to get
documents by id? Or run full-text searches? Where are the fields that you
want to search on, in the documents or in the tasks? And how do you want to
get results back? Is there a relation between the two tables?
Those are some of the questions that you'd need to ask yourself to decide
how to structure your data. If you can add some details we'll try and help
you with this.
Cheers
Luca
On Tuesday, September 17, 2013 12:18:51 PM UTC+2, igo...@tomax.co.ilwrote:
Let's suppose that I have two tables in my project's DB (today it's SQL
Server) - Documents and Tasks. It was decided to store every data inserted
into the DB in Elasticsearch.
I am beginning to learn ES and I'm wondering how many indices should I
have - one for all data or one index for every DB entity (Documents index
and Tasks index).
If the two tables have a relation between them, then you can try and
represent it in elasticsearch or flatten your data. I would suggest you to
have a look at this
blogpost: Elasticsearch Platform — Find real-time answers at scale | Elastic
. It basically explains what options you have and the difference between
them.
The reason why I asked how you want to get the results back, was becasue if
you for instance use nested documents, you can only retrieve the parent
document and its children, while with parent child the documents can have a
separate update lifecycle but you can't retrieve both parent and children
in the same call. Also, do consider the "manual" option, flattening your
data, which is usually the most performant one!
Cheers
Luca
On Tuesday, September 17, 2013 12:53:08 PM UTC+2, igo...@tomax.co.il wrote:
Hi Luca,
Thank you very much for your answer.
Are you going to get documents by id? "no" (today, but who knows what
we'll want tomorrow?)
Run full-text searches? Of course!
Where are the fields that you want to search on, in the documents or in
the tasks? (In both)
How do you want to get results back? I'd prefer to get them as list of
objects defined in our project.
Is there a relation between the two tables? Sure. Every task has one or
more documents.
Thank you in advance,
Igor
On Tuesday, September 17, 2013 1:47:18 PM UTC+3, Luca Cavanna wrote:
Hi Igor,
the data design phase is really important. And it's mainly driven by how
you are going to use your data. For instance, are you going to get
documents by id? Or run full-text searches? Where are the fields that you
want to search on, in the documents or in the tasks? And how do you want to
get results back? Is there a relation between the two tables?
Those are some of the questions that you'd need to ask yourself to decide
how to structure your data. If you can add some details we'll try and help
you with this.
Cheers
Luca
On Tuesday, September 17, 2013 12:18:51 PM UTC+2, igo...@tomax.co.ilwrote:
Let's suppose that I have two tables in my project's DB (today it's SQL
Server) - Documents and Tasks. It was decided to store every data inserted
into the DB in Elasticsearch.
I am beginning to learn ES and I'm wondering how many indices should I
have - one for all data or one index for every DB entity (Documents index
and Tasks index).
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.