Thank you very Much,
The solutions you suggested are very closer to my problem, but I need
something more. Here is my project requirement
Continuously my application receives data (TB), I have to index every
file, and from the GUI I should display the data (matching data)
After 90 days I will delete the 1st day data, and store 91 data in it, for
this I am using 11 machines, 1 is master 10 are slaves, so that, I can
store first 10 days data in one shard, 10-20 days of data in another shard
and so on, when user asks to display data between the dates, I can fire the
query on specific shard using routing. Indexing, and searching is easy with
this approach, but* I am unable to route first 10 days of data in to one
shard.*
And also I need faster queries, because in 90 days data may be extended to
300-400 TB
-
So I want to divide the data based on Date, so when a query is fired I ac
search in 30 TB.*
-
Looking for your response*
On Thursday, 25 July 2013 12:32:11 UTC+5:30, Alexander Reelsen wrote:
Hey,
dont think in shards, but rather in indexes. For example, have one index
per day and use aliases to group all indices for the data of one week or
one month. The question is, why do you want to define what is going in
which shard (or which index). Do you want to have faster queries? Do you
want to to offload older data to weaker machines (a common scenario, which
is possible with the allocation feature)? Maybe you can elaborate a bit
there.
See:
Elasticsearch Platform — Find real-time answers at scale | Elastic
Elasticsearch Platform — Find real-time answers at scale | Elastic (See shard
allocation)
Hope this helps for a start. If not, please explain your use case.
--Alex
On Thu, Jul 25, 2013 at 8:29 AM, Mahesh Parimi <mahesh.p...@gmail.com<javascript:>
wrote:
Hi
I want to index 30 days of data, in each day i will get GB's of data
I Have 30 days of data, i have 10 shards, i want to route in such away ,
that first 3 days of data should go to first shard, next 3 days of data
should go to next shard ..etc
I am routing with date field, but i found data belongs to 1, 10,17 is
going to 1 st 2, 11, .. going to next shard...etc .
Looking for u r response
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.