Shard

Elastic Stack Elasticsearch
1

Hi,
I am kind of new to this project.I have some doubts regarding
Shard,replica and index.

As per as i know
1.When we create 10k index with 2 shards and 1 replica on node 1 ,then
if I up node 2,node 2 must have 2 shards of 10k memory and node 1 as
well..
2.On node 1, 2 shards will be created each shard having 5k memory.
Can u please clarify me whether above two scenarios are correct?

What is the concept behind creating more than one shard on single
node or on multiple nodes?

Its better if u tell any documents on basics of elastic search.

Thanks,
Kalyan.

2

A complete index is broken up into shards.

So if you have 10k documents in a index , and you assign 2 shards and 0
replica , 5k documents will sit on one node , and the rest of 5k document
will sit on the next node.

When a search request comes , the request is applied on both nodes and the
result is merged to obtain the final result.
Thus depending on the size of your index , you need to specify the number
of shards.

If you increase the replica , a copy of data in each node is stored in some
other node too. (This is to make sure even if one node goes down , you get
to have the entire data)

Also one thing to remember here ,- you cant increase the number of shards
after you set it for a index. But you can increase number of replica.

Thanks
Vineeth

On Wed, Nov 2, 2011 at 1:59 PM, kalyan kalyanc007@gmail.com wrote:

Hi,
I am kind of new to this project.I have some doubts regarding
Shard,replica and index.

As per as i know
1.When we create 10k index with 2 shards and 1 replica on node 1 ,then
if I up node 2,node 2 must have 2 shards of 10k memory and node 1 as
well..
2.On node 1, 2 shards will be created each shard having 5k memory.
Can u please clarify me whether above two scenarios are correct?

What is the concept behind creating more than one shard on single
node or on multiple nodes?

Its better if u tell any documents on basics of Elasticsearch.

Thanks,
Kalyan.

3

Thanks Vineeth,

As u said,If I mention 2 shards(0 replica),then I must have 2 nodes so
that my 10k data can be shared on two nodes?

On Nov 2, 2:00 pm, Vineeth Mohan vineethmo...@algotree.com wrote:

A complete index is broken up into shards.

So if you have 10k documents in a index , and you assign 2 shards and 0
replica , 5k documents will sit on one node , and the rest of 5k document
will sit on the next node.

When a search request comes , the request is applied on both nodes and the
result is merged to obtain the final result.
Thus depending on the size of your index , you need to specify the number
of shards.

If you increase the replica , a copy of data in each node is stored in some
other node too. (This is to make sure even if one node goes down , you get
to have the entire data)

Also one thing to remember here ,- you cant increase the number of shards
after you set it for a index. But you can increase number of replica.

Thanks
Vineeth

On Wed, Nov 2, 2011 at 1:59 PM, kalyan kalyanc...@gmail.com wrote:

Hi,
I am kind of new to this project.I have some doubts regarding
Shard,replica and index.

As per as i know
1.When we create 10k index with 2 shards and 1 replica on node 1 ,then
if I up node 2,node 2 must have 2 shards of 10k memory and node 1 as
well..
2.On node 1, 2 shards will be created each shard having 5k memory.
Can u please clarify me whether above two scenarios are correct?

What is the concept behind creating more than one shard on single
node or on multiple nodes?

Its better if u tell any documents on basics of Elasticsearch.

Thanks,
Kalyan.

4

You need not , it fills into available space (as far as i know).

If there is only 1 node and specified shard is 2 , it compromises with a
single node.
If another node is added to the cluster at any time , it repartition to 2
nodes automatically.

Thanks
VIneeth

On Wed, Nov 2, 2011 at 2:53 PM, kalyan kalyanc007@gmail.com wrote:

Thanks Vineeth,

As u said,If I mention 2 shards(0 replica),then I must have 2 nodes so
that my 10k data can be shared on two nodes?

On Nov 2, 2:00 pm, Vineeth Mohan vineethmo...@algotree.com wrote:

A complete index is broken up into shards.

So if you have 10k documents in a index , and you assign 2 shards and 0
replica , 5k documents will sit on one node , and the rest of 5k document
will sit on the next node.

When a search request comes , the request is applied on both nodes and
the
result is merged to obtain the final result.
Thus depending on the size of your index , you need to specify the number
of shards.

If you increase the replica , a copy of data in each node is stored in
some
other node too. (This is to make sure even if one node goes down , you
get
to have the entire data)

Also one thing to remember here ,- you cant increase the number of shards
after you set it for a index. But you can increase number of replica.

Thanks
Vineeth

On Wed, Nov 2, 2011 at 1:59 PM, kalyan kalyanc...@gmail.com wrote:

Hi,
I am kind of new to this project.I have some doubts regarding
Shard,replica and index.

As per as i know
1.When we create 10k index with 2 shards and 1 replica on node 1 ,then
if I up node 2,node 2 must have 2 shards of 10k memory and node 1 as
well..
2.On node 1, 2 shards will be created each shard having 5k memory.
Can u please clarify me whether above two scenarios are correct?

What is the concept behind creating more than one shard on single
node or on multiple nodes?

Its better if u tell any documents on basics of Elasticsearch.

Thanks,
Kalyan.

5

Ok thanks,
Now can u tell me,how can I come to know that how many files can be stored
in Elasticsearch?
Please explain with one use case like if i have some memory and created 10
shards 2 replicas(for example)....?

thanks,
kalyan
On Wed, Nov 2, 2011 at 3:10 PM, Vineeth Mohan vineethmohan@algotree.comwrote:

You need not , it fills into available space (as far as i know).

If there is only 1 node and specified shard is 2 , it compromises with a
single node.
If another node is added to the cluster at any time , it repartition to 2
nodes automatically.

Thanks
VIneeth

On Wed, Nov 2, 2011 at 2:53 PM, kalyan kalyanc007@gmail.com wrote:

Thanks Vineeth,

As u said,If I mention 2 shards(0 replica),then I must have 2 nodes so
that my 10k data can be shared on two nodes?

On Nov 2, 2:00 pm, Vineeth Mohan vineethmo...@algotree.com wrote:

A complete index is broken up into shards.

So if you have 10k documents in a index , and you assign 2 shards and 0
replica , 5k documents will sit on one node , and the rest of 5k
document
will sit on the next node.

When a search request comes , the request is applied on both nodes and
the
result is merged to obtain the final result.
Thus depending on the size of your index , you need to specify the
number
of shards.

If you increase the replica , a copy of data in each node is stored in
some
other node too. (This is to make sure even if one node goes down , you
get
to have the entire data)

Also one thing to remember here ,- you cant increase the number of
shards
after you set it for a index. But you can increase number of replica.

Thanks
Vineeth

On Wed, Nov 2, 2011 at 1:59 PM, kalyan kalyanc...@gmail.com wrote:

Hi,
I am kind of new to this project.I have some doubts regarding
Shard,replica and index.

As per as i know
1.When we create 10k index with 2 shards and 1 replica on node 1 ,then
if I up node 2,node 2 must have 2 shards of 10k memory and node 1 as
well..
2.On node 1, 2 shards will be created each shard having 5k memory.
Can u please clarify me whether above two scenarios are correct?

What is the concept behind creating more than one shard on single
node or on multiple nodes?

Its better if u tell any documents on basics of Elasticsearch.

Thanks,
Kalyan.

6

It depends on the machine where the node is running.
I guess the node keeps on adding documents until the disk is full.

Thanks
Vineeth

On Wed, Nov 2, 2011 at 3:44 PM, kalyan chowdary kalyanc007@gmail.comwrote:

Ok thanks,
Now can u tell me,how can I come to know that how many files can be stored
in Elasticsearch?
Please explain with one use case like if i have some memory and created 10
shards 2 replicas(for example)....?

thanks,
kalyan

On Wed, Nov 2, 2011 at 3:10 PM, Vineeth Mohan vineethmohan@algotree.comwrote:

You need not , it fills into available space (as far as i know).

If there is only 1 node and specified shard is 2 , it compromises with a
single node.
If another node is added to the cluster at any time , it repartition to 2
nodes automatically.

Thanks
VIneeth

On Wed, Nov 2, 2011 at 2:53 PM, kalyan kalyanc007@gmail.com wrote:

Thanks Vineeth,

As u said,If I mention 2 shards(0 replica),then I must have 2 nodes so
that my 10k data can be shared on two nodes?

On Nov 2, 2:00 pm, Vineeth Mohan vineethmo...@algotree.com wrote:

A complete index is broken up into shards.

So if you have 10k documents in a index , and you assign 2 shards and 0
replica , 5k documents will sit on one node , and the rest of 5k
document
will sit on the next node.

When a search request comes , the request is applied on both nodes
and the
result is merged to obtain the final result.
Thus depending on the size of your index , you need to specify the
number
of shards.

If you increase the replica , a copy of data in each node is stored in
some
other node too. (This is to make sure even if one node goes down , you
get
to have the entire data)

Also one thing to remember here ,- you cant increase the number of
shards
after you set it for a index. But you can increase number of replica.

Thanks
Vineeth

On Wed, Nov 2, 2011 at 1:59 PM, kalyan kalyanc...@gmail.com wrote:

Hi,
I am kind of new to this project.I have some doubts regarding
Shard,replica and index.

As per as i know
1.When we create 10k index with 2 shards and 1 replica on node 1
,then
if I up node 2,node 2 must have 2 shards of 10k memory and node 1 as
well..
2.On node 1, 2 shards will be created each shard having 5k memory.
Can u please clarify me whether above two scenarios are correct?

What is the concept behind creating more than one shard on single
node or on multiple nodes?

Its better if u tell any documents on basics of Elasticsearch.

Thanks,
Kalyan.