Clarification needed on stats

Hi,

I am using elasticsearch to index the documents. I created a cluster of ES
with 2 node work as master and node both. Below is the result of the
_cluster/health api:
{
"cluster_name" : "es-cluster",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 2,
"active_primary_shards" : 10,
"active_shards" : 20,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0
}

When I call the stats API, it prints the statistics about the index. I
looked on the output docs:count and size of the index which is 174005 and
138.2mb, marked blue as below. However, in total section, it shows docs:
count: 348010 and size:276.4mb marked red as below. Could someone please
explain this difference to me? Is this usual behavior or is something wrong
with my configuration?

{"ok":true,"_shards":{"total":10,"successful":10,"failed":0},"_all":{"primaries":{
"docs":{"count":174005,"deleted":0},"store":{"size":"138.2mb"
,"size_in_bytes":144921313,"throttle_time":"0s","throttle_time_in_millis":0},
"indexing":"index_total":174005,"index_time":"28.3s","index_time_in_millis":28339,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":"total":0,"time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},
"search":"query_total":790,"query_time":"1.1s","query_time_in_millis":1134,"query_current":0,"fetch_total":55,"fetch_time":"7ms","fetch_time_in_millis":7,"fetch_current":0}},"total":{
"docs":{"count":348010,"deleted":0},"store":{"size":"276.4mb"
,"size_in_bytes":289840531,"throttle_time":"0s","throttle_time_in_millis":0},"indexing":{"index_total":348008,"index_time":"56.9s","index_time_in_millis":56906,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},"search":{"query_total":1580,"query_time":"2.2s","query_time_in_millis":2293,"query_current":0,"fetch_total":109,"fetch_time":"12ms","fetch_time_in_millis":12,"fetch_current":0}},"indices":{"txlog":{"primaries":{"docs":{"count":174005,"deleted":0},"store":{"size":"138.2mb","size_in_bytes":144921313,"throttle_time":"0s","throttle_time_in_millis":0},"indexing":{"index_total":174005,"index_time":"28.3s","index_time_in_millis":28339,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},"search":{"query_total":790,"query_time":"1.1s","query_time_in_millis":1134,"query_current":0,"fetch_total":55,"fetch_time":"7ms","fetch_time_in_millis":7,"fetch_current":0}},"total":{"docs":{"count":348010,"deleted":0},"store":{"size":"276.4mb","size_in_bytes":289840531,"throttle_time":"0s","throttle_time_in_millis":0},"indexing":{"index_total":348008,"index_time":"56.9s","index_time_in_millis":56906,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},"search":{"query_total":1580,"query_time":"2.2s","query_time_in_millis":2293,"query_current":0,"fetch_total":109,"fetch_time":"12ms","fetch_time_in_millis":12,"fetch_current":0}}}}}}

Thanks

Ashutosh

--

The answer is written in the stats:

"active_primary_shards" : 10,
"active_shards" : 20

Seems like you have 10 shards and 1 replica per shard. So "primaries":{
"docs":{"count":174005 is for the primary shards and "total":{
"docs":{"count":348010 is for total, including primaries and replicas.
Therefore double number.

Greets
Andrej

--

Thank you Andrej for the clarification. It means I should modify the cluster settings to configure shards and replicas. I'll have to add more nodes in near future.

Please suggest if you have any input on this.

Thanks
Ashutosh

On Nov 14, 2012, at 8:23 PM, Andrej Rosenheinrich andrej.rosenheinrich@unister.de wrote:

The answer is written in the stats:

"active_primary_shards" : 10,
"active_shards" : 20

Seems like you have 10 shards and 1 replica per shard. So "primaries":{"docs":{"count":174005 is for the primary shards and "total":{"docs":{"count":348010 is for total, including primaries and replicas. Therefore double number.

Greets
Andrej

--

--