# Shard dividing advise **URL:** \ **Category:** Elasticsearch\ **Created:** [July 28, 2015, 5:06pm UTC](https://discuss.elastic.co/t/shard-dividing-advise/26438 "2015-07-28T17:06:13Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![nitram](https://avatars.discourse-cdn.com/v4/letter/n/5fc32e/32.png) [@nitram](https://discuss.elastic.co/u/nitram)\ **Post date:** [July 28, 2015, 5:06pm UTC](https://discuss.elastic.co/t/shard-dividing-advise/26438/1 "2015-07-28T17:06:14Z")
Hello, im basically about to turn my 4 months testing system into production but the only concern is whether I decide correctly on initial mapping and shard allocation. My plan is to store using logstash logs from approximately 15 sources (applications) from csv into elasticsrarch. Monthly it takes approx 30 (2mil each) mil documents all together. I would like to have it available one year (after just to close it and open when needed). Im use kibana4 and huge aggregations (unique count etc.) and I dont have extra powerfull cluster (only 70GB RAM, 10 nodes). My questions: 1. What would be best setting of shards and replicas for performance for search (to have one index each month with all sources and replicated it 10 times or to have 1 index per source and month and splitted into multiple shards ???) 2. Im planning to use use doc\_values for fields that dont need to be analyzed - can I set doc\_values for .RAW field to benefit from analyzed fields and doc\_values from non analyzed fields ? 3. Im planning to have 2 master nodes with 1/2 of RAM to HEAP(4+4 GB) and rest will be slaves with all memory to HEAP (each approx 8-16GB) - Is it suitable? 4. Logstash will run only on one master, because all the csv will be stored there 5. It is worthy to deal with removing fields as \_path, \_souce, \_message etc automatically generated by logstash in terms of speed? Thanks for any kind of advice nitraM ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 28, 2015, 10:09pm UTC](https://discuss.elastic.co/t/shard-dividing-advise/26438/2 "2015-07-28T22:09:53Z")
1. Better to have multiple shards with a single replica. Have a shard per node. 2. Yep. 3. Are your masters holding data? 4. Ok 5. Depends. Removing `_source` means you cannot reindex without the original data. ---
**Author:** ![nitram](https://avatars.discourse-cdn.com/v4/letter/n/5fc32e/32.png) [@nitram](https://discuss.elastic.co/u/nitram)\ **Post date:** [July 29, 2015, 5:41am UTC](https://discuss.elastic.co/t/shard-dividing-advise/26438/3 "2015-07-29T05:41:41Z")
1. I see, so when i assume to have 10 nodes in total i will set up on master nide shards: 5 replicas: 1 and to have single index per month. Splitting indexes for sources will make mo difference? logstash-%yy-%mm compares to logstash-%app-%yy-%mm, In Kibana using logstash-\* 2. yes master will hold data, or is better not to ? Thanks ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 29, 2015, 5:48am UTC](https://discuss.elastic.co/t/shard-dividing-advise/26438/4 "2015-07-29T05:48:00Z")
1. Don't set that in config, better to define it when you create the index. You can create multiple indices, just don't go crazy. 2. Better not to, but depends on if you can afford dedicated master nodes. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:58pm UTC](https://discuss.elastic.co/t/shard-dividing-advise/26438/5 "2017-07-05T23:58:28Z")