# Splitting the elastic index to am and pm **URL:** \ **Category:** Elasticsearch\ **Created:** [July 10, 2017, 5:43pm UTC](https://discuss.elastic.co/t/splitting-the-elastic-index-to-am-and-pm/92513 "2017-07-10T17:43:35Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![putharekulu](https://avatars.discourse-cdn.com/v4/letter/p/a9adbd/32.png) [@putharekulu](https://discuss.elastic.co/u/putharekulu)\ **Post date:** [July 10, 2017, 5:43pm UTC](https://discuss.elastic.co/t/splitting-the-elastic-index-to-am-and-pm/92513/1 "2017-07-10T17:43:35Z")
i have an elastic index on date basis and it should get 1B documents/day. in the current scenario i am able to get only 500-600M and it is too slow [like getting yesterday's logs today to elastic] . I am thinking of splitting the index into AM and PMso as to improve performance. but i am not too sure if elastic can differentiate the difference between am and pm. I have a shell script that creates the mapping a day before, date=`date --date="+1 day" +%Y-%m-%d` can i just append AM/PM to date=`date --date="+1 day" +%Y-%m-%d-%p` . ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 10, 2017, 5:46pm UTC](https://discuss.elastic.co/t/splitting-the-elastic-index-to-am-and-pm/92513/2 "2017-07-10T17:46:52Z")
What is too slow? Indexing? Querying? ---
**Author:** ![putharekulu](https://avatars.discourse-cdn.com/v4/letter/p/a9adbd/32.png) [@putharekulu](https://discuss.elastic.co/u/putharekulu)\ **Post date:** [July 10, 2017, 6:16pm UTC](https://discuss.elastic.co/t/splitting-the-elastic-index-to-am-and-pm/92513/3 "2017-07-10T18:16:24Z")
Indexing as well as querying. But at this point i am only worried about indexing.i have a 5 node cluster with c4.8\*large cluster instance type and have 2 indexes per day with one of them getting 100M/day and the other 1billion/day. no issues with the one getting 100M/day . ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 10, 2017, 6:27pm UTC](https://discuss.elastic.co/t/splitting-the-elastic-index-to-am-and-pm/92513/4 "2017-07-10T18:27:00Z")
How many shards have you got configured for the larger index? How large do the shards get? ---
**Author:** ![putharekulu](https://avatars.discourse-cdn.com/v4/letter/p/a9adbd/32.png) [@putharekulu](https://discuss.elastic.co/u/putharekulu)\ **Post date:** [July 10, 2017, 6:34pm UTC](https://discuss.elastic.co/t/splitting-the-elastic-index-to-am-and-pm/92513/5 "2017-07-10T18:34:43Z")
I have 5 shards for the larger index. each shard is showing around 150gb for yesterday. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 10, 2017, 6:37pm UTC](https://discuss.elastic.co/t/splitting-the-elastic-index-to-am-and-pm/92513/6 "2017-07-10T18:37:43Z")
You might benefit from using the [rollover index API](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/indices-rollover-index.html) as this allows you to set a target size with respect to number of documents and/or time. You could then rollover to a new index based on the shard size rather than strictly by time, which would make it easier to deal with increasing or peaky loads. ---
**Author:** ![putharekulu](https://avatars.discourse-cdn.com/v4/letter/p/a9adbd/32.png) [@putharekulu](https://discuss.elastic.co/u/putharekulu)\ **Post date:** [July 10, 2017, 6:49pm UTC](https://discuss.elastic.co/t/splitting-the-elastic-index-to-am-and-pm/92513/7 "2017-07-10T18:49:30Z")
Thanks Christian.Will implement this and post the results. ---
**Author:** ![putharekulu](https://avatars.discourse-cdn.com/v4/letter/p/a9adbd/32.png) [@putharekulu](https://discuss.elastic.co/u/putharekulu)\ **Post date:** [July 10, 2017, 8:15pm UTC](https://discuss.elastic.co/t/splitting-the-elastic-index-to-am-and-pm/92513/8 "2017-07-10T20:15:28Z")
followed the steps as in the link you provided me.... from the shell script did the following curl -X PUT '[http://server:9200/test-'$date-1](http://server:9200/test-'%24date-1) -d '{ "aliases": { "test": {} } }' checked from the command line curl -XGET [http://server:9200/test/\_alias](http://server:9200/test/_alias) {"test-2017-07-11-1":{"aliases":{"test":{}}}} inserted data into the index using curl -XPUT [http://server:9200/test/log/1](http://server:9200/test/log/1) -d '{"message" : "Test"}' - 1 document is inserted into the index tried to rollover curl -X POST '[http://server:9200/test/\_rollover](http://server:9200/test/_rollover)' -d '{ "conditions": { "max\_docs": 1 } }' when i run the above i am getting {"error":"InvalidTypeNameException[mapping type name [_rollover] can't start with '_']","status":400}. Can you let me know how i can resolve this? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 7, 2017, 8:15pm UTC](https://discuss.elastic.co/t/splitting-the-elastic-index-to-am-and-pm/92513/9 "2017-08-07T20:15:36Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.