# Request for elastic search configuration **URL:** \ **Category:** Elasticsearch\ **Created:** [December 19, 2013, 12:53am UTC](https://discuss.elastic.co/t/request-for-elastic-search-configuration/14943 "2013-12-19T00:53:45Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![srikanth\_ramineni](https://avatars.discourse-cdn.com/v4/letter/s/aca169/32.png) [@srikanth\_ramineni](https://discuss.elastic.co/u/srikanth_ramineni)\ **Post date:** [December 19, 2013, 12:53am UTC](https://discuss.elastic.co/t/request-for-elastic-search-configuration/14943/1 "2013-12-19T00:53:45Z")
Hi Team, We have below server configurations. We are trying to load more than 500 million data into elastic. When data reaches 40 millions, shards are going down and data is deleting . Can you please help us with best configuration for above requirement. Below are the server details we have 4 nodes. Each node contains CPU: 4 , RAM: 16G , Storage: 122G ES\_HEAP\_SIZE 8 G Thanks, Srikanth. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 19, 2013, 1:02am UTC](https://discuss.elastic.co/t/request-for-elastic-search-configuration/14943/2 "2013-12-19T01:02:58Z")
It will help if you post (via a gist/pastebin/etc) what you have now. Also what sort of data is it, how are you loading it, what versions of things are you running on (OS, java, ES, etc etc) Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 19 December 2013 11:53, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com) wrote: > Hi Team, > > We have below server configurations. We are trying to load more than 500 > million data into elastic. When data reaches 40 millions, shards are going > down and data is deleting . Can you please help us with best configuration > for above requirement. > > Below are the server details > > we have 4 nodes. > > Each node contains > CPU: 4 , RAM: 16G , Storage: 122G > ES\_HEAP\_SIZE 8 G > > Thanks, > Srikanth. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com) > . > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624YPB%3DkkJ3eOFRRc%2Bw%2BqG3junYQWpiD3EzPp677BLTbwvQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624YPB%3DkkJ3eOFRRc%2Bw%2BqG3junYQWpiD3EzPp677BLTbwvQ%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![srikanth\_ramineni](https://avatars.discourse-cdn.com/v4/letter/s/aca169/32.png) [@srikanth\_ramineni](https://discuss.elastic.co/u/srikanth_ramineni)\ **Post date:** [December 19, 2013, 3:18am UTC](https://discuss.elastic.co/t/request-for-elastic-search-configuration/14943/3 "2013-12-19T03:18:33Z")
Hi Mark Walkom, Thanks for your quick response... Sorry , I am not sure how to post from gist/pastebin/etc. Below are your requested details Operating system : Linux ES version : Elasticsearch-0.90.7 JAVA Version :JDK 1.7 Data : we didn't have any parent child relations. Just we are loading with auto generated primary key.(29,666,482 documents= 21.9 GB). Data Loading : we are loading data into elastic by writing 20 threads. Client creation : Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", "UCS\_DEV\_ES\_CLUSTER").build(); elasticClient = new TransportClient(settings).addTransportAddress(new InetSocketTransportAddress("server1", 9300)) .addTransportAddress(new InetSocketTransportAddress("server2", 9300)); Below are the server details we have 4 nodes. Each node contains CPU: 4 , RAM: 16G , Storage: 122G ES\_HEAP\_SIZE 8 G Please find enclosed elasticsearch.yml file which we are using. Thanks, Srikanth. On Wed, Dec 18, 2013 at 5:02 PM, Mark Walkom [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)wrote: > It will help if you post (via a gist/pastebin/etc) what you have now. > > Also what sort of data is it, how are you loading it, what versions of > things are you running on (OS, java, ES, etc etc) > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > On 19 December 2013 11:53, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com)wrote: > > > Hi Team, > > > > We have below server configurations. We are trying to load more than 500 > > million data into elastic. When data reaches 40 millions, shards are going > > down and data is deleting . Can you please help us with best configuration > > for above requirement. > > > > Below are the server details > > > > we have 4 nodes. > > > > Each node contains > > CPU: 4 , RAM: 16G , Storage: 122G > > ES\_HEAP\_SIZE 8 G > > > > Thanks, > > Srikanth. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com) > > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAEM624YPB%3DkkJ3eOFRRc%2Bw%2BqG3junYQWpiD3EzPp677BLTbwvQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624YPB%3DkkJ3eOFRRc%2Bw%2BqG3junYQWpiD3EzPp677BLTbwvQ%40mail.gmail.com) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGuNUAV5A\_Vj9k285nAW\_xW7UdnVr5K5n1DBndBLo%3D9KiQ8BaQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGuNUAV5A_Vj9k285nAW_xW7UdnVr5K5n1DBndBLo%3D9KiQ8BaQ%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 19, 2013, 3:28am UTC](https://discuss.elastic.co/t/request-for-elastic-search-configuration/14943/4 "2013-12-19T03:28:47Z")
[http://pastebin.com/](http://pastebin.com/) - Please don't send attachments to the list, it goes to hundreds/thousands of people. What JDK is it, oracle or open? What sort of data is it - logs? Web pages? Text? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 19 December 2013 14:18, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com) wrote: > Hi Mark Walkom, > > Thanks for your quick response... > > Sorry , I am not sure how to post from gist/pastebin/etc. Below are your > requested details > > Operating system : Linux > ES version : Elasticsearch-0.90.7 > JAVA Version :JDK 1.7 > Data : we didn't have any parent child relations. Just we are loading with > auto generated primary key.(29,666,482 documents= 21.9 GB). > > Data Loading : we are loading data into elastic by writing 20 threads. > > Client creation : > > Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", > "UCS\_DEV\_ES\_CLUSTER").build(); > > elasticClient = new TransportClient(settings).addTransportAddress(new > InetSocketTransportAddress("server1", 9300)) > .addTransportAddress(new InetSocketTransportAddress("server2", 9300)); > Below are the server details > > we have 4 nodes. > > Each node contains > CPU: 4 , RAM: 16G , Storage: 122G > ES\_HEAP\_SIZE 8 G > > Please find enclosed elasticsearch.yml file which we are using. > > Thanks, > Srikanth. > > On Wed, Dec 18, 2013 at 5:02 PM, Mark Walkom [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)wrote: > > > It will help if you post (via a gist/pastebin/etc) what you have now. > > > > Also what sort of data is it, how are you loading it, what versions of > > things are you running on (OS, java, ES, etc etc) > > > > Regards, > > Mark Walkom > > > > Infrastructure Engineer > > Campaign Monitor > > email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > > > On 19 December 2013 11:53, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com)wrote: > > > > > Hi Team, > > > > > > We have below server configurations. We are trying to load more than > > > 500 million data into elastic. When data reaches 40 millions, shards are > > > going down and data is deleting . Can you please help us with best > > > configuration for above requirement. > > > > > > Below are the server details > > > > > > we have 4 nodes. > > > > > > Each node contains > > > CPU: 4 , RAM: 16G , Storage: 122G > > > ES\_HEAP\_SIZE 8 G > > > > > > Thanks, > > > Srikanth. > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com) > > > . > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAEM624YPB%3DkkJ3eOFRRc%2Bw%2BqG3junYQWpiD3EzPp677BLTbwvQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624YPB%3DkkJ3eOFRRc%2Bw%2BqG3junYQWpiD3EzPp677BLTbwvQ%40mail.gmail.com) > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAGuNUAV5A\_Vj9k285nAW\_xW7UdnVr5K5n1DBndBLo%3D9KiQ8BaQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGuNUAV5A_Vj9k285nAW_xW7UdnVr5K5n1DBndBLo%3D9KiQ8BaQ%40mail.gmail.com) > . > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624b2JyfYmC5\_mZkz%2BXQe\_4hzrr79ALx3Hj0ww6%3DTeqHOHQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624b2JyfYmC5_mZkz%2BXQe_4hzrr79ALx3Hj0ww6%3DTeqHOHQ%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![srikanth\_ramineni](https://avatars.discourse-cdn.com/v4/letter/s/aca169/32.png) [@srikanth\_ramineni](https://discuss.elastic.co/u/srikanth_ramineni)\ **Post date:** [December 19, 2013, 3:49am UTC](https://discuss.elastic.co/t/request-for-elastic-search-configuration/14943/5 "2013-12-19T03:49:20Z")
Hi Mark Walkom, I have updated the detail in pastebin . Pleae find below url for details. [http://pastebin.com/62aPBkPc](http://pastebin.com/62aPBkPc) Please get back to me for further quries. thanks, Srikanth. On Wed, Dec 18, 2013 at 7:28 PM, Mark Walkom [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)wrote: > [http://pastebin.com/](http://pastebin.com/) - Please don't send attachments to the list, it goes > to hundreds/thousands of people. > > What JDK is it, oracle or open? > What sort of data is it - logs? Web pages? Text? > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > On 19 December 2013 14:18, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com)wrote: > > > Hi Mark Walkom, > > > > Thanks for your quick response... > > > > Sorry , I am not sure how to post from gist/pastebin/etc. Below are > > your requested details > > > > Operating system : Linux > > ES version : Elasticsearch-0.90.7 > > JAVA Version :JDK 1.7 > > Data : we didn't have any parent child relations. Just we are loading > > with auto generated primary key.(29,666,482 documents= 21.9 GB). > > > > Data Loading : we are loading data into elastic by writing 20 threads. > > > > Client creation : > > > > Settings settings = ImmutableSettings.settingsBuilder().put(" > > cluster.name", "UCS\_DEV\_ES\_CLUSTER").build(); > > > > elasticClient = new TransportClient(settings).addTransportAddress(new > > InetSocketTransportAddress("server1", 9300)) > > .addTransportAddress(new InetSocketTransportAddress("server2", 9300)); > > Below are the server details > > > > we have 4 nodes. > > > > Each node contains > > CPU: 4 , RAM: 16G , Storage: 122G > > ES\_HEAP\_SIZE 8 G > > > > Please find enclosed elasticsearch.yml file which we are using. > > > > Thanks, > > Srikanth. > > > > On Wed, Dec 18, 2013 at 5:02 PM, Mark Walkom [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)wrote: > > > > > It will help if you post (via a gist/pastebin/etc) what you have now. > > > > > > Also what sort of data is it, how are you loading it, what versions of > > > things are you running on (OS, java, ES, etc etc) > > > > > > Regards, > > > Mark Walkom > > > > > > Infrastructure Engineer > > > Campaign Monitor > > > email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > > > > > On 19 December 2013 11:53, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com)wrote: > > > > > > > Hi Team, > > > > > > > > We have below server configurations. We are trying to load more than > > > > 500 million data into elastic. When data reaches 40 millions, shards are > > > > going down and data is deleting . Can you please help us with best > > > > configuration for above requirement. > > > > > > > > Below are the server details > > > > > > > > we have 4 nodes. > > > > > > > > Each node contains > > > > CPU: 4 , RAM: 16G , Storage: 122G > > > > ES\_HEAP\_SIZE 8 G > > > > > > > > Thanks, > > > > Srikanth. > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com) > > > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAEM624YPB%3DkkJ3eOFRRc%2Bw%2BqG3junYQWpiD3EzPp677BLTbwvQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624YPB%3DkkJ3eOFRRc%2Bw%2BqG3junYQWpiD3EzPp677BLTbwvQ%40mail.gmail.com) > > > . > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAGuNUAV5A\_Vj9k285nAW\_xW7UdnVr5K5n1DBndBLo%3D9KiQ8BaQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGuNUAV5A_Vj9k285nAW_xW7UdnVr5K5n1DBndBLo%3D9KiQ8BaQ%40mail.gmail.com) > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAEM624b2JyfYmC5\_mZkz%2BXQe\_4hzrr79ALx3Hj0ww6%3DTeqHOHQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624b2JyfYmC5_mZkz%2BXQe_4hzrr79ALx3Hj0ww6%3DTeqHOHQ%40mail.gmail.com) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGuNUAUNS3gwX-jNjz4\_PL7TyY3qG0d0zCb9Dm5WHsRpCRQQ3Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGuNUAUNS3gwX-jNjz4_PL7TyY3qG0d0zCb9Dm5WHsRpCRQQ3Q%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 19, 2013, 4:01am UTC](https://discuss.elastic.co/t/request-for-elastic-search-configuration/14943/6 "2013-12-19T04:01:35Z")
You will want to use Oracle java rather than OpenJDK, you will get a lot better performance. Now next question, what are you using for monitoring your cluster? Are you using plugins like ElasticHQ, kopf, bighead, head, paramedic etc etc. If not install one of those and it will give you insight into what is happening with your cluster and what possible causes for the problems might be. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 19 December 2013 14:49, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com) wrote: > Hi Mark Walkom, > > I have updated the detail in pastebin . Pleae find below url for details. > > [http://pastebin.com/62aPBkPc](http://pastebin.com/62aPBkPc) > > Please get back to me for further quries. > > thanks, > Srikanth. > > On Wed, Dec 18, 2013 at 7:28 PM, Mark Walkom [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)wrote: > > > [http://pastebin.com/](http://pastebin.com/) - Please don't send attachments to the list, it > > goes to hundreds/thousands of people. > > > > What JDK is it, oracle or open? > > What sort of data is it - logs? Web pages? Text? > > > > Regards, > > Mark Walkom > > > > Infrastructure Engineer > > Campaign Monitor > > email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > > > On 19 December 2013 14:18, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com)wrote: > > > > > Hi Mark Walkom, > > > > > > Thanks for your quick response... > > > > > > Sorry , I am not sure how to post from gist/pastebin/etc. Below are > > > your requested details > > > > > > Operating system : Linux > > > ES version : Elasticsearch-0.90.7 > > > JAVA Version :JDK 1.7 > > > Data : we didn't have any parent child relations. Just we are loading > > > with auto generated primary key.(29,666,482 documents= 21.9 GB). > > > > > > Data Loading : we are loading data into elastic by writing 20 threads. > > > > > > Client creation : > > > > > > Settings settings = ImmutableSettings.settingsBuilder().put(" > > > cluster.name", "UCS\_DEV\_ES\_CLUSTER").build(); > > > > > > elasticClient = new TransportClient(settings).addTransportAddress(new > > > InetSocketTransportAddress("server1", 9300)) > > > .addTransportAddress(new InetSocketTransportAddress("server2", 9300)); > > > Below are the server details > > > > > > we have 4 nodes. > > > > > > Each node contains > > > CPU: 4 , RAM: 16G , Storage: 122G > > > ES\_HEAP\_SIZE 8 G > > > > > > Please find enclosed elasticsearch.yml file which we are using. > > > > > > Thanks, > > > Srikanth. > > > > > > On Wed, Dec 18, 2013 at 5:02 PM, Mark Walkom [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)wrote: > > > > > > > It will help if you post (via a gist/pastebin/etc) what you have now. > > > > > > > > Also what sort of data is it, how are you loading it, what versions of > > > > things are you running on (OS, java, ES, etc etc) > > > > > > > > Regards, > > > > Mark Walkom > > > > > > > > Infrastructure Engineer > > > > Campaign Monitor > > > > email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > > > > > > > On 19 December 2013 11:53, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com)wrote: > > > > > > > > > Hi Team, > > > > > > > > > > We have below server configurations. We are trying to load more than > > > > > 500 million data into elastic. When data reaches 40 millions, shards are > > > > > going down and data is deleting . Can you please help us with best > > > > > configuration for above requirement. > > > > > > > > > > Below are the server details > > > > > > > > > > we have 4 nodes. > > > > > > > > > > Each node contains > > > > > CPU: 4 , RAM: 16G , Storage: 122G > > > > > ES\_HEAP\_SIZE 8 G > > > > > > > > > > Thanks, > > > > > Srikanth. > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com) > > > > > . > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > > > [https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe) > > > > . > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAEM624YPB%3DkkJ3eOFRRc%2Bw%2BqG3junYQWpiD3EzPp677BLTbwvQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624YPB%3DkkJ3eOFRRc%2Bw%2BqG3junYQWpiD3EzPp677BLTbwvQ%40mail.gmail.com) > > > > . > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAGuNUAV5A\_Vj9k285nAW\_xW7UdnVr5K5n1DBndBLo%3D9KiQ8BaQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGuNUAV5A_Vj9k285nAW_xW7UdnVr5K5n1DBndBLo%3D9KiQ8BaQ%40mail.gmail.com) > > > . > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAEM624b2JyfYmC5\_mZkz%2BXQe\_4hzrr79ALx3Hj0ww6%3DTeqHOHQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624b2JyfYmC5_mZkz%2BXQe_4hzrr79ALx3Hj0ww6%3DTeqHOHQ%40mail.gmail.com) > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAGuNUAUNS3gwX-jNjz4\_PL7TyY3qG0d0zCb9Dm5WHsRpCRQQ3Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGuNUAUNS3gwX-jNjz4_PL7TyY3qG0d0zCb9Dm5WHsRpCRQQ3Q%40mail.gmail.com) > . > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624ZLXXZMzchGSewoDhK9J3kK0WZjqxw8Pth9g2E-%3Du29MQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624ZLXXZMzchGSewoDhK9J3kK0WZjqxw8Pth9g2E-%3Du29MQ%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![srikanth\_ramineni](https://avatars.discourse-cdn.com/v4/letter/s/aca169/32.png) [@srikanth\_ramineni](https://discuss.elastic.co/u/srikanth_ramineni)\ **Post date:** [December 19, 2013, 4:30am UTC](https://discuss.elastic.co/t/request-for-elastic-search-configuration/14943/7 "2013-12-19T04:30:18Z")
Hi, We installed head plugin for monitoring the cluster. once it loaded 27GB, cluster is going to unavailable. We are not able to understand exact problem.. Can you please suggest elasticsearch.yml file configurations for 4 nodes for single cluster. we are loading data from oracle to elastic by using java threads.. Thanks, Srikanth. On Wed, Dec 18, 2013 at 8:01 PM, Mark Walkom [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)wrote: > You will want to use Oracle java rather than OpenJDK, you will get a lot > better performance. > > Now next question, what are you using for monitoring your cluster? Are you > using plugins like ElasticHQ, kopf, bighead, head, paramedic etc etc. If > not install one of those and it will give you insight into what is > happening with your cluster and what possible causes for the problems might > be. > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > On 19 December 2013 14:49, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com)wrote: > > > Hi Mark Walkom, > > > > I have updated the detail in pastebin . Pleae find below url for details. > > > > [http://pastebin.com/62aPBkPc](http://pastebin.com/62aPBkPc) > > > > Please get back to me for further quries. > > > > thanks, > > Srikanth. > > > > On Wed, Dec 18, 2013 at 7:28 PM, Mark Walkom [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)wrote: > > > > > [http://pastebin.com/](http://pastebin.com/) - Please don't send attachments to the list, it > > > goes to hundreds/thousands of people. > > > > > > What JDK is it, oracle or open? > > > What sort of data is it - logs? Web pages? Text? > > > > > > Regards, > > > Mark Walkom > > > > > > Infrastructure Engineer > > > Campaign Monitor > > > email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > > > > > On 19 December 2013 14:18, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com)wrote: > > > > > > > Hi Mark Walkom, > > > > > > > > Thanks for your quick response... > > > > > > > > Sorry , I am not sure how to post from gist/pastebin/etc. Below are > > > > your requested details > > > > > > > > Operating system : Linux > > > > ES version : Elasticsearch-0.90.7 > > > > JAVA Version :JDK 1.7 > > > > Data : we didn't have any parent child relations. Just we are loading > > > > with auto generated primary key.(29,666,482 documents= 21.9 GB). > > > > > > > > Data Loading : we are loading data into elastic by writing 20 threads. > > > > > > > > Client creation : > > > > > > > > Settings settings = ImmutableSettings.settingsBuilder().put(" > > > > cluster.name", "UCS\_DEV\_ES\_CLUSTER").build(); > > > > > > > > elasticClient = new TransportClient(settings).addTransportAddress(new > > > > InetSocketTransportAddress("server1", 9300)) > > > > .addTransportAddress(new InetSocketTransportAddress("server2", 9300)); > > > > Below are the server details > > > > > > > > we have 4 nodes. > > > > > > > > Each node contains > > > > CPU: 4 , RAM: 16G , Storage: 122G > > > > ES\_HEAP\_SIZE 8 G > > > > > > > > Please find enclosed elasticsearch.yml file which we are using. > > > > > > > > Thanks, > > > > Srikanth. > > > > > > > > On Wed, Dec 18, 2013 at 5:02 PM, Mark Walkom \<[markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > > > > > > > > > wrote: > > > > > > > > > It will help if you post (via a gist/pastebin/etc) what you have now. > > > > > > > > > > Also what sort of data is it, how are you loading it, what versions of > > > > > things are you running on (OS, java, ES, etc etc) > > > > > > > > > > Regards, > > > > > Mark Walkom > > > > > > > > > > Infrastructure Engineer > > > > > Campaign Monitor > > > > > email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > > > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > > > > > > > > > On 19 December 2013 11:53, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com)wrote: > > > > > > > > > > > Hi Team, > > > > > > > > > > > > We have below server configurations. We are trying to load more than > > > > > > 500 million data into elastic. When data reaches 40 millions, shards are > > > > > > going down and data is deleting . Can you please help us with best > > > > > > configuration for above requirement. > > > > > > > > > > > > Below are the server details > > > > > > > > > > > > we have 4 nodes. > > > > > > > > > > > > Each node contains > > > > > > CPU: 4 , RAM: 16G , Storage: 122G > > > > > > ES\_HEAP\_SIZE 8 G > > > > > > > > > > > > Thanks, > > > > > > Srikanth. > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > > > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com) > > > > > > . > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to a topic in the > > > > > Google Groups "elasticsearch" group. > > > > > To unsubscribe from this topic, visit > > > > > [https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe) > > > > > . > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAEM624YPB%3DkkJ3eOFRRc%2Bw%2BqG3junYQWpiD3EzPp677BLTbwvQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624YPB%3DkkJ3eOFRRc%2Bw%2BqG3junYQWpiD3EzPp677BLTbwvQ%40mail.gmail.com) > > > > > . > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAGuNUAV5A\_Vj9k285nAW\_xW7UdnVr5K5n1DBndBLo%3D9KiQ8BaQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGuNUAV5A_Vj9k285nAW_xW7UdnVr5K5n1DBndBLo%3D9KiQ8BaQ%40mail.gmail.com) > > > > . > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAEM624b2JyfYmC5\_mZkz%2BXQe\_4hzrr79ALx3Hj0ww6%3DTeqHOHQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624b2JyfYmC5_mZkz%2BXQe_4hzrr79ALx3Hj0ww6%3DTeqHOHQ%40mail.gmail.com) > > > . > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAGuNUAUNS3gwX-jNjz4\_PL7TyY3qG0d0zCb9Dm5WHsRpCRQQ3Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGuNUAUNS3gwX-jNjz4_PL7TyY3qG0d0zCb9Dm5WHsRpCRQQ3Q%40mail.gmail.com) > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAEM624ZLXXZMzchGSewoDhK9J3kK0WZjqxw8Pth9g2E-%3Du29MQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624ZLXXZMzchGSewoDhK9J3kK0WZjqxw8Pth9g2E-%3Du29MQ%40mail.gmail.com) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGuNUAWyNOXr90qf%2BdUu9NqG1J%3DJwiDm-2yXJ5FcuPNy28oznA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGuNUAWyNOXr90qf%2BdUu9NqG1J%3DJwiDm-2yXJ5FcuPNy28oznA%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 19, 2013, 6:12am UTC](https://discuss.elastic.co/t/request-for-elastic-search-configuration/14943/8 "2013-12-19T06:12:00Z")
Anything in logs? -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 19 déc. 2013 à 05:30, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com) a écrit : Hi, We installed head plugin for monitoring the cluster. once it loaded 27GB, cluster is going to unavailable. We are not able to understand exact problem.. Can you please suggest elasticsearch.yml file configurations for 4 nodes for single cluster. we are loading data from oracle to elastic by using java threads.. Thanks, Srikanth. > On Wed, Dec 18, 2013 at 8:01 PM, Mark Walkom [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) wrote: > You will want to use Oracle java rather than OpenJDK, you will get a lot better performance. > > Now next question, what are you using for monitoring your cluster? Are you using plugins like ElasticHQ, kopf, bighead, head, paramedic etc etc. If not install one of those and it will give you insight into what is happening with your cluster and what possible causes for the problems might be. > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > > On 19 December 2013 14:49, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com) wrote: > > Hi Mark Walkom, > > > > I have updated the detail in pastebin . Pleae find below url for details. > > > > [http://pastebin.com/62aPBkPc](http://pastebin.com/62aPBkPc) > > > > Please get back to me for further quries. > > > > thanks, > > Srikanth. > > > > > On Wed, Dec 18, 2013 at 7:28 PM, Mark Walkom [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) wrote: > > > [http://pastebin.com/](http://pastebin.com/) - Please don't send attachments to the list, it goes to hundreds/thousands of people. > > > > > > What JDK is it, oracle or open? > > > What sort of data is it - logs? Web pages? Text? > > > > > > Regards, > > > Mark Walkom > > > > > > Infrastructure Engineer > > > Campaign Monitor > > > email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > > > > > > On 19 December 2013 14:18, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com) wrote: > > > > Hi Mark Walkom, > > > > > > > > Thanks for your quick response... > > > > > > > > Sorry , I am not sure how to post from gist/pastebin/etc. Below are your requested details > > > > > > > > Operating system : Linux > > > > ES version : Elasticsearch-0.90.7 > > > > JAVA Version :JDK 1.7 > > > > Data : we didn't have any parent child relations. Just we are loading with auto generated primary key.(29,666,482 documents= 21.9 GB). > > > > > > > > Data Loading : we are loading data into elastic by writing 20 threads. > > > > > > > > Client creation : > > > > > > > > Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", "UCS\_DEV\_ES\_CLUSTER").build(); > > > > > > > > ``` > > > > elasticClient = new TransportClient(settings).addTransportAddress(new InetSocketTransportAddress("server1", 9300)) > > > > .addTransportAddress(new InetSocketTransportAddress("server2", 9300)); > > > > > > > > ``` > > > > > > > > Below are the server details > > > > > > > > we have 4 nodes. > > > > > > > > Each node contains > > > > CPU: 4 , RAM: 16G , Storage: 122G > > > > ES\_HEAP\_SIZE 8 G > > > > > > > > Please find enclosed elasticsearch.yml file which we are using. > > > > > > > > Thanks, > > > > Srikanth. > > > > > > > > > On Wed, Dec 18, 2013 at 5:02 PM, Mark Walkom [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) wrote: > > > > > It will help if you post (via a gist/pastebin/etc) what you have now. > > > > > > > > > > Also what sort of data is it, how are you loading it, what versions of things are you running on (OS, java, ES, etc etc) > > > > > > > > > > Regards, > > > > > Mark Walkom > > > > > > > > > > Infrastructure Engineer > > > > > Campaign Monitor > > > > > email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > > > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > > > > > > > > > > On 19 December 2013 11:53, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com) wrote: > > > > > > Hi Team, > > > > > > > > > > > > We have below server configurations. We are trying to load more than 500 million data into elastic. When data reaches 40 millions, shards are going down and data is deleting . Can you please help us with best configuration for above requirement. > > > > > > > > > > > > Below are the server details > > > > > > > > > > > > we have 4 nodes. > > > > > > > > > > > > Each node contains > > > > > > CPU: 4 , RAM: 16G , Storage: 122G > > > > > > ES\_HEAP\_SIZE 8 G > > > > > > > > > > > > Thanks, > > > > > > Srikanth. > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > > > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/436a22c0-7aca-4502-b229-db91c0ebce01%40googlegroups.com). > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe). > > > > > To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624YPB%3DkkJ3eOFRRc%2Bw%2BqG3junYQWpiD3EzPp677BLTbwvQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624YPB%3DkkJ3eOFRRc%2Bw%2BqG3junYQWpiD3EzPp677BLTbwvQ%40mail.gmail.com). > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGuNUAV5A\_Vj9k285nAW\_xW7UdnVr5K5n1DBndBLo%3D9KiQ8BaQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGuNUAV5A_Vj9k285nAW_xW7UdnVr5K5n1DBndBLo%3D9KiQ8BaQ%40mail.gmail.com). > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624b2JyfYmC5\_mZkz%2BXQe\_4hzrr79ALx3Hj0ww6%3DTeqHOHQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624b2JyfYmC5_mZkz%2BXQe_4hzrr79ALx3Hj0ww6%3DTeqHOHQ%40mail.gmail.com). > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGuNUAUNS3gwX-jNjz4\_PL7TyY3qG0d0zCb9Dm5WHsRpCRQQ3Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGuNUAUNS3gwX-jNjz4_PL7TyY3qG0d0zCb9Dm5WHsRpCRQQ3Q%40mail.gmail.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/QRGwAlqL-qY/unsubscribe). > To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624ZLXXZMzchGSewoDhK9J3kK0WZjqxw8Pth9g2E-%3Du29MQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624ZLXXZMzchGSewoDhK9J3kK0WZjqxw8Pth9g2E-%3Du29MQ%40mail.gmail.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGuNUAWyNOXr90qf%2BdUu9NqG1J%3DJwiDm-2yXJ5FcuPNy28oznA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGuNUAWyNOXr90qf%2BdUu9NqG1J%3DJwiDm-2yXJ5FcuPNy28oznA%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/29E8DADA-93DD-4A4A-BA22-AA3DEB6652A4%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/29E8DADA-93DD-4A4A-BA22-AA3DEB6652A4%40pilato.fr). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![shailendra\_pathak9](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shailendra_pathak9/32/1906_2.png) [@shailendra\_pathak9](https://discuss.elastic.co/u/shailendra_pathak9)\ **Post date:** [December 19, 2013, 10:44am UTC](https://discuss.elastic.co/t/request-for-elastic-search-configuration/14943/9 "2013-12-19T10:44:39Z")
Hi Srikanth,and all elasticsearch developers. i need your help regarding deploying ES server using chef solo..i have made one ES instance ..i have 7 servers .can you please provide full procedures for the same.. Thanks On Thursday, December 19, 2013 6:23:45 AM UTC+5:30, srikanth ramineni wrote: > Hi Team, > > We have below server configurations. We are trying to load more than 500 > million data into elastic. When data reaches 40 millions, shards are going > down and data is deleting . Can you please help us with best configuration > for above requirement. > > Below are the server details > > we have 4 nodes. > > Each node contains > CPU: 4 , RAM: 16G , Storage: 122G > ES\_HEAP\_SIZE 8 G > > Thanks, > Srikanth. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4e65841e-3edc-4a1a-b16a-dbf2c265a27c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4e65841e-3edc-4a1a-b16a-dbf2c265a27c%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 19, 2013, 6:47pm UTC](https://discuss.elastic.co/t/request-for-elastic-search-configuration/14943/10 "2013-12-19T18:47:32Z")
Are you looking for this? [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/tutorials/deploying-elasticsearch-with-chef-solo/) -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 19 déc. 2013 à 11:44, [shailendra.pathak9@gmail.com](mailto:shailendra.pathak9@gmail.com) a écrit : Hi Srikanth,and all elasticsearch developers. i need your help regarding deploying ES server using chef solo..i have made one ES instance ..i have 7 servers .can you please provide full procedures for the same.. Thanks > On Thursday, December 19, 2013 6:23:45 AM UTC+5:30, srikanth ramineni wrote: > Hi Team, > > We have below server configurations. We are trying to load more than 500 million data into elastic. When data reaches 40 millions, shards are going down and data is deleting . Can you please help us with best configuration for above requirement. > > Below are the server details > > we have 4 nodes. > > Each node contains > CPU: 4 , RAM: 16G , Storage: 122G > ES\_HEAP\_SIZE 8 G > > Thanks, > Srikanth. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4e65841e-3edc-4a1a-b16a-dbf2c265a27c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4e65841e-3edc-4a1a-b16a-dbf2c265a27c%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/BA543D32-8882-4978-86AA-0FBC16EF77C1%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/BA543D32-8882-4978-86AA-0FBC16EF77C1%40pilato.fr). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:00am UTC](https://discuss.elastic.co/t/request-for-elastic-search-configuration/14943/11 "2017-07-06T02:00:07Z")