# Elasticsearch hardware requirement **URL:** \ **Category:** Elasticsearch\ **Created:** [March 28, 2014, 8:23am UTC](https://discuss.elastic.co/t/elasticsearch-hardware-requirement/16670 "2014-03-28T08:23:25Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Jorge\_Roman](https://avatars.discourse-cdn.com/v4/letter/j/5f9b8f/32.png) [@Jorge\_Roman](https://discuss.elastic.co/u/Jorge_Roman)\ **Post date:** [March 28, 2014, 8:23am UTC](https://discuss.elastic.co/t/elasticsearch-hardware-requirement/16670/1 "2014-03-28T08:23:25Z")
Hi list, I currently have a 4 nodes cluster to collect varnish logs of my platform, The cluster is dealing with almost 80Mill of documents (2000 documents per second) and sometime I have some performance problems. Let me explain how i do it. In all frontend server i have a logstash agent sending Varnish logs to a redis instance in one of my elasticsearch machines. In that machine other logstash process pull the events and store it in the elasticsearch cluster. My problem come when the logs begin to queue in redis because elasticsearch cluster is not able to store so fast. The cluster machines (virtual machines) have 4 CPU and 4GB of RAM and I think are not enough to deal with 2000-3000 events per second. I have been reading a lot about this issue and everybody have machines bigger than mine. What would be the right size for the machines? I have an index per day which size is almost 100GB, and I've read that the size of your RAM should be the same that your indexes size, but that is not possible for me. If I add more nodes to the cluster the performance will be better? Thanks in advance! Jorge -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d21ce426-7612-47a1-9da0-56441d544892%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d21ce426-7612-47a1-9da0-56441d544892%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Binh\_Ly\_2](https://avatars.discourse-cdn.com/v4/letter/b/d07c76/32.png) [@Binh\_Ly\_2](https://discuss.elastic.co/u/Binh_Ly_2)\ **Post date:** [March 28, 2014, 2:06pm UTC](https://discuss.elastic.co/t/elasticsearch-hardware-requirement/16670/2 "2014-03-28T14:06:18Z")
The best way is to test it. Take 1 node with say 16GB of RAM and allocate 8GB to ES. Then start pushing 1 day worth of logs into an index with 5 shards and 0 replicas on that one node AND run your typical queries. Take measurements like throughput/dps, query latency, ram usage, cpu, and disk. Then you'll know how much you can do on a single node and then start extrapolating from there. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9ee91a2d-2941-4327-82f0-2793eb1cb242%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9ee91a2d-2941-4327-82f0-2793eb1cb242%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Jorge\_Roman](https://avatars.discourse-cdn.com/v4/letter/j/5f9b8f/32.png) [@Jorge\_Roman](https://discuss.elastic.co/u/Jorge_Roman)\ **Post date:** [April 2, 2014, 8:32am UTC](https://discuss.elastic.co/t/elasticsearch-hardware-requirement/16670/3 "2014-04-02T08:32:46Z")
Thanks for the reply. I have done the test with 1 node (16GB RAM and 8CPUs, allocating 8GB to ES), and I have been able to deal with all events with only 1 node. Now i¹m trying to find out where is the bottleneck. Next step, I¹m gonna try to benchmarking elasticsearch without external elements in order to take measures. Best Regards! Jorge Román Novalbos CEO [jroman@servotic.com](mailto:jroman@servotic.com) 679 99 08 62 [http://www.linkedin.com/in/jorgeromanwebperformance](http://www.linkedin.com/in/jorgeromanwebperformance) [https://twitter.com/servoticsl](https://twitter.com/servoticsl)[http://www.facebook.com/servotic](http://www.facebook.com/servotic) skype:jorgeroman1980?call [http://www.servotic.com](http://www.servotic.com) Este mensaje es solamente para la persona a la que va dirigido. Puede contener información confidencial o legalmente protegida. Si usted ha recibido este mensaje por error, le rogamos que borre de su sistema el mensaje inmediatamente y notifíquelo al remitente. No debe, directa o indirectamente, usar, revelar, distribuir, imprimir o copiar ninguna de las partes de este mensaje si no es usted el destinatario. En cumplimiento de la Ley Orgánica 15/1999, de Protección de Datos de Carácter Personal le informamos que su dirección de correo electrónico, sus datos personales y de empresa pasarán a formar parte de nuestro fichero de Clientes y Proveedores, registrado ante la Agencia de Protección de Datos. En cumplimiento de la Ley 34/2002, de Servicios de la Sociedad de la Información y el Comercio Electrónico, le informamos que esta dirección de correo electrónico podrá ser utilizada para el envío de información comercial o promocional de nuestra organización. Si no desea recibir información o desea ejercitar sus derechos de acceso, rectificación, cancelación y oposición, le rogamos nos lo comunique vía correo electrónico a la siguiente dirección: [lopd@servotic.com](mailto:lopd@servotic.com) De: Binh Ly [binhly\_es@yahoo.com](mailto:binhly_es@yahoo.com) Responder a: [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) Fecha: Fri, 28 Mar 2014 07:06:18 -0700 (PDT) Para: [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) Asunto: Re: Elasticsearch hardware requirement The best way is to test it. Take 1 node with say 16GB of RAM and allocate 8GB to ES. Then start pushing 1 day worth of logs into an index with 5 shards and 0 replicas on that one node AND run your typical queries. Take measurements like throughput/dps, query latency, ram usage, cpu, and disk. Then you'll know how much you can do on a single node and then start extrapolating from there. -- You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/tVywigD5iU8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/tVywigD5iU8/unsubscribe). To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9ee91a2d-2941-4327-82f0-2793](https://groups.google.com/d/msgid/elasticsearch/9ee91a2d-2941-4327-82f0-2793) eb1cb242%[40googlegroups.com](http://40googlegroups.com) \<[https://groups.google.com/d/msgid/elasticsearch/9ee91a2d-2941-4327-82f0-279](https://groups.google.com/d/msgid/elasticsearch/9ee91a2d-2941-4327-82f0-279) 3eb1cb242%[40googlegroups.com?utm\_medium=email&utm\_source=footer](http://40googlegroups.com?utm_medium=email&utm_source=footer)\> . For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CF619208.1737D%jroman%40servotic.com](https://groups.google.com/d/msgid/elasticsearch/CF619208.1737D%25jroman%40servotic.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:38am UTC](https://discuss.elastic.co/t/elasticsearch-hardware-requirement/16670/4 "2017-07-06T01:38:49Z")