Data per node in ES

Elastic Stack Elasticsearch
1

Hi ,

We are analyzing ES for storing our log data (~ 400 GB/Day) and will be
integrating Logstash and ES. What is the maximum amount of data that can
be stored on one node of ES ?

Regards,
Gaurav

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c1b5123f-51a7-41b4-9915-d4ea705d23de%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Depends.
How much disk do you have? RAM? CPU? Java version and release? ES version?
What's your query load like? Are you doing lots of aggregates or facets?

The best way to know is to start using ELK on an platform indicative of
your intended server size and then see how much data a single node can
handle, then extrapolate.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 27 August 2014 06:24, Gaurav Tiwari gtinside@gmail.com wrote:

Hi ,

We are analyzing ES for storing our log data (~ 400 GB/Day) and will be
integrating Logstash and ES. What is the maximum amount of data that can
be stored on one node of ES ?

Regards,
Gaurav

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c1b5123f-51a7-41b4-9915-d4ea705d23de%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c1b5123f-51a7-41b4-9915-d4ea705d23de%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624YwMUZhTQ4Bt-DD67CQ3Z2ykA83ZA4GmyH_yjtBLAeZPA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

We are planning to use >1 TB SSD for our nodes, typical node machine with
24 core CPU and ~ 200 GB RAM. Our ELK stack : Elastic Search v1.3.2 ,
logstash 1.4.1 with Java 1.7 and Kibana 3.0

Gaurav

On Tue, Aug 26, 2014 at 7:28 PM, Mark Walkom markw@campaignmonitor.com
wrote:

Depends.
How much disk do you have? RAM? CPU? Java version and release? ES version?
What's your query load like? Are you doing lots of aggregates or facets?

The best way to know is to start using ELK on an platform indicative of
your intended server size and then see how much data a single node can
handle, then extrapolate.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 27 August 2014 06:24, Gaurav Tiwari gtinside@gmail.com wrote:

Hi ,

We are analyzing ES for storing our log data (~ 400 GB/Day) and will be
integrating Logstash and ES. What is the maximum amount of data that can
be stored on one node of ES ?

Regards,
Gaurav

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c1b5123f-51a7-41b4-9915-d4ea705d23de%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c1b5123f-51a7-41b4-9915-d4ea705d23de%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/sjyJBROX2c8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEM624YwMUZhTQ4Bt-DD67CQ3Z2ykA83ZA4GmyH_yjtBLAeZPA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAEM624YwMUZhTQ4Bt-DD67CQ3Z2ykA83ZA4GmyH_yjtBLAeZPA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAFYfToeV0RmVpZxa-4Cqew6iCgaf%2BwNHgzj6hiJg5qBP6qPeXA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.