# ES JVM memory usage consistently above 90%
**URL:** https://discuss.elastic.co/t/es-jvm-memory-usage-consistently-above-90/22800
**Category:** Elasticsearch
**Created:** [March 22, 2015, 7:30am UTC](https://discuss.elastic.co/t/es-jvm-memory-usage-consistently-above-90/22800 "2015-03-22T07:30:57Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@Yogesh\_Kansal](https://discuss.elastic.co/u/Yogesh_Kansal)
#### Post date: [March 22, 2015, 7:30am UTC](https://discuss.elastic.co/t/es-jvm-memory-usage-consistently-above-90/22800/1 "2015-03-22T07:30:57Z")
Hi,
I have set up elasticsearch on one node and am using the Twitter river to
index tweets. It has been going fine with almost 50M tweets indexed so far
in 13 days.
When I started indexing, the JVM usage (observed via Marvel) hovered
between 10-20%, then started remaining around 30-40% but for the past 3-4
days it has continuously been above 90%, reaching 99% at times!
I restarted elasticsearch thinking it might get resolved but as soon as I
switched it back on, the JVM usage went back to 90%.
Why is this happening and how can I remedy it? (The JVM memory is the
default 990.75MB)
Thanks
Yogesh
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/78d4c994-db38-40ce-8adf-db5b43adae2b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/78d4c994-db38-40ce-8adf-db5b43adae2b%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@jpotisch](https://discuss.elastic.co/u/jpotisch)
#### Post date: [March 23, 2015, 1:12pm UTC](https://discuss.elastic.co/t/es-jvm-memory-usage-consistently-above-90/22800/2 "2015-03-23T13:12:04Z")
Hi Yogesh. I am not 100% sure of this, so if someone else posts a reply
that differs from mine you should probably go with theirs, but I think this
is correct behavior.
Unless there is another process demanding memory from the OS, there is no
harm whatsoever in having the JVM consume all available RAM. It allows
Elasticsearch to put more things in memory, serve more cache hits, etc. You
definitely do not want the JVM to constantly run garbage collection so that
it can return memory to the OS that it's just going to want to request
again as soon as the system gets busy.
You should make sure you are following recommended memory practices in
[Heap: Sizing and Swapping | Elasticsearch: The Definitive Guide [2.x] | Elastic](http://www.elastic.co/guide/en/elasticsearch/guide/current/heap-sizing.html).
Hope that helps (and is accurate)!
-joel
On Sunday, March 22, 2015 at 3:30:57 AM UTC-4, Yogesh wrote:
> Hi,
>
> I have set up elasticsearch on one node and am using the Twitter river to
> index tweets. It has been going fine with almost 50M tweets indexed so far
> in 13 days.
> When I started indexing, the JVM usage (observed via Marvel) hovered
> between 10-20%, then started remaining around 30-40% but for the past 3-4
> days it has continuously been above 90%, reaching 99% at times!
> I restarted elasticsearch thinking it might get resolved but as soon as I
> switched it back on, the JVM usage went back to 90%.
>
> Why is this happening and how can I remedy it? (The JVM memory is the
> default 990.75MB)
>
> Thanks
> Yogesh
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0dd31898-83f8-414a-a85c-a8d5f1a64d4b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0dd31898-83f8-414a-a85c-a8d5f1a64d4b%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@mjdude5](https://discuss.elastic.co/u/mjdude5)
#### Post date: [March 23, 2015, 3:55pm UTC](https://discuss.elastic.co/t/es-jvm-memory-usage-consistently-above-90/22800/3 "2015-03-23T15:55:40Z")
Are you saying JVM is using 99% of the system memory or 99% of the heap?
If it's 99% of the available heap that's bad and you will have cluster
instability. I suggest increasing your JVM heap size if you can, I can't
find it right now but I remember a blog post that used twitter as a
benchmark and they also could get to ~50M documents with the default 1G
heap.
On Sunday, March 22, 2015 at 3:30:57 AM UTC-4, Yogesh wrote:
> Hi,
>
> I have set up elasticsearch on one node and am using the Twitter river to
> index tweets. It has been going fine with almost 50M tweets indexed so far
> in 13 days.
> When I started indexing, the JVM usage (observed via Marvel) hovered
> between 10-20%, then started remaining around 30-40% but for the past 3-4
> days it has continuously been above 90%, reaching 99% at times!
> I restarted elasticsearch thinking it might get resolved but as soon as I
> switched it back on, the JVM usage went back to 90%.
>
> Why is this happening and how can I remedy it? (The JVM memory is the
> default 990.75MB)
>
> Thanks
> Yogesh
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@Yogesh\_Kansal](https://discuss.elastic.co/u/Yogesh_Kansal)
#### Post date: [March 24, 2015, 5:01pm UTC](https://discuss.elastic.co/t/es-jvm-memory-usage-consistently-above-90/22800/4 "2015-03-24T17:01:03Z")
Thanks Joel and mjdude. What I mean is that ES is using 99% of the heap
memory (I think since Marvel showed memory as 1 GB which corresponds to the
heap, my RAM is 50GB)
I've increased the ES\_HEAP\_SIZE to 10g. But another problem has appeared
and I'm freaked out because of it!
So, after restarting my ES (curl shutdown) to increase the heap, the Marvel
has stopped showing me my data (it still shows that disk memory is lower so
that means the data is still on disk) and upon searching Sense shows
"IndexMissingException[[my\_new\_twitter\_river] missing]"
Why is this happening?!?!
On Mon, Mar 23, 2015 at 9:25 PM, [mjdude5@gmail.com](mailto:mjdude5@gmail.com) wrote:
> Are you saying JVM is using 99% of the system memory or 99% of the heap?
> If it's 99% of the available heap that's bad and you will have cluster
> instability. I suggest increasing your JVM heap size if you can, I can't
> find it right now but I remember a blog post that used twitter as a
> benchmark and they also could get to ~50M documents with the default 1G
> heap.
>
> On Sunday, March 22, 2015 at 3:30:57 AM UTC-4, Yogesh wrote:
>
> > Hi,
> >
> > I have set up elasticsearch on one node and am using the Twitter river to
> > index tweets. It has been going fine with almost 50M tweets indexed so far
> > in 13 days.
> > When I started indexing, the JVM usage (observed via Marvel) hovered
> > between 10-20%, then started remaining around 30-40% but for the past 3-4
> > days it has continuously been above 90%, reaching 99% at times!
> > I restarted elasticsearch thinking it might get resolved but as soon as I
> > switched it back on, the JVM usage went back to 90%.
> >
> > Why is this happening and how can I remedy it? (The JVM memory is the
> > default 990.75MB)
> >
> > Thanks
> > Yogesh
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> [https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe](https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe).
> To unsubscribe from this group and all its topics, send an email to
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> To view this discussion on the web visit
> [https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com)
> [https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm_medium=email&utm_source=footer)
> .
>
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CADM0w%3Dh%3DMgfP%3DDSR0PmhVgXAGcApMDxZE%3D4\_jVJXbRwk8CzTHw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CADM0w%3Dh%3DMgfP%3DDSR0PmhVgXAGcApMDxZE%3D4_jVJXbRwk8CzTHw%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@mjdude5](https://discuss.elastic.co/u/mjdude5)
#### Post date: [March 24, 2015, 5:32pm UTC](https://discuss.elastic.co/t/es-jvm-memory-usage-consistently-above-90/22800/5 "2015-03-24T17:32:33Z")
When it restarted did it attach to the wrong data directory? Take a look
at \_nodes/\_local/stats?pretty and check the 'data' directory location. Has
the cluster recovered after the restart? Check \_cluster/health?pretty as
well.
On Tuesday, March 24, 2015 at 1:01:52 PM UTC-4, Yogesh wrote:
> Thanks Joel and mjdude. What I mean is that ES is using 99% of the heap
> memory (I think since Marvel showed memory as 1 GB which corresponds to the
> heap, my RAM is 50GB)
> I've increased the ES\_HEAP\_SIZE to 10g. But another problem has appeared
> and I'm freaked out because of it!
>
> So, after restarting my ES (curl shutdown) to increase the heap, the
> Marvel has stopped showing me my data (it still shows that disk memory is
> lower so that means the data is still on disk) and upon searching Sense
> shows "IndexMissingException[[my\_new\_twitter\_river] missing]"
>
> Why is this happening?!?!
>
> On Mon, Mar 23, 2015 at 9:25 PM, \<[mjd...@gmail.com](mailto:mjd...@gmail.com) \\> wrote:
>
> > Are you saying JVM is using 99% of the system memory or 99% of the heap?
> > If it's 99% of the available heap that's bad and you will have cluster
> > instability. I suggest increasing your JVM heap size if you can, I can't
> > find it right now but I remember a blog post that used twitter as a
> > benchmark and they also could get to ~50M documents with the default 1G
> > heap.
> >
> > On Sunday, March 22, 2015 at 3:30:57 AM UTC-4, Yogesh wrote:
> >
> > > Hi,
> > >
> > > I have set up elasticsearch on one node and am using the Twitter river
> > > to index tweets. It has been going fine with almost 50M tweets indexed so
> > > far in 13 days.
> > > When I started indexing, the JVM usage (observed via Marvel) hovered
> > > between 10-20%, then started remaining around 30-40% but for the past 3-4
> > > days it has continuously been above 90%, reaching 99% at times!
> > > I restarted elasticsearch thinking it might get resolved but as soon as
> > > I switched it back on, the JVM usage went back to 90%.
> > >
> > > Why is this happening and how can I remedy it? (The JVM memory is the
> > > default 990.75MB)
> > >
> > > Thanks
> > > Yogesh
> >
> > --
> > You received this message because you are subscribed to a topic in the
> > Google Groups "elasticsearch" group.
> > To unsubscribe from this topic, visit
> > [https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe](https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe).
> > To unsubscribe from this group and all its topics, send an email to
> > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com)
> > [https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
> >
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@Yogesh\_Kansal](https://discuss.elastic.co/u/Yogesh_Kansal)
#### Post date: [March 24, 2015, 5:55pm UTC](https://discuss.elastic.co/t/es-jvm-memory-usage-consistently-above-90/22800/6 "2015-03-24T17:55:56Z")
Thanks a lot mjdude! It does seem like it attached to the wrong data
directory.
In elasticsearch/data/tool/nodes there are two 0 and 1. My data is in 0 but
node stats shows the data directory as elasticsearch/data/tool/nodes/1.
Now, how do I change this?
On Tue, Mar 24, 2015 at 11:02 PM, [mjdude5@gmail.com](mailto:mjdude5@gmail.com) wrote:
> When it restarted did it attach to the wrong data directory? Take a look
> at \_nodes/\_local/stats?pretty and check the 'data' directory location. Has
> the cluster recovered after the restart? Check \_cluster/health?pretty as
> well.
>
> On Tuesday, March 24, 2015 at 1:01:52 PM UTC-4, Yogesh wrote:
>
> > Thanks Joel and mjdude. What I mean is that ES is using 99% of the heap
> > memory (I think since Marvel showed memory as 1 GB which corresponds to the
> > heap, my RAM is 50GB)
> > I've increased the ES\_HEAP\_SIZE to 10g. But another problem has appeared
> > and I'm freaked out because of it!
> >
> > So, after restarting my ES (curl shutdown) to increase the heap, the
> > Marvel has stopped showing me my data (it still shows that disk memory is
> > lower so that means the data is still on disk) and upon searching Sense
> > shows "IndexMissingException[[my\_new\_twitter\_river] missing]"
> >
> > Why is this happening?!?!
> >
> > On Mon, Mar 23, 2015 at 9:25 PM, [mjd...@gmail.com](mailto:mjd...@gmail.com) wrote:
> >
> > > Are you saying JVM is using 99% of the system memory or 99% of the
> > > heap? If it's 99% of the available heap that's bad and you will have
> > > cluster instability. I suggest increasing your JVM heap size if you can, I
> > > can't find it right now but I remember a blog post that used twitter as a
> > > benchmark and they also could get to ~50M documents with the default 1G
> > > heap.
> > >
> > > On Sunday, March 22, 2015 at 3:30:57 AM UTC-4, Yogesh wrote:
> > >
> > > > Hi,
> > > >
> > > > I have set up elasticsearch on one node and am using the Twitter river
> > > > to index tweets. It has been going fine with almost 50M tweets indexed so
> > > > far in 13 days.
> > > > When I started indexing, the JVM usage (observed via Marvel) hovered
> > > > between 10-20%, then started remaining around 30-40% but for the past 3-4
> > > > days it has continuously been above 90%, reaching 99% at times!
> > > > I restarted elasticsearch thinking it might get resolved but as soon as
> > > > I switched it back on, the JVM usage went back to 90%.
> > > >
> > > > Why is this happening and how can I remedy it? (The JVM memory is the
> > > > default 990.75MB)
> > > >
> > > > Thanks
> > > > Yogesh
> > >
> > > --
> > > You received this message because you are subscribed to a topic in the
> > > Google Groups "elasticsearch" group.
> > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > topic/elasticsearch/kMSwBqpe2N4/unsubscribe.
> > > To unsubscribe from this group and all its topics, send an email to
> > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%
> > > [40googlegroups.com](http://40googlegroups.com)
> > > [https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > .
> > >
> > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> >
> > --
> > You received this message because you are subscribed to a topic in the
> > Google Groups "elasticsearch" group.
> > To unsubscribe from this topic, visit
> > [https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe](https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe).
> > To unsubscribe from this group and all its topics, send an email to
> > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com)
> > [https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
>
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CADM0w%3DhB3Qxs2a01yLtcLPaKUodOh2jimniow76fPLZ2oz12aQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CADM0w%3DhB3Qxs2a01yLtcLPaKUodOh2jimniow76fPLZ2oz12aQ%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@mjdude5](https://discuss.elastic.co/u/mjdude5)
#### Post date: [March 24, 2015, 6:02pm UTC](https://discuss.elastic.co/t/es-jvm-memory-usage-consistently-above-90/22800/7 "2015-03-24T18:02:29Z")
Sometimes that happens when the new node starts up via monit or other
automated thing before the old node is fully shutdown. I'd suggest
shutting down the node and verify it's done via ps before allowing the new
node to start up. In the case of monit if the check is hitting the http
port then it'll think it's down before it actually fully quits.
On Tuesday, March 24, 2015 at 1:56:54 PM UTC-4, Yogesh wrote:
> Thanks a lot mjdude! It does seem like it attached to the wrong data
> directory.
> In elasticsearch/data/tool/nodes there are two 0 and 1. My data is in 0
> but node stats shows the data directory as elasticsearch/data/tool/nodes/1.
> Now, how do I change this?
>
> On Tue, Mar 24, 2015 at 11:02 PM, \<[mjd...@gmail.com](mailto:mjd...@gmail.com) \\> wrote:
>
> > When it restarted did it attach to the wrong data directory? Take a look
> > at \_nodes/\_local/stats?pretty and check the 'data' directory location. Has
> > the cluster recovered after the restart? Check \_cluster/health?pretty as
> > well.
> >
> > On Tuesday, March 24, 2015 at 1:01:52 PM UTC-4, Yogesh wrote:
> >
> > > Thanks Joel and mjdude. What I mean is that ES is using 99% of the heap
> > > memory (I think since Marvel showed memory as 1 GB which corresponds to the
> > > heap, my RAM is 50GB)
> > > I've increased the ES\_HEAP\_SIZE to 10g. But another problem has appeared
> > > and I'm freaked out because of it!
> > >
> > > So, after restarting my ES (curl shutdown) to increase the heap, the
> > > Marvel has stopped showing me my data (it still shows that disk memory is
> > > lower so that means the data is still on disk) and upon searching Sense
> > > shows "IndexMissingException[[my\_new\_twitter\_river] missing]"
> > >
> > > Why is this happening?!?!
> > >
> > > On Mon, Mar 23, 2015 at 9:25 PM, [mjd...@gmail.com](mailto:mjd...@gmail.com) wrote:
> > >
> > > > Are you saying JVM is using 99% of the system memory or 99% of the
> > > > heap? If it's 99% of the available heap that's bad and you will have
> > > > cluster instability. I suggest increasing your JVM heap size if you can, I
> > > > can't find it right now but I remember a blog post that used twitter as a
> > > > benchmark and they also could get to ~50M documents with the default 1G
> > > > heap.
> > > >
> > > > On Sunday, March 22, 2015 at 3:30:57 AM UTC-4, Yogesh wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I have set up elasticsearch on one node and am using the Twitter river
> > > > > to index tweets. It has been going fine with almost 50M tweets indexed so
> > > > > far in 13 days.
> > > > > When I started indexing, the JVM usage (observed via Marvel) hovered
> > > > > between 10-20%, then started remaining around 30-40% but for the past 3-4
> > > > > days it has continuously been above 90%, reaching 99% at times!
> > > > > I restarted elasticsearch thinking it might get resolved but as soon
> > > > > as I switched it back on, the JVM usage went back to 90%.
> > > > >
> > > > > Why is this happening and how can I remedy it? (The JVM memory is the
> > > > > default 990.75MB)
> > > > >
> > > > > Thanks
> > > > > Yogesh
> > > >
> > > > --
> > > > You received this message because you are subscribed to a topic in the
> > > > Google Groups "elasticsearch" group.
> > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > topic/elasticsearch/kMSwBqpe2N4/unsubscribe.
> > > > To unsubscribe from this group and all its topics, send an email to
> > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%
> > > > [40googlegroups.com](http://40googlegroups.com)
> > > > [https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > .
> > > >
> > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > >
> > > --
> > > You received this message because you are subscribed to a topic in the
> > > Google Groups "elasticsearch" group.
> > > To unsubscribe from this topic, visit
> > > [https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe](https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe).
> > > To unsubscribe from this group and all its topics, send an email to
> > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > > To view this discussion on the web visit
> > > [https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com)
> > > [https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > .
> >
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@Yogesh\_Kansal](https://discuss.elastic.co/u/Yogesh_Kansal)
#### Post date: [March 24, 2015, 6:15pm UTC](https://discuss.elastic.co/t/es-jvm-memory-usage-consistently-above-90/22800/8 "2015-03-24T18:15:59Z")
Thanks. Mine is a one node cluster so I am simply using the XCURL to shut
it down and the doing "bin/elasticsearch -d" to start it up.
To check if it has shutdown, I try to hit it using http. So, now how do I
start it with the node/0 data directory?
There is nothing there in node/1 data directory but I don't suppose
deleting it would be the solution? (Sorry for the basic questions I am new
to this!)
On Tue, Mar 24, 2015 at 11:32 PM, [mjdude5@gmail.com](mailto:mjdude5@gmail.com) wrote:
> Sometimes that happens when the new node starts up via monit or other
> automated thing before the old node is fully shutdown. I'd suggest
> shutting down the node and verify it's done via ps before allowing the new
> node to start up. In the case of monit if the check is hitting the http
> port then it'll think it's down before it actually fully quits.
>
> On Tuesday, March 24, 2015 at 1:56:54 PM UTC-4, Yogesh wrote:
>
> > Thanks a lot mjdude! It does seem like it attached to the wrong data
> > directory.
> > In elasticsearch/data/tool/nodes there are two 0 and 1. My data is in 0
> > but node stats shows the data directory as elasticsearch/data/tool/nodes/
> > 1.
> > Now, how do I change this?
> >
> > On Tue, Mar 24, 2015 at 11:02 PM, [mjd...@gmail.com](mailto:mjd...@gmail.com) wrote:
> >
> > > When it restarted did it attach to the wrong data directory? Take a
> > > look at \_nodes/\_local/stats?pretty and check the 'data' directory
> > > location. Has the cluster recovered after the restart? Check
> > > \_cluster/health?pretty as well.
> > >
> > > On Tuesday, March 24, 2015 at 1:01:52 PM UTC-4, Yogesh wrote:
> > >
> > > > Thanks Joel and mjdude. What I mean is that ES is using 99% of the heap
> > > > memory (I think since Marvel showed memory as 1 GB which corresponds to the
> > > > heap, my RAM is 50GB)
> > > > I've increased the ES\_HEAP\_SIZE to 10g. But another problem has
> > > > appeared and I'm freaked out because of it!
> > > >
> > > > So, after restarting my ES (curl shutdown) to increase the heap, the
> > > > Marvel has stopped showing me my data (it still shows that disk memory is
> > > > lower so that means the data is still on disk) and upon searching Sense
> > > > shows "IndexMissingException[[my\_new\_twitter\_river] missing]"
> > > >
> > > > Why is this happening?!?!
> > > >
> > > > On Mon, Mar 23, 2015 at 9:25 PM, [mjd...@gmail.com](mailto:mjd...@gmail.com) wrote:
> > > >
> > > > > Are you saying JVM is using 99% of the system memory or 99% of the
> > > > > heap? If it's 99% of the available heap that's bad and you will have
> > > > > cluster instability. I suggest increasing your JVM heap size if you can, I
> > > > > can't find it right now but I remember a blog post that used twitter as a
> > > > > benchmark and they also could get to ~50M documents with the default 1G
> > > > > heap.
> > > > >
> > > > > On Sunday, March 22, 2015 at 3:30:57 AM UTC-4, Yogesh wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I have set up elasticsearch on one node and am using the Twitter
> > > > > > river to index tweets. It has been going fine with almost 50M tweets
> > > > > > indexed so far in 13 days.
> > > > > > When I started indexing, the JVM usage (observed via Marvel) hovered
> > > > > > between 10-20%, then started remaining around 30-40% but for the past 3-4
> > > > > > days it has continuously been above 90%, reaching 99% at times!
> > > > > > I restarted elasticsearch thinking it might get resolved but as soon
> > > > > > as I switched it back on, the JVM usage went back to 90%.
> > > > > >
> > > > > > Why is this happening and how can I remedy it? (The JVM memory is the
> > > > > > default 990.75MB)
> > > > > >
> > > > > > Thanks
> > > > > > Yogesh
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to a topic in the
> > > > > Google Groups "elasticsearch" group.
> > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/to](https://groups.google.com/d/to)
> > > > > pic/elasticsearch/kMSwBqpe2N4/unsubscribe.
> > > > > To unsubscribe from this group and all its topics, send an email to
> > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40goo
> > > > > [glegroups.com](http://glegroups.com)
> > > > > [https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > > .
> > > > >
> > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > > >
> > > > --
> > > > You received this message because you are subscribed to a topic in the
> > > > Google Groups "elasticsearch" group.
> > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > topic/elasticsearch/kMSwBqpe2N4/unsubscribe.
> > > > To unsubscribe from this group and all its topics, send an email to
> > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%
> > > > [40googlegroups.com](http://40googlegroups.com)
> > > > [https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > .
> > >
> > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> >
> > --
> > You received this message because you are subscribed to a topic in the
> > Google Groups "elasticsearch" group.
> > To unsubscribe from this topic, visit
> > [https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe](https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe).
> > To unsubscribe from this group and all its topics, send an email to
> > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%40googlegroups.com)
> > [https://groups.google.com/d/msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
>
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CADM0w%3DhfUE7RyW6ua1VPJVHq014%2BUxD%3D7BeQaQE34B7%2BJuBrxA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CADM0w%3DhfUE7RyW6ua1VPJVHq014%2BUxD%3D7BeQaQE34B7%2BJuBrxA%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@mjdude5](https://discuss.elastic.co/u/mjdude5)
#### Post date: [March 24, 2015, 8:06pm UTC](https://discuss.elastic.co/t/es-jvm-memory-usage-consistently-above-90/22800/9 "2015-03-24T20:06:51Z")
As long as it's the first ES instance starting on that node it'll grab 0
instead of 1. I don't know if you can explicitly set the data node
directory in the config.
On Tuesday, March 24, 2015 at 2:16:57 PM UTC-4, Yogesh wrote:
> Thanks. Mine is a one node cluster so I am simply using the XCURL to shut
> it down and the doing "bin/elasticsearch -d" to start it up.
> To check if it has shutdown, I try to hit it using http. So, now how do I
> start it with the node/0 data directory?
> There is nothing there in node/1 data directory but I don't suppose
> deleting it would be the solution? (Sorry for the basic questions I am new
> to this!)
>
> On Tue, Mar 24, 2015 at 11:32 PM, \<[mjd...@gmail.com](mailto:mjd...@gmail.com) \\> wrote:
>
> > Sometimes that happens when the new node starts up via monit or other
> > automated thing before the old node is fully shutdown. I'd suggest
> > shutting down the node and verify it's done via ps before allowing the new
> > node to start up. In the case of monit if the check is hitting the http
> > port then it'll think it's down before it actually fully quits.
> >
> > On Tuesday, March 24, 2015 at 1:56:54 PM UTC-4, Yogesh wrote:
> >
> > > Thanks a lot mjdude! It does seem like it attached to the wrong data
> > > directory.
> > > In elasticsearch/data/tool/nodes there are two 0 and 1. My data is in 0
> > > but node stats shows the data directory as elasticsearch/data/tool/nodes/
> > > 1.
> > > Now, how do I change this?
> > >
> > > On Tue, Mar 24, 2015 at 11:02 PM, [mjd...@gmail.com](mailto:mjd...@gmail.com) wrote:
> > >
> > > > When it restarted did it attach to the wrong data directory? Take a
> > > > look at \_nodes/\_local/stats?pretty and check the 'data' directory
> > > > location. Has the cluster recovered after the restart? Check
> > > > \_cluster/health?pretty as well.
> > > >
> > > > On Tuesday, March 24, 2015 at 1:01:52 PM UTC-4, Yogesh wrote:
> > > >
> > > > > Thanks Joel and mjdude. What I mean is that ES is using 99% of the
> > > > > heap memory (I think since Marvel showed memory as 1 GB which corresponds
> > > > > to the heap, my RAM is 50GB)
> > > > > I've increased the ES\_HEAP\_SIZE to 10g. But another problem has
> > > > > appeared and I'm freaked out because of it!
> > > > >
> > > > > So, after restarting my ES (curl shutdown) to increase the heap, the
> > > > > Marvel has stopped showing me my data (it still shows that disk memory is
> > > > > lower so that means the data is still on disk) and upon searching Sense
> > > > > shows "IndexMissingException[[my\_new\_twitter\_river] missing]"
> > > > >
> > > > > Why is this happening?!?!
> > > > >
> > > > > On Mon, Mar 23, 2015 at 9:25 PM, [mjd...@gmail.com](mailto:mjd...@gmail.com) wrote:
> > > > >
> > > > > > Are you saying JVM is using 99% of the system memory or 99% of the
> > > > > > heap? If it's 99% of the available heap that's bad and you will have
> > > > > > cluster instability. I suggest increasing your JVM heap size if you can, I
> > > > > > can't find it right now but I remember a blog post that used twitter as a
> > > > > > benchmark and they also could get to ~50M documents with the default 1G
> > > > > > heap.
> > > > > >
> > > > > > On Sunday, March 22, 2015 at 3:30:57 AM UTC-4, Yogesh wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I have set up elasticsearch on one node and am using the Twitter
> > > > > > > river to index tweets. It has been going fine with almost 50M tweets
> > > > > > > indexed so far in 13 days.
> > > > > > > When I started indexing, the JVM usage (observed via Marvel) hovered
> > > > > > > between 10-20%, then started remaining around 30-40% but for the past 3-4
> > > > > > > days it has continuously been above 90%, reaching 99% at times!
> > > > > > > I restarted elasticsearch thinking it might get resolved but as soon
> > > > > > > as I switched it back on, the JVM usage went back to 90%.
> > > > > > >
> > > > > > > Why is this happening and how can I remedy it? (The JVM memory is
> > > > > > > the default 990.75MB)
> > > > > > >
> > > > > > > Thanks
> > > > > > > Yogesh
> > > > > >
> > > > > > --
> > > > > > You received this message because you are subscribed to a topic in
> > > > > > the Google Groups "elasticsearch" group.
> > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/to](https://groups.google.com/d/to)
> > > > > > pic/elasticsearch/kMSwBqpe2N4/unsubscribe.
> > > > > > To unsubscribe from this group and all its topics, send an email to
> > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > > msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40goo
> > > > > > [glegroups.com](http://glegroups.com)
> > > > > > [https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > > > .
> > > > > >
> > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to a topic in the
> > > > > Google Groups "elasticsearch" group.
> > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > topic/elasticsearch/kMSwBqpe2N4/unsubscribe.
> > > > > To unsubscribe from this group and all its topics, send an email to
> > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%
> > > > > [40googlegroups.com](http://40googlegroups.com)
> > > > > [https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > > .
> > > >
> > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > >
> > > --
> > > You received this message because you are subscribed to a topic in the
> > > Google Groups "elasticsearch" group.
> > > To unsubscribe from this topic, visit
> > > [https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe](https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe).
> > > To unsubscribe from this group and all its topics, send an email to
> > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > > To view this discussion on the web visit
> > > [https://groups.google.com/d/msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%40googlegroups.com)
> > > [https://groups.google.com/d/msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > .
> >
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/5c3bf7fb-7a1d-4467-a2df-bee4db0293c8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/5c3bf7fb-7a1d-4467-a2df-bee4db0293c8%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@Yogesh\_Kansal](https://discuss.elastic.co/u/Yogesh_Kansal)
#### Post date: [March 24, 2015, 8:10pm UTC](https://discuss.elastic.co/t/es-jvm-memory-usage-consistently-above-90/22800/10 "2015-03-24T20:10:26Z")
Thanks. Will try starting again.
By the way, what will be the impact if I just delete the node/1 (then the
node.max\_local\_storage\_nodes to 1 and start elasticsearch) ?
On Wed, Mar 25, 2015 at 1:36 AM, [mjdude5@gmail.com](mailto:mjdude5@gmail.com) wrote:
> As long as it's the first ES instance starting on that node it'll grab 0
> instead of 1. I don't know if you can explicitly set the data node
> directory in the config.
>
> On Tuesday, March 24, 2015 at 2:16:57 PM UTC-4, Yogesh wrote:
>
> > Thanks. Mine is a one node cluster so I am simply using the XCURL to shut
> > it down and the doing "bin/elasticsearch -d" to start it up.
> > To check if it has shutdown, I try to hit it using http. So, now how do I
> > start it with the node/0 data directory?
> > There is nothing there in node/1 data directory but I don't suppose
> > deleting it would be the solution? (Sorry for the basic questions I am new
> > to this!)
> >
> > On Tue, Mar 24, 2015 at 11:32 PM, [mjd...@gmail.com](mailto:mjd...@gmail.com) wrote:
> >
> > > Sometimes that happens when the new node starts up via monit or other
> > > automated thing before the old node is fully shutdown. I'd suggest
> > > shutting down the node and verify it's done via ps before allowing the new
> > > node to start up. In the case of monit if the check is hitting the http
> > > port then it'll think it's down before it actually fully quits.
> > >
> > > On Tuesday, March 24, 2015 at 1:56:54 PM UTC-4, Yogesh wrote:
> > >
> > > > Thanks a lot mjdude! It does seem like it attached to the wrong data
> > > > directory.
> > > > In elasticsearch/data/tool/nodes there are two 0 and 1. My data is in 0
> > > > but node stats shows the data directory as elasticsearch/data/tool/nodes/
> > > > 1.
> > > > Now, how do I change this?
> > > >
> > > > On Tue, Mar 24, 2015 at 11:02 PM, [mjd...@gmail.com](mailto:mjd...@gmail.com) wrote:
> > > >
> > > > > When it restarted did it attach to the wrong data directory? Take a
> > > > > look at \_nodes/\_local/stats?pretty and check the 'data' directory
> > > > > location. Has the cluster recovered after the restart? Check
> > > > > \_cluster/health?pretty as well.
> > > > >
> > > > > On Tuesday, March 24, 2015 at 1:01:52 PM UTC-4, Yogesh wrote:
> > > > >
> > > > > > Thanks Joel and mjdude. What I mean is that ES is using 99% of the
> > > > > > heap memory (I think since Marvel showed memory as 1 GB which corresponds
> > > > > > to the heap, my RAM is 50GB)
> > > > > > I've increased the ES\_HEAP\_SIZE to 10g. But another problem has
> > > > > > appeared and I'm freaked out because of it!
> > > > > >
> > > > > > So, after restarting my ES (curl shutdown) to increase the heap, the
> > > > > > Marvel has stopped showing me my data (it still shows that disk memory is
> > > > > > lower so that means the data is still on disk) and upon searching Sense
> > > > > > shows "IndexMissingException[[my\_new\_twitter\_river] missing]"
> > > > > >
> > > > > > Why is this happening?!?!
> > > > > >
> > > > > > On Mon, Mar 23, 2015 at 9:25 PM, [mjd...@gmail.com](mailto:mjd...@gmail.com) wrote:
> > > > > >
> > > > > > > Are you saying JVM is using 99% of the system memory or 99% of the
> > > > > > > heap? If it's 99% of the available heap that's bad and you will have
> > > > > > > cluster instability. I suggest increasing your JVM heap size if you can, I
> > > > > > > can't find it right now but I remember a blog post that used twitter as a
> > > > > > > benchmark and they also could get to ~50M documents with the default 1G
> > > > > > > heap.
> > > > > > >
> > > > > > > On Sunday, March 22, 2015 at 3:30:57 AM UTC-4, Yogesh wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I have set up elasticsearch on one node and am using the Twitter
> > > > > > > > river to index tweets. It has been going fine with almost 50M tweets
> > > > > > > > indexed so far in 13 days.
> > > > > > > > When I started indexing, the JVM usage (observed via Marvel)
> > > > > > > > hovered between 10-20%, then started remaining around 30-40% but for the
> > > > > > > > past 3-4 days it has continuously been above 90%, reaching 99% at times!
> > > > > > > > I restarted elasticsearch thinking it might get resolved but as
> > > > > > > > soon as I switched it back on, the JVM usage went back to 90%.
> > > > > > > >
> > > > > > > > Why is this happening and how can I remedy it? (The JVM memory is
> > > > > > > > the default 990.75MB)
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > Yogesh
> > > > > > >
> > > > > > > --
> > > > > > > You received this message because you are subscribed to a topic in
> > > > > > > the Google Groups "elasticsearch" group.
> > > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/to](https://groups.google.com/d/to)
> > > > > > > pic/elasticsearch/kMSwBqpe2N4/unsubscribe.
> > > > > > > To unsubscribe from this group and all its topics, send an email to
> > > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > To view this discussion on the web visit
> > > > > > > [https://groups.google.com/d/msgid/elasticsearch/0f990291-fa2](https://groups.google.com/d/msgid/elasticsearch/0f990291-fa2)
> > > > > > > 0-4bba-881f-3f378985c8c9%[40googlegroups.com](http://40googlegroups.com)
> > > > > > > [https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > > > > .
> > > > > > >
> > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > > > > >
> > > > > > --
> > > > > > You received this message because you are subscribed to a topic in the
> > > > > > Google Groups "elasticsearch" group.
> > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/to](https://groups.google.com/d/to)
> > > > > > pic/elasticsearch/kMSwBqpe2N4/unsubscribe.
> > > > > > To unsubscribe from this group and all its topics, send an email to
> > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > > msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40goo
> > > > > > [glegroups.com](http://glegroups.com)
> > > > > > [https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > > > .
> > > > >
> > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > > >
> > > > --
> > > > You received this message because you are subscribed to a topic in the
> > > > Google Groups "elasticsearch" group.
> > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > topic/elasticsearch/kMSwBqpe2N4/unsubscribe.
> > > > To unsubscribe from this group and all its topics, send an email to
> > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%
> > > > [40googlegroups.com](http://40googlegroups.com)
> > > > [https://groups.google.com/d/msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > .
> > >
> > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> >
> > --
> > You received this message because you are subscribed to a topic in the
> > Google Groups "elasticsearch" group.
> > To unsubscribe from this topic, visit
> > [https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe](https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe).
> > To unsubscribe from this group and all its topics, send an email to
> > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/5c3bf7fb-7a1d-4467-a2df-bee4db0293c8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/5c3bf7fb-7a1d-4467-a2df-bee4db0293c8%40googlegroups.com)
> > [https://groups.google.com/d/msgid/elasticsearch/5c3bf7fb-7a1d-4467-a2df-bee4db0293c8%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/5c3bf7fb-7a1d-4467-a2df-bee4db0293c8%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
>
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CADM0w%3Di-g0x0\_p3gdciaJcDhVRTSQk%2B7QDp%2BL%3Dt0W8DOeA-OEA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CADM0w%3Di-g0x0_p3gdciaJcDhVRTSQk%2B7QDp%2BL%3Dt0W8DOeA-OEA%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@Yogesh\_Kansal](https://discuss.elastic.co/u/Yogesh_Kansal)
#### Post date: [March 25, 2015, 7:51am UTC](https://discuss.elastic.co/t/es-jvm-memory-usage-consistently-above-90/22800/11 "2015-03-25T07:51:31Z")
Thanks guys. The previous ES had not stopped fully, so killed it, then
deleted the node/1 directory, then set the node.max\_local\_storage\_nodes to
1 in yml.
Now everything works smoothly!
I've also increased the heap space. Though I hope ES doesn't consume the
entire 10g heap space!
On Wednesday, March 25, 2015 at 1:41:16 AM UTC+5:30, Yogesh wrote:
> Thanks. Will try starting again.
> By the way, what will be the impact if I just delete the node/1 (then the
> node.max\_local\_storage\_nodes to 1 and start elasticsearch) ?
>
> On Wed, Mar 25, 2015 at 1:36 AM, [mjdude5@gmail.com](mailto:mjdude5@gmail.com) wrote:
>
> > As long as it's the first ES instance starting on that node it'll grab 0
> > instead of 1. I don't know if you can explicitly set the data node
> > directory in the config.
> >
> > On Tuesday, March 24, 2015 at 2:16:57 PM UTC-4, Yogesh wrote:
> >
> > > Thanks. Mine is a one node cluster so I am simply using the XCURL to
> > > shut it down and the doing "bin/elasticsearch -d" to start it up.
> > > To check if it has shutdown, I try to hit it using http. So, now how do
> > > I start it with the node/0 data directory?
> > > There is nothing there in node/1 data directory but I don't suppose
> > > deleting it would be the solution? (Sorry for the basic questions I am new
> > > to this!)
> > >
> > > On Tue, Mar 24, 2015 at 11:32 PM, [mjd...@gmail.com](mailto:mjd...@gmail.com) wrote:
> > >
> > > > Sometimes that happens when the new node starts up via monit or other
> > > > automated thing before the old node is fully shutdown. I'd suggest
> > > > shutting down the node and verify it's done via ps before allowing the new
> > > > node to start up. In the case of monit if the check is hitting the http
> > > > port then it'll think it's down before it actually fully quits.
> > > >
> > > > On Tuesday, March 24, 2015 at 1:56:54 PM UTC-4, Yogesh wrote:
> > > >
> > > > > Thanks a lot mjdude! It does seem like it attached to the wrong data
> > > > > directory.
> > > > > In elasticsearch/data/tool/nodes there are two 0 and 1. My data is in
> > > > > 0 but node stats shows the data directory as elasticsearch/data/tool/nodes/
> > > > > 1.
> > > > > Now, how do I change this?
> > > > >
> > > > > On Tue, Mar 24, 2015 at 11:02 PM, [mjd...@gmail.com](mailto:mjd...@gmail.com) wrote:
> > > > >
> > > > > > When it restarted did it attach to the wrong data directory? Take a
> > > > > > look at \_nodes/\_local/stats?pretty and check the 'data' directory
> > > > > > location. Has the cluster recovered after the restart? Check
> > > > > > \_cluster/health?pretty as well.
> > > > > >
> > > > > > On Tuesday, March 24, 2015 at 1:01:52 PM UTC-4, Yogesh wrote:
> > > > > >
> > > > > > > Thanks Joel and mjdude. What I mean is that ES is using 99% of the
> > > > > > > heap memory (I think since Marvel showed memory as 1 GB which corresponds
> > > > > > > to the heap, my RAM is 50GB)
> > > > > > > I've increased the ES\_HEAP\_SIZE to 10g. But another problem has
> > > > > > > appeared and I'm freaked out because of it!
> > > > > > >
> > > > > > > So, after restarting my ES (curl shutdown) to increase the heap, the
> > > > > > > Marvel has stopped showing me my data (it still shows that disk memory is
> > > > > > > lower so that means the data is still on disk) and upon searching Sense
> > > > > > > shows "IndexMissingException[[my\_new\_twitter\_river] missing]"
> > > > > > >
> > > > > > > Why is this happening?!?!
> > > > > > >
> > > > > > > On Mon, Mar 23, 2015 at 9:25 PM, [mjd...@gmail.com](mailto:mjd...@gmail.com) wrote:
> > > > > > >
> > > > > > > > Are you saying JVM is using 99% of the system memory or 99% of the
> > > > > > > > heap? If it's 99% of the available heap that's bad and you will have
> > > > > > > > cluster instability. I suggest increasing your JVM heap size if you can, I
> > > > > > > > can't find it right now but I remember a blog post that used twitter as a
> > > > > > > > benchmark and they also could get to ~50M documents with the default 1G
> > > > > > > > heap.
> > > > > > > >
> > > > > > > > On Sunday, March 22, 2015 at 3:30:57 AM UTC-4, Yogesh wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > I have set up elasticsearch on one node and am using the Twitter
> > > > > > > > > river to index tweets. It has been going fine with almost 50M tweets
> > > > > > > > > indexed so far in 13 days.
> > > > > > > > > When I started indexing, the JVM usage (observed via Marvel)
> > > > > > > > > hovered between 10-20%, then started remaining around 30-40% but for the
> > > > > > > > > past 3-4 days it has continuously been above 90%, reaching 99% at times!
> > > > > > > > > I restarted elasticsearch thinking it might get resolved but as
> > > > > > > > > soon as I switched it back on, the JVM usage went back to 90%.
> > > > > > > > >
> > > > > > > > > Why is this happening and how can I remedy it? (The JVM memory is
> > > > > > > > > the default 990.75MB)
> > > > > > > > >
> > > > > > > > > Thanks
> > > > > > > > > Yogesh
> > > > > > > >
> > > > > > > > --
> > > > > > > > You received this message because you are subscribed to a topic in
> > > > > > > > the Google Groups "elasticsearch" group.
> > > > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > > > > topic/elasticsearch/kMSwBqpe2N4/unsubscribe.
> > > > > > > > To unsubscribe from this group and all its topics, send an email to
> > > > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > To view this discussion on the web visit
> > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/0f990291-fa2](https://groups.google.com/d/msgid/elasticsearch/0f990291-fa2)
> > > > > > > > 0-4bba-881f-3f378985c8c9%[40googlegroups.com](http://40googlegroups.com)
> > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0f990291-fa20-4bba-881f-3f378985c8c9%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > > > > > .
> > > > > > > >
> > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > > > > > >
> > > > > > > --
> > > > > > > You received this message because you are subscribed to a topic in
> > > > > > > the Google Groups "elasticsearch" group.
> > > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/to](https://groups.google.com/d/to)
> > > > > > > pic/elasticsearch/kMSwBqpe2N4/unsubscribe.
> > > > > > > To unsubscribe from this group and all its topics, send an email to
> > > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > > > msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40goo
> > > > > > > [glegroups.com](http://glegroups.com)
> > > > > > > [https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/057e53f7-3207-4df9-bbc4-80cea03e189f%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > > > > .
> > > > > >
> > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to a topic in the
> > > > > Google Groups "elasticsearch" group.
> > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > topic/elasticsearch/kMSwBqpe2N4/unsubscribe.
> > > > > To unsubscribe from this group and all its topics, send an email to
> > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%
> > > > > [40googlegroups.com](http://40googlegroups.com)
> > > > > [https://groups.google.com/d/msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/a3ca6aa0-f3d9-4067-933c-1a2eb0a15075%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > > .
> > > >
> > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > >
> > > --
> > > You received this message because you are subscribed to a topic in the
> > > Google Groups "elasticsearch" group.
> > > To unsubscribe from this topic, visit
> > > [https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe](https://groups.google.com/d/topic/elasticsearch/kMSwBqpe2N4/unsubscribe).
> > > To unsubscribe from this group and all its topics, send an email to
> > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > > To view this discussion on the web visit
> > > [https://groups.google.com/d/msgid/elasticsearch/5c3bf7fb-7a1d-4467-a2df-bee4db0293c8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/5c3bf7fb-7a1d-4467-a2df-bee4db0293c8%40googlegroups.com)
> > > [https://groups.google.com/d/msgid/elasticsearch/5c3bf7fb-7a1d-4467-a2df-bee4db0293c8%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/5c3bf7fb-7a1d-4467-a2df-bee4db0293c8%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > .
> >
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/ba9ddf48-bde9-4da4-b852-b666b6842c3f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ba9ddf48-bde9-4da4-b852-b666b6842c3f%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 12:24am UTC](https://discuss.elastic.co/t/es-jvm-memory-usage-consistently-above-90/22800/12 "2017-07-06T00:24:12Z")