# ES for persistent data on single node =\> performance problems
**URL:** https://discuss.elastic.co/t/es-for-persistent-data-on-single-node-performance-problems/12443
**Category:** Elasticsearch
**Created:** [June 17, 2013, 9:08am UTC](https://discuss.elastic.co/t/es-for-persistent-data-on-single-node-performance-problems/12443 "2013-06-17T09:08:45Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@panmari](https://discuss.elastic.co/u/panmari)
#### Post date: [June 17, 2013, 9:08am UTC](https://discuss.elastic.co/t/es-for-persistent-data-on-single-node-performance-problems/12443/1 "2013-06-17T09:08:45Z")
Hey everyone
I'm running ES on a single server through a single node with persistent
data. Following some guides I found online, I added the following to
elasticsearch.yml:
# Not refreshing the index
index.refresh\_interval: -1
# Uses less memory this way (default 1)
index.term\_index\_divisor: 5
# also less memory usage
index.term\_index\_interval: 256
Nevertheless, the server becomes really sluggish after some time (about a
week). The last things I see in the log are:
[2013-06-16 21:15:23,349][DEBUG][action.search.type] [Pretty
Persuasions] [186556171] Failed to execute fetch phase
org.elasticsearch.search.SearchContextMissingException: No search context
found for id [186556171]
at org.elasticsearch.search.SearchService.findContext(SearchService.java:
455)
at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.
java:428)
at org.elasticsearch.search.action.SearchServiceTransportAction.
sendExecuteFetch(SearchServiceTransportAction.java:345)
at org.elasticsearch.action.search.type.
TransportSearchQueryThenFetchAction$AsyncAction.executeFetch(
TransportSearchQueryThenFetchAction.java:149)
at org.elasticsearch.action.search.type.
TransportSearchQueryThenFetchAction$AsyncAction$2.run(
TransportSearchQueryThenFetchAction.java:136)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
java:615)
at java.lang.Thread.run(Thread.java:722)
[2013-06-16 21:18:19,913][WARN][monitor.jvm] [Pretty
Persuasions] [gc][ParNew][867853][376928] duration [1.2s], collections [1]/[
1.2s], total [1.2s]/[2.6h], memory [854.3mb]-\>[531.4mb]/[2.9gb], all\_pools
{[Code Cache] [7.7mb]-\>[7.7mb]/[48mb]}{[Par Eden Space] [502.8mb]-\>[366.7kb
]/[532.5mb]}{[Par Survivor Space] [28.4mb]-\>[66.5mb]/[66.5mb]}{[CMS Old Gen]
[323mb]-\>[464.5mb]/[2.3gb]}{[CMS Perm Gen] [31.8mb]-\>[31.8mb]/[82mb]}
[2013-06-16 21:21:24,234][INFO][monitor.jvm] [Pretty
Persuasions] [gc][ParNew][868037][377034] duration [997ms], collections [1
]/[1.1s], total [997ms]/[2.6h], memory [1.1gb]-\>[1010.8mb]/[2.9gb],all\_pools
{[Code Cache] [7.7mb]-\>[7.7mb]/[48mb]}{[Par Eden Space] [406.3mb]-\>[943kb]/[
532.5mb]}{[Par Survivor Space] [23.3mb]-\>[66.5mb]/[66.5mb]}{[CMS Old Gen] [
796.2mb]-\>[943.4mb]/[2.3gb]}{[CMS Perm Gen] [31.8mb]-\>[31.8mb]/[82mb]}
When I restart the server, the performance goes up again. Did I make some
mistake in my configuration?
Thanks in advance.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@brian\_yoder](https://discuss.elastic.co/u/brian_yoder)
#### Post date: [June 17, 2013, 1:53pm UTC](https://discuss.elastic.co/t/es-for-persistent-data-on-single-node-performance-problems/12443/2 "2013-06-17T13:53:44Z")
Hi!
For my single-node Elasticsearch instances (and indeed even for the 3-node
cluster), I don't ever set any of those configurations; the defaults work
fine.
For my Java bulk-loader, I temporarily set the index refresh rate to 120s
(not -1; 120s was recommended somewhere in Shay's documentation), then
bulk-load in chunks of 4096 documents at a time, and then set the refresh
rate back to 1s (the default) when done.
For one-at-a-time updates, even under moderate load, I always leave the
default refresh rate.
This strategy never seems to have any issues such as the one you describe.
Hope this helps!
Brian
On Monday, June 17, 2013 5:08:45 AM UTC-4, panmari wrote:
> Hey everyone
>
> I'm running ES on a single server through a single node with persistent
> data. Following some guides I found online, I added the following to
> elasticsearch.yml:
>
> # Not refreshing the index
>
> index.refresh\_interval: -1
>
> # Uses less memory this way (default 1)
>
> index.term\_index\_divisor: 5
>
> # also less memory usage
>
> index.term\_index\_interval: 256
>
> Nevertheless, the server becomes really sluggish after some time (about a
> week). The last things I see in the log are:
>
> [2013-06-16 21:15:23,349][DEBUG][action.search.type] [Pretty
> Persuasions] [186556171] Failed to execute fetch phase
> org.elasticsearch.search.SearchContextMissingException: No search context
> found for id [186556171]
> at org.elasticsearch.search.SearchService.findContext(SearchService.java:
> 455)
> at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService
> .java:428)
> at org.elasticsearch.search.action.SearchServiceTransportAction.
> sendExecuteFetch(SearchServiceTransportAction.java:345)
> at org.elasticsearch.action.search.type.
> TransportSearchQueryThenFetchAction$AsyncAction.executeFetch(
> TransportSearchQueryThenFetchAction.java:149)
> at org.elasticsearch.action.search.type.
> TransportSearchQueryThenFetchAction$AsyncAction$2.run(
> TransportSearchQueryThenFetchAction.java:136)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
> java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
> java:615)
> at java.lang.Thread.run(Thread.java:722)
> [2013-06-16 21:18:19,913][WARN][monitor.jvm] [Pretty
> Persuasions] [gc][ParNew][867853][376928] duration [1.2s], collections [1
> ]/[1.2s], total [1.2s]/[2.6h], memory [854.3mb]-\>[531.4mb]/[2.9gb],all\_pools
> {[Code Cache] [7.7mb]-\>[7.7mb]/[48mb]}{[Par Eden Space] [502.8mb]-\>[
> 366.7kb]/[532.5mb]}{[Par Survivor Space] [28.4mb]-\>[66.5mb]/[66.5mb]}{[CMS
> Old Gen] [323mb]-\>[464.5mb]/[2.3gb]}{[CMS Perm Gen] [31.8mb]-\>[31.8mb]/[
> 82mb]}
> [2013-06-16 21:21:24,234][INFO][monitor.jvm] [Pretty
> Persuasions] [gc][ParNew][868037][377034] duration [997ms], collections [1
> ]/[1.1s], total [997ms]/[2.6h], memory [1.1gb]-\>[1010.8mb]/[2.9gb],all\_pools
> {[Code Cache] [7.7mb]-\>[7.7mb]/[48mb]}{[Par Eden Space] [406.3mb]-\>[943kb
> ]/[532.5mb]}{[Par Survivor Space] [23.3mb]-\>[66.5mb]/[66.5mb]}{[CMS Old
> Gen] [796.2mb]-\>[943.4mb]/[2.3gb]}{[CMS Perm Gen] [31.8mb]-\>[31.8mb]/[82mb
> ]}
>
> When I restart the server, the performance goes up again. Did I make some
> mistake in my configuration?
>
> Thanks in advance.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@panmari](https://discuss.elastic.co/u/panmari)
#### Post date: [June 18, 2013, 2:50pm UTC](https://discuss.elastic.co/t/es-for-persistent-data-on-single-node-performance-problems/12443/3 "2013-06-18T14:50:49Z")
Hey Brian
I'll try vanilla configuration then... How much heap space are you giving
ES? And how much memory do you have available?
On Monday, June 17, 2013 3:53:44 PM UTC+2, InquiringMind wrote:
> Hi!
>
> For my single-node Elasticsearch instances (and indeed even for the 3-node
> cluster), I don't ever set any of those configurations; the defaults work
> fine.
>
> For my Java bulk-loader, I temporarily set the index refresh rate to 120s
> (not -1; 120s was recommended somewhere in Shay's documentation), then
> bulk-load in chunks of 4096 documents at a time, and then set the refresh
> rate back to 1s (the default) when done.
>
> For one-at-a-time updates, even under moderate load, I always leave the
> default refresh rate.
>
> This strategy never seems to have any issues such as the one you describe.
>
> Hope this helps!
>
> Brian
>
> On Monday, June 17, 2013 5:08:45 AM UTC-4, panmari wrote:
>
> > Hey everyone
> >
> > I'm running ES on a single server through a single node with persistent
> > data. Following some guides I found online, I added the following to
> > elasticsearch.yml:
> >
> > # Not refreshing the index
> >
> > index.refresh\_interval: -1
> >
> > # Uses less memory this way (default 1)
> >
> > index.term\_index\_divisor: 5
> >
> > # also less memory usage
> >
> > index.term\_index\_interval: 256
> >
> > Nevertheless, the server becomes really sluggish after some time (about a
> > week). The last things I see in the log are:
> >
> > [2013-06-16 21:15:23,349][DEBUG][action.search.type] [Pretty
> > Persuasions] [186556171] Failed to execute fetch phase
> > org.elasticsearch.search.SearchContextMissingException: No search
> > context found for id [186556171]
> > at org.elasticsearch.search.SearchService.findContext(SearchService.java
> > :455)
> > at org.elasticsearch.search.SearchService.executeFetchPhase(
> > SearchService.java:428)
> > at org.elasticsearch.search.action.SearchServiceTransportAction.
> > sendExecuteFetch(SearchServiceTransportAction.java:345)
> > at org.elasticsearch.action.search.type.
> > TransportSearchQueryThenFetchAction$AsyncAction.executeFetch(
> > TransportSearchQueryThenFetchAction.java:149)
> > at org.elasticsearch.action.search.type.
> > TransportSearchQueryThenFetchAction$AsyncAction$2.run(
> > TransportSearchQueryThenFetchAction.java:136)
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
> > java:1145)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
> > .java:615)
> > at java.lang.Thread.run(Thread.java:722)
> > [2013-06-16 21:18:19,913][WARN][monitor.jvm] [Pretty
> > Persuasions] [gc][ParNew][867853][376928] duration [1.2s], collections [1
> > ]/[1.2s], total [1.2s]/[2.6h], memory [854.3mb]-\>[531.4mb]/[2.9gb],all\_pools
> > {[Code Cache] [7.7mb]-\>[7.7mb]/[48mb]}{[Par Eden Space] [502.8mb]-\>[
> > 366.7kb]/[532.5mb]}{[Par Survivor Space] [28.4mb]-\>[66.5mb]/[66.5mb]}{[CMS
> > Old Gen] [323mb]-\>[464.5mb]/[2.3gb]}{[CMS Perm Gen] [31.8mb]-\>[31.8mb]/[
> > 82mb]}
> > [2013-06-16 21:21:24,234][INFO][monitor.jvm] [Pretty
> > Persuasions] [gc][ParNew][868037][377034] duration [997ms], collections [
> > 1]/[1.1s], total [997ms]/[2.6h], memory [1.1gb]-\>[1010.8mb]/[2.9gb],all\_pools
> > {[Code Cache] [7.7mb]-\>[7.7mb]/[48mb]}{[Par Eden Space] [406.3mb]-\>[943kb
> > ]/[532.5mb]}{[Par Survivor Space] [23.3mb]-\>[66.5mb]/[66.5mb]}{[CMS Old
> > Gen] [796.2mb]-\>[943.4mb]/[2.3gb]}{[CMS Perm Gen] [31.8mb]-\>[31.8mb]/[
> > 82mb]}
> >
> > When I restart the server, the performance goes up again. Did I make some
> > mistake in my configuration?
> >
> > Thanks in advance.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@brian\_yoder](https://discuss.elastic.co/u/brian_yoder)
#### Post date: [June 18, 2013, 3:38pm UTC](https://discuss.elastic.co/t/es-for-persistent-data-on-single-node-performance-problems/12443/4 "2013-06-18T15:38:18Z")
On my laptop (MacBook i7 quad core, 8 GB of RAM, but single slow disk), I
take all the defaults. On our single-node cluster testing machines (not
testing ES, but testing that uses ES to replace a production server), we
like to keep it as tiny as possible (lots of other things running on one
machine); in this case we start it with the -server option to the JVM, and
then set the memory to 256MB. Seems to work fine.
Brian
On Tuesday, June 18, 2013 10:50:49 AM UTC-4, panmari wrote:
> Hey Brian
>
> I'll try vanilla configuration then... How much heap space are you giving
> ES? And how much memory do you have available?
>
> On Monday, June 17, 2013 3:53:44 PM UTC+2, InquiringMind wrote:
>
> > Hi!
> >
> > For my single-node Elasticsearch instances (and indeed even for the
> > 3-node cluster), I don't ever set any of those configurations; the defaults
> > work fine.
> >
> > For my Java bulk-loader, I temporarily set the index refresh rate to 120s
> > (not -1; 120s was recommended somewhere in Shay's documentation), then
> > bulk-load in chunks of 4096 documents at a time, and then set the refresh
> > rate back to 1s (the default) when done.
> >
> > For one-at-a-time updates, even under moderate load, I always leave the
> > default refresh rate.
> >
> > This strategy never seems to have any issues such as the one you describe.
> >
> > Hope this helps!
> >
> > Brian
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:30am UTC](https://discuss.elastic.co/t/es-for-persistent-data-on-single-node-performance-problems/12443/5 "2017-07-06T02:30:35Z")