style=
>
> ...
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d94a7db0-977c-4282-a442-b2a85ebbbab4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d94a7db0-977c-4282-a442-b2a85ebbbab4%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@jprante](https://discuss.elastic.co/u/jprante)
#### Post date: [September 25, 2014, 8:21am UTC](https://discuss.elastic.co/t/elasticsearch-threads-behaves-different-from-each-other/19954/4 "2014-09-25T08:21:27Z")
Do not manipulate the threadpool to "fixed" and do not use such high
numbers like 100,600, 5000, 50000.... This will sooner or later congest
your machine. The long list of threads in OS is just one (harmless) symptom
of a misconfiguration. Use the default setting.
Do not use 80% buffer for index. This hurts search performance and cache
resources. Use the default setting.
Do not use cache field type "soft". This hides cache and GC problems and
gives cryptic exceptions plus very bad performance in spikes. Use the
default setting.
I hope the single node is a development machine. For better performance,
use at least 3 nodes on 3 machines.
Jörg
> indices.memory.index\_buffer\_size: 80%
> index.store.compress.stored: true
> index.store.fs.lock: none
>
> threadpool.search.type: fixed
> threadpool.search.size: 600
> threadpool.search.queue\_size: 10000
>
> threadpool.bulk.type: fixed
> threadpool.bulk.size: 600
> threadpool.bulk.queue\_size: 10000
>
> threadpool.index.type: fixed
> threadpool.index.size: 100
> threadpool.index.queue\_size: 5000
>
> index.cache.field.type: soft
> index.cache.field.max\_size: 50000
> index.cache.field.expire: 24h
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFJiaGvdZ1DUpDjUkW7ZskXAeJSzRgdS9uTasW9S-\_uTA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFJiaGvdZ1DUpDjUkW7ZskXAeJSzRgdS9uTasW9S-_uTA%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@Umut\_Yerci](https://discuss.elastic.co/u/Umut_Yerci)
#### Post date: [September 25, 2014, 11:26am UTC](https://discuss.elastic.co/t/elasticsearch-threads-behaves-different-from-each-other/19954/5 "2014-09-25T11:26:29Z")
Thank you for your answer Jörg. I remove them in conf file but my
performance problem continues. I have another question about it. My
network stat is that;
curl -XGET '[http://localhost:9200/\_nodes/stats/network?human&pretty](http://localhost:9200/_nodes/stats/network?human&pretty)'
{
"cluster\_name" : "test-cluster",
"nodes" : {
"XB95yJZhS7WLAPBm9O994Q" : {
"timestamp" : 1411643924755,
"name" : "test",
"transport\_address" : "inet[/127.0.0.1:9300]",
"host" : "test-host",
"ip" : ["inet[/127.0.0.1:9300]", "NONE" ],
"attributes" : {
"master" : "true"
},
"network" : {
"tcp" : {
"active\_opens" : 794549,
"passive\_opens" : 489890,
"curr\_estab" : 402,
"in\_segs" : 99304476,
"out\_segs" : 100905758,
"retrans\_segs" : 53037,
"estab\_resets" : 10989,
"attempt\_fails" : 420884,
"in\_errs" : 18226,
"out\_rsts" : 439001
}
}
}
}
}
Have you any idea about why curr\_estab value is so big? I use
elasticsearch.py for insertion and only 8 connections are active for
indexing data on 2 processes.
On Thursday, September 25, 2014 11:21:35 AM UTC+3, Jörg Prante wrote:
> Do not manipulate the threadpool to "fixed" and do not use such high
> numbers like 100,600, 5000, 50000.... This will sooner or later congest
> your machine. The long list of threads in OS is just one (harmless) symptom
> of a misconfiguration. Use the default setting.
>
> Do not use 80% buffer for index. This hurts search performance and cache
> resources. Use the default setting.
>
> Do not use cache field type "soft". This hides cache and GC problems and
> gives cryptic exceptions plus very bad performance in spikes. Use the
> default setting.
>
> I hope the single node is a development machine. For better performance,
> use at least 3 nodes on 3 machines.
>
> Jörg
>
> > indices.memory.index\_buffer\_size: 80%
> > index.store.compress.stored: true
> > index.store.fs.lock: none
> >
> > threadpool.search.type: fixed
> > threadpool.search.size: 600
> > threadpool.search.queue\_size: 10000
> >
> > threadpool.bulk.type: fixed
> > threadpool.bulk.size: 600
> > threadpool.bulk.queue\_size: 10000
> >
> > threadpool.index.type: fixed
> > threadpool.index.size: 100
> > threadpool.index.queue\_size: 5000
> >
> > index.cache.field.type: soft
> > index.cache.field.max\_size: 50000
> > index.cache.field.expire: 24h
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/82a3d79b-19a4-4493-aab2-0dcfdd062bbe%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/82a3d79b-19a4-4493-aab2-0dcfdd062bbe%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@jprante](https://discuss.elastic.co/u/jprante)
#### Post date: [September 25, 2014, 4:07pm UTC](https://discuss.elastic.co/t/elasticsearch-threads-behaves-different-from-each-other/19954/6 "2014-09-25T16:07:43Z")
What you see are some of the TCP/IP stack counters of your hardware network
interface since it was started. It is provided by the OS and read by sigar
> **[VMware Products](https://www.vmware.com/products.html)**
>
> With a digital foundation built on VMware products, you can build, run, manage, connect, deliver, and protect all types of applications – everywhere. Learn how.
It is not related to ES or to ES specific connections. It's just a
nice-to-have if you do not want to enter CLI and execute OS commands.
Jörg
On Thu, Sep 25, 2014 at 1:26 PM, Umut Yerci [umutyerci@gmail.com](mailto:umutyerci@gmail.com) wrote:
> Thank you for your answer Jörg. I remove them in conf file but my
> performance problem continues. I have another question about it. My
> network stat is that;
>
> curl -XGET '[http://localhost:9200/\_nodes/stats/network?human&pretty](http://localhost:9200/_nodes/stats/network?human&pretty)'
> {
> "cluster\_name" : "test-cluster",
> "nodes" : {
> "XB95yJZhS7WLAPBm9O994Q" : {
> "timestamp" : 1411643924755,
> "name" : "test",
> "transport\_address" : "inet[/127.0.0.1:9300]",
> "host" : "test-host",
> "ip" : ["inet[/127.0.0.1:9300]", "NONE" ],
> "attributes" : {
> "master" : "true"
> },
> "network" : {
> "tcp" : {
> "active\_opens" : 794549,
> "passive\_opens" : 489890,
> "curr\_estab" : 402,
> "in\_segs" : 99304476,
> "out\_segs" : 100905758,
> "retrans\_segs" : 53037,
> "estab\_resets" : 10989,
> "attempt\_fails" : 420884,
> "in\_errs" : 18226,
> "out\_rsts" : 439001
> }
> }
> }
> }
> }
>
> Have you any idea about why curr\_estab value is so big? I use
> elasticsearch.py for insertion and only 8 connections are active for
> indexing data on 2 processes.
>
> On Thursday, September 25, 2014 11:21:35 AM UTC+3, Jörg Prante wrote:
>
> > Do not manipulate the threadpool to "fixed" and do not use such high
> > numbers like 100,600, 5000, 50000.... This will sooner or later congest
> > your machine. The long list of threads in OS is just one (harmless) symptom
> > of a misconfiguration. Use the default setting.
> >
> > Do not use 80% buffer for index. This hurts search performance and cache
> > resources. Use the default setting.
> >
> > Do not use cache field type "soft". This hides cache and GC problems and
> > gives cryptic exceptions plus very bad performance in spikes. Use the
> > default setting.
> >
> > I hope the single node is a development machine. For better performance,
> > use at least 3 nodes on 3 machines.
> >
> > Jörg
> >
> > > indices.memory.index\_buffer\_size: 80%
> > > index.store.compress.stored: true
> > > index.store.fs.lock: none
> > >
> > > threadpool.search.type: fixed
> > > threadpool.search.size: 600
> > > threadpool.search.queue\_size: 10000
> > >
> > > threadpool.bulk.type: fixed
> > > threadpool.bulk.size: 600
> > > threadpool.bulk.queue\_size: 10000
> > >
> > > threadpool.index.type: fixed
> > > threadpool.index.size: 100
> > > threadpool.index.queue\_size: 5000
> > >
> > > index.cache.field.type: soft
> > > index.cache.field.max\_size: 50000
> > > index.cache.field.expire: 24h
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/82a3d79b-19a4-4493-aab2-0dcfdd062bbe%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/82a3d79b-19a4-4493-aab2-0dcfdd062bbe%40googlegroups.com)
> > [https://groups.google.com/d/msgid/elasticsearch/82a3d79b-19a4-4493-aab2-0dcfdd062bbe%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/82a3d79b-19a4-4493-aab2-0dcfdd062bbe%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
>
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHxVEmOD7CGGhhR8aLRXxmMW9gR-5DWgTT%3Dm9VTm3Z20A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHxVEmOD7CGGhhR8aLRXxmMW9gR-5DWgTT%3Dm9VTm3Z20A%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@Umut\_Yerci](https://discuss.elastic.co/u/Umut_Yerci)
#### Post date: [September 26, 2014, 7:20am UTC](https://discuss.elastic.co/t/elasticsearch-threads-behaves-different-from-each-other/19954/7 "2014-09-26T07:20:55Z")
Thank you for answers Jörg. I really appreciated your helps :-).
On Thursday, September 25, 2014 7:07:51 PM UTC+3, Jörg Prante wrote:
> What you see are some of the TCP/IP stack counters of your hardware
> network interface since it was started. It is provided by the OS and read
> by sigar
> [VMware Products](http://www.hyperic.com/support/docs/sigar/org/hyperic/sigar/Tcp.html)
>
> It is not related to ES or to ES specific connections. It's just a
> nice-to-have if you do not want to enter CLI and execute OS commands.
>
> Jörg
>
> On Thu, Sep 25, 2014 at 1:26 PM, Umut Yerci \<[umut...@gmail.com](mailto:umut...@gmail.com)
> \\> wrote:
>
> > Thank you for your answer Jörg. I remove them in conf file but my
> > performance problem continues. I have another question about it. My
> > network stat is that;
> >
> > curl -XGET '[http://localhost:9200/\_nodes/stats/network?human&pretty](http://localhost:9200/_nodes/stats/network?human&pretty)'
> > {
> > "cluster\_name" : "test-cluster",
> > "nodes" : {
> > "XB95yJZhS7WLAPBm9O994Q" : {
> > "timestamp" : 1411643924755,
> > "name" : "test",
> > "transport\_address" : "inet[/127.0.0.1:9300]",
> > "host" : "test-host",
> > "ip" : ["inet[/127.0.0.1:9300]", "NONE" ],
> > "attributes" : {
> > "master" : "true"
> > },
> > "network" : {
> > "tcp" : {
> > "active\_opens" : 794549,
> > "passive\_opens" : 489890,
> > "curr\_estab" : 402,
> > "in\_segs" : 99304476,
> > "out\_segs" : 100905758,
> > "retrans\_segs" : 53037,
> > "estab\_resets" : 10989,
> > "attempt\_fails" : 420884,
> > "in\_errs" : 18226,
> > "out\_rsts" : 439001
> > }
> > }
> > }
> > }
> > }
> >
> > Have you any idea about why curr\_estab value is so big? I use
> > elasticsearch.py for insertion and only 8 connections are active for
> > indexing data on 2 processes.
> >
> > On Thursday, September 25, 2014 11:21:35 AM UTC+3, Jörg Prante wrote:
> >
> > > Do not manipulate the threadpool to "fixed" and do not use such high
> > > numbers like 100,600, 5000, 50000.... This will sooner or later congest
> > > your machine. The long list of threads in OS is just one (harmless) symptom
> > > of a misconfiguration. Use the default setting.
> > >
> > > Do not use 80% buffer for index. This hurts search performance and cache
> > > resources. Use the default setting.
> > >
> > > Do not use cache field type "soft". This hides cache and GC problems and
> > > gives cryptic exceptions plus very bad performance in spikes. Use the
> > > default setting.
> > >
> > > I hope the single node is a development machine. For better performance,
> > > use at least 3 nodes on 3 machines.
> > >
> > > Jörg
> > >
> > > > indices.memory.index\_buffer\_size: 80%
> > > > index.store.compress.stored: true
> > > > index.store.fs.lock: none
> > > >
> > > > threadpool.search.type: fixed
> > > > threadpool.search.size: 600
> > > > threadpool.search.queue\_size: 10000
> > > >
> > > > threadpool.bulk.type: fixed
> > > > threadpool.bulk.size: 600
> > > > threadpool.bulk.queue\_size: 10000
> > > >
> > > > threadpool.index.type: fixed
> > > > threadpool.index.size: 100
> > > > threadpool.index.queue\_size: 5000
> > > >
> > > > index.cache.field.type: soft
> > > > index.cache.field.max\_size: 50000
> > > > index.cache.field.expire: 24h
> > >
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "elasticsearch" group.
> > > To unsubscribe from this group and stop receiving emails from it, send an
> > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > > To view this discussion on the web visit
> > > [https://groups.google.com/d/msgid/elasticsearch/82a3d79b-19a4-4493-aab2-0dcfdd062bbe%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/82a3d79b-19a4-4493-aab2-0dcfdd062bbe%40googlegroups.com)
> > > [https://groups.google.com/d/msgid/elasticsearch/82a3d79b-19a4-4493-aab2-0dcfdd062bbe%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/82a3d79b-19a4-4493-aab2-0dcfdd062bbe%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > .
> >
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0b496fb3-46d8-44bf-8dc4-991e370a1d7b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0b496fb3-46d8-44bf-8dc4-991e370a1d7b%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 12:59am UTC](https://discuss.elastic.co/t/elasticsearch-threads-behaves-different-from-each-other/19954/8 "2017-07-06T00:59:46Z")