# Interesting Issue
**URL:** https://discuss.elastic.co/t/interesting-issue/18110
**Category:** Elasticsearch
**Created:** [June 14, 2014, 4:06pm UTC](https://discuss.elastic.co/t/interesting-issue/18110 "2014-06-14T16:06:12Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@John\_Luko](https://discuss.elastic.co/u/John_Luko)
#### Post date: [June 14, 2014, 4:06pm UTC](https://discuss.elastic.co/t/interesting-issue/18110/1 "2014-06-14T16:06:12Z")
We are running Elasticsearch with three nodes. The problem we keep running
into is eventually Kibana begins to run extremely slow and soon after
becomes unresponsive. If you hit refresh the wheels just keep spinning and
nothing gets displayed. Restarting the cluster appears to correct this
issue, but within 24 hours it begins again. Below is our health:
{
"cluster\_name" : "elasticsearch",
"status" : "green",
"timed\_out" : false,
"number\_of\_nodes" : 4,
"number\_of\_data\_nodes" : 3,
"active\_primary\_shards" : 341,
"active\_shards" : 682,
"relocating\_shards" : 2,
"initializing\_shards" : 0,
"unassigned\_shards" : 0
}
When we initially set everything up we had a single node and we did have a
span of three weeks where we didn't have to restart the Elasticsearch
service. When we begin to pump netflow data in we began to have issues. I
thought perhaps because we had logstash and ES running on one node that was
causing the issue. Thus we added two virtual nodes and had one of them
host the logstash for just Netflow. I thought with the clustering the
issue would be resolved, but sadly I still have to start and stop the
services everyday. When I look back at the data it flows in the entire
time until flat lining for a few hours and then picks up again once I
restart the services.
Thanks in advance!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [June 14, 2014, 4:25pm UTC](https://discuss.elastic.co/t/interesting-issue/18110/2 "2014-06-14T16:25:34Z")
First guess is that you are running out of memory or so.
What can you see in elasticsearch logs?
Don't you see some WARN about GC?
--
David 😉
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 14 juin 2014 à 18:06, John Luko [johnluko@gmail.com](mailto:johnluko@gmail.com) a écrit :
We are running Elasticsearch with three nodes. The problem we keep running into is eventually Kibana begins to run extremely slow and soon after becomes unresponsive. If you hit refresh the wheels just keep spinning and nothing gets displayed. Restarting the cluster appears to correct this issue, but within 24 hours it begins again. Below is our health:
{
"cluster\_name" : "elasticsearch",
"status" : "green",
"timed\_out" : false,
"number\_of\_nodes" : 4,
"number\_of\_data\_nodes" : 3,
"active\_primary\_shards" : 341,
"active\_shards" : 682,
"relocating\_shards" : 2,
"initializing\_shards" : 0,
"unassigned\_shards" : 0
}
When we initially set everything up we had a single node and we did have a span of three weeks where we didn't have to restart the Elasticsearch service. When we begin to pump netflow data in we began to have issues. I thought perhaps because we had logstash and ES running on one node that was causing the issue. Thus we added two virtual nodes and had one of them host the logstash for just Netflow. I thought with the clustering the issue would be resolved, but sadly I still have to start and stop the services everyday. When I look back at the data it flows in the entire time until flat lining for a few hours and then picks up again once I restart the services.
## Thanks in advance!
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/80DCF447-DF89-4CD1-A52C-0DF21D14FC37%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/80DCF447-DF89-4CD1-A52C-0DF21D14FC37%40pilato.fr).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@John\_Luko](https://discuss.elastic.co/u/John_Luko)
#### Post date: [June 14, 2014, 4:44pm UTC](https://discuss.elastic.co/t/interesting-issue/18110/3 "2014-06-14T16:44:02Z")
Is there a particular log I should look at? In elasticsearch.log I am
seeing a lot of parser failures for a timestamp.
On Saturday, June 14, 2014 12:26:47 PM UTC-4, David Pilato wrote:
> First guess is that you are running out of memory or so.
> What can you see in elasticsearch logs?
>
> Don't you see some WARN about GC?
>
> --
> David 😉
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>
> Le 14 juin 2014 Ã 18:06, John Luko \<[john...@gmail.com](mailto:john...@gmail.com) \\> a
> écrit :
>
> We are running Elasticsearch with three nodes. The problem we keep
> running into is eventually Kibana begins to run extremely slow and soon
> after becomes unresponsive. If you hit refresh the wheels just keep
> spinning and nothing gets displayed. Restarting the cluster appears to
> correct this issue, but within 24 hours it begins again. Below is our
> health:
>
> {
> "cluster\_name" : "elasticsearch",
> "status" : "green",
> "timed\_out" : false,
> "number\_of\_nodes" : 4,
> "number\_of\_data\_nodes" : 3,
> "active\_primary\_shards" : 341,
> "active\_shards" : 682,
> "relocating\_shards" : 2,
> "initializing\_shards" : 0,
> "unassigned\_shards" : 0
> }
>
> When we initially set everything up we had a single node and we did have a
> span of three weeks where we didn't have to restart the Elasticsearch
> service. When we begin to pump netflow data in we began to have issues. I
> thought perhaps because we had logstash and ES running on one node that was
> causing the issue. Thus we added two virtual nodes and had one of them
> host the logstash for just Netflow. I thought with the clustering the
> issue would be resolved, but sadly I still have to start and stop the
> services everyday. When I look back at the data it flows in the entire
> time until flat lining for a few hours and then picks up again once I
> restart the services.
>
> Thanks in advance!
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> To view this discussion on the web visit
> [https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com)
> [https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com?utm_medium=email&utm_source=footer)
> .
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9a357da0-9ac6-4aaf-853d-f76f453c27d0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9a357da0-9ac6-4aaf-853d-f76f453c27d0%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [June 14, 2014, 5:02pm UTC](https://discuss.elastic.co/t/interesting-issue/18110/4 "2014-06-14T17:02:25Z")
May be you should gist your logs and copy the link here?
--
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)
@dadoonet | @elasticsearchfr
Le 14 juin 2014 à 18:44:07, John Luko ([johnluko@gmail.com](mailto:johnluko@gmail.com)) a écrit:
Is there a particular log I should look at? In elasticsearch.log I am seeing a lot of parser failures for a timestamp.
On Saturday, June 14, 2014 12:26:47 PM UTC-4, David Pilato wrote:
First guess is that you are running out of memory or so.
What can you see in elasticsearch logs?
Don't you see some WARN about GC?
--
David 😉
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 14 juin 2014 à 18:06, John Luko [john...@gmail.com](mailto:john...@gmail.com) a écrit :
We are running Elasticsearch with three nodes. The problem we keep running into is eventually Kibana begins to run extremely slow and soon after becomes unresponsive. If you hit refresh the wheels just keep spinning and nothing gets displayed. Restarting the cluster appears to correct this issue, but within 24 hours it begins again. Below is our health:
{
"cluster\_name" : "elasticsearch",
"status" : "green",
"timed\_out" : false,
"number\_of\_nodes" : 4,
"number\_of\_data\_nodes" : 3,
"active\_primary\_shards" : 341,
"active\_shards" : 682,
"relocating\_shards" : 2,
"initializing\_shards" : 0,
"unassigned\_shards" : 0
}
When we initially set everything up we had a single node and we did have a span of three weeks where we didn't have to restart the Elasticsearch service. When we begin to pump netflow data in we began to have issues. I thought perhaps because we had logstash and ES running on one node that was causing the issue. Thus we added two virtual nodes and had one of them host the logstash for just Netflow. I thought with the clustering the issue would be resolved, but sadly I still have to start and stop the services everyday. When I look back at the data it flows in the entire time until flat lining for a few hours and then picks up again once I restart the services.
## Thanks in advance!
## You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9a357da0-9ac6-4aaf-853d-f76f453c27d0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9a357da0-9ac6-4aaf-853d-f76f453c27d0%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/etPan.539c8021.79e2a9e3.a69e%40MacBook-Air-de-David.local](https://groups.google.com/d/msgid/elasticsearch/etPan.539c8021.79e2a9e3.a69e%40MacBook-Air-de-David.local).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@John\_Luko](https://discuss.elastic.co/u/John_Luko)
#### Post date: [June 14, 2014, 5:06pm UTC](https://discuss.elastic.co/t/interesting-issue/18110/5 "2014-06-14T17:06:42Z")
Sorry I am a n00b when it comes to ES. How would I go about that?
On Saturday, June 14, 2014 1:02:45 PM UTC-4, David Pilato wrote:
> May be you should gist your logs and copy the link here?
>
> --
> _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr
> [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr)
>
> Le 14 juin 2014 Ã 18:44:07, John Luko ([john...@gmail.com](mailto:john...@gmail.com) \) a
> écrit:
>
> Is there a particular log I should look at? In elasticsearch.log I am
> seeing a lot of parser failures for a timestamp.
>
> On Saturday, June 14, 2014 12:26:47 PM UTC-4, David Pilato wrote:
>
> > First guess is that you are running out of memory or so.
> > What can you see in elasticsearch logs?
> >
> > Don't you see some WARN about GC?
> >
> > --
> > David 😉
> > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
> >
> > Le 14 juin 2014 à 18:06, John Luko [john...@gmail.com](mailto:john...@gmail.com) a écrit :
> >
> > We are running Elasticsearch with three nodes. The problem we keep
> > running into is eventually Kibana begins to run extremely slow and soon
> > after becomes unresponsive. If you hit refresh the wheels just keep
> > spinning and nothing gets displayed. Restarting the cluster appears to
> > correct this issue, but within 24 hours it begins again. Below is our
> > health:
> >
> > {
> > "cluster\_name" : "elasticsearch",
> > "status" : "green",
> > "timed\_out" : false,
> > "number\_of\_nodes" : 4,
> > "number\_of\_data\_nodes" : 3,
> > "active\_primary\_shards" : 341,
> > "active\_shards" : 682,
> > "relocating\_shards" : 2,
> > "initializing\_shards" : 0,
> > "unassigned\_shards" : 0
> > }
> >
> > When we initially set everything up we had a single node and we did have
> > a span of three weeks where we didn't have to restart the Elasticsearch
> > service. When we begin to pump netflow data in we began to have issues. I
> > thought perhaps because we had logstash and ES running on one node that was
> > causing the issue. Thus we added two virtual nodes and had one of them
> > host the logstash for just Netflow. I thought with the clustering the
> > issue would be resolved, but sadly I still have to start and stop the
> > services everyday. When I look back at the data it flows in the entire
> > time until flat lining for a few hours and then picks up again once I
> > restart the services.
> >
> > ## Thanks in advance!
> >
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com)
> > [https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> To view this discussion on the web visit
> [https://groups.google.com/d/msgid/elasticsearch/9a357da0-9ac6-4aaf-853d-f76f453c27d0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9a357da0-9ac6-4aaf-853d-f76f453c27d0%40googlegroups.com)
> [https://groups.google.com/d/msgid/elasticsearch/9a357da0-9ac6-4aaf-853d-f76f453c27d0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/9a357da0-9ac6-4aaf-853d-f76f453c27d0%40googlegroups.com?utm_medium=email&utm_source=footer)
> .
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6908936b-02e7-46f9-873d-9fe3f2d1d152%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6908936b-02e7-46f9-873d-9fe3f2d1d152%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [June 14, 2014, 5:16pm UTC](https://discuss.elastic.co/t/interesting-issue/18110/6 "2014-06-14T17:16:32Z")
Upload your logs on [gist.github.com](http://gist.github.com) and paste the link here.
May be some more details here: [http://www.elasticsearch.org/help/](http://www.elasticsearch.org/help/)
--
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)
@dadoonet | @elasticsearchfr
Le 14 juin 2014 à 19:06:46, John Luko ([johnluko@gmail.com](mailto:johnluko@gmail.com)) a écrit:
Sorry I am a n00b when it comes to ES. How would I go about that?
On Saturday, June 14, 2014 1:02:45 PM UTC-4, David Pilato wrote:
May be you should gist your logs and copy the link here?
--
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)
@dadoonet | @elasticsearchfr
Le 14 juin 2014 à 18:44:07, John Luko ([john...@gmail.com](mailto:john...@gmail.com)) a écrit:
Is there a particular log I should look at? In elasticsearch.log I am seeing a lot of parser failures for a timestamp.
On Saturday, June 14, 2014 12:26:47 PM UTC-4, David Pilato wrote:
First guess is that you are running out of memory or so.
What can you see in elasticsearch logs?
Don't you see some WARN about GC?
--
David 😉
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 14 juin 2014 à 18:06, John Luko [john...@gmail.com](mailto:john...@gmail.com) a écrit :
We are running Elasticsearch with three nodes. The problem we keep running into is eventually Kibana begins to run extremely slow and soon after becomes unresponsive. If you hit refresh the wheels just keep spinning and nothing gets displayed. Restarting the cluster appears to correct this issue, but within 24 hours it begins again. Below is our health:
{
"cluster\_name" : "elasticsearch",
"status" : "green",
"timed\_out" : false,
"number\_of\_nodes" : 4,
"number\_of\_data\_nodes" : 3,
"active\_primary\_shards" : 341,
"active\_shards" : 682,
"relocating\_shards" : 2,
"initializing\_shards" : 0,
"unassigned\_shards" : 0
}
When we initially set everything up we had a single node and we did have a span of three weeks where we didn't have to restart the Elasticsearch service. When we begin to pump netflow data in we began to have issues. I thought perhaps because we had logstash and ES running on one node that was causing the issue. Thus we added two virtual nodes and had one of them host the logstash for just Netflow. I thought with the clustering the issue would be resolved, but sadly I still have to start and stop the services everyday. When I look back at the data it flows in the entire time until flat lining for a few hours and then picks up again once I restart the services.
## Thanks in advance!
## You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
## You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9a357da0-9ac6-4aaf-853d-f76f453c27d0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9a357da0-9ac6-4aaf-853d-f76f453c27d0%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6908936b-02e7-46f9-873d-9fe3f2d1d152%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6908936b-02e7-46f9-873d-9fe3f2d1d152%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/etPan.539c8371.12200854.a69e%40MacBook-Air-de-David.local](https://groups.google.com/d/msgid/elasticsearch/etPan.539c8371.12200854.a69e%40MacBook-Air-de-David.local).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@John\_Luko](https://discuss.elastic.co/u/John_Luko)
#### Post date: [June 15, 2014, 5:35pm UTC](https://discuss.elastic.co/t/interesting-issue/18110/7 "2014-06-15T17:35:10Z")
Thanks for the help thus far! I'll be posting the logs on Monday when I
get in the office, but in the mean time it has happened again. When I went
to Kibana I got an actual error that is pointing me in the right direction
I believe:
Oops! SearchPhaseExecutionException[Failed to execute phase [query], all
shards failed; shardFailures
{[CN48pxdyTUWBc8Iyft5vpQ][logstash-2014.06.15][2]:
EsRejectedExecutionException[rejected execution (queue capacity 1000) on
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$4@b597ac6]}]
We have I believe 24 cores on our main box so I figure I can set the thread
pool size to 120 and the queue size to -1?
On Saturday, June 14, 2014 1:16:52 PM UTC-4, David Pilato wrote:
> Upload your logs on [gist.github.com](http://gist.github.com) and paste the link here.
>
> May be some more details here: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help/)
>
> --
> _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr
> [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr)
>
> Le 14 juin 2014 Ã 19:06:46, John Luko ([john...@gmail.com](mailto:john...@gmail.com) \) a
> écrit:
>
> Sorry I am a n00b when it comes to ES. How would I go about that?
>
> On Saturday, June 14, 2014 1:02:45 PM UTC-4, David Pilato wrote:
>
> > May be you should gist your logs and copy the link here?
> >
> > ```
> > --
> >
> > ```
> >
> > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_
> > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr
> > [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr)
> >
> > Le 14 juin 2014 à 18:44:07, John Luko ([john...@gmail.com](mailto:john...@gmail.com)) a écrit:
> >
> > Is there a particular log I should look at? In elasticsearch.log I am
> > seeing a lot of parser failures for a timestamp.
> >
> > On Saturday, June 14, 2014 12:26:47 PM UTC-4, David Pilato wrote:
> >
> > > First guess is that you are running out of memory or so.
> > > What can you see in elasticsearch logs?
> > >
> > > Don't you see some WARN about GC?
> > >
> > > --
> > > David 😉
> > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
> > >
> > > Le 14 juin 2014 à 18:06, John Luko [john...@gmail.com](mailto:john...@gmail.com) a écrit :
> > >
> > > We are running Elasticsearch with three nodes. The problem we keep
> > > running into is eventually Kibana begins to run extremely slow and soon
> > > after becomes unresponsive. If you hit refresh the wheels just keep
> > > spinning and nothing gets displayed. Restarting the cluster appears to
> > > correct this issue, but within 24 hours it begins again. Below is our
> > > health:
> > >
> > > {
> > > "cluster\_name" : "elasticsearch",
> > > "status" : "green",
> > > "timed\_out" : false,
> > > "number\_of\_nodes" : 4,
> > > "number\_of\_data\_nodes" : 3,
> > > "active\_primary\_shards" : 341,
> > > "active\_shards" : 682,
> > > "relocating\_shards" : 2,
> > > "initializing\_shards" : 0,
> > > "unassigned\_shards" : 0
> > > }
> > >
> > > When we initially set everything up we had a single node and we did have
> > > a span of three weeks where we didn't have to restart the Elasticsearch
> > > service. When we begin to pump netflow data in we began to have issues. I
> > > thought perhaps because we had logstash and ES running on one node that was
> > > causing the issue. Thus we added two virtual nodes and had one of them
> > > host the logstash for just Netflow. I thought with the clustering the
> > > issue would be resolved, but sadly I still have to start and stop the
> > > services everyday. When I look back at the data it flows in the entire
> > > time until flat lining for a few hours and then picks up again once I
> > > restart the services.
> > >
> > > ## Thanks in advance!
> > >
> > > You received this message because you are subscribed to the Google
> > > Groups "elasticsearch" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > To view this discussion on the web visit
> > > [https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com)
> > > [https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d088f293-9dae-475a-a15e-076b3b773680%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > .
> > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/9a357da0-9ac6-4aaf-853d-f76f453c27d0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9a357da0-9ac6-4aaf-853d-f76f453c27d0%40googlegroups.com)
> > [https://groups.google.com/d/msgid/elasticsearch/9a357da0-9ac6-4aaf-853d-f76f453c27d0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/9a357da0-9ac6-4aaf-853d-f76f453c27d0%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/6908936b-02e7-46f9-873d-9fe3f2d1d152%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6908936b-02e7-46f9-873d-9fe3f2d1d152%40googlegroups.com)
> > [https://groups.google.com/d/msgid/elasticsearch/6908936b-02e7-46f9-873d-9fe3f2d1d152%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6908936b-02e7-46f9-873d-9fe3f2d1d152%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/42b28459-4f9a-4a80-9b7d-5d75f63d8fd4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/42b28459-4f9a-4a80-9b7d-5d75f63d8fd4%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:22am UTC](https://discuss.elastic.co/t/interesting-issue/18110/8 "2017-07-06T01:22:25Z")