Kibana never fully loads, stays searching

Hi,

Strange issue. Kibana and indexing works correctly at first but after
several weeks, the UI always shows that Kibana is searching. No matter what
I try to query or time interval, it's always searching and never returns
anything. I don't see anything funky in the logs and if I delete all my
indices, everything works fine. The only thing I can think of is that every
6 days, I delete an index that is 6 days old, because I don't have enough
machines and disk space.

Cluster health seems okay:

{

  • cluster_name: "prod",
  • status: "green",
  • timed_out: false,
  • number_of_nodes: 12,
  • number_of_data_nodes: 11,
  • active_primary_shards: 154,
  • active_shards: 154,
  • relocating_shards: 0,
  • initializing_shards: 0,
  • unassigned_shards: 0,
  • number_of_pending_tasks: 0

}

Running on Ec2. performance looks okay on each node. My indexes total about
8TB across 11 machines. I have one machine set as a read only ES node and
that's where Kibana lives as well. The instance types are m3.2xlarge.

Any suggestions would be appreciated. Thanks!

Tony

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9313a568-a2d6-4ffa-b5b1-629349fc058a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.