Poor query performance after removing types for ES6


In preparation for the removal of types I'm experimenting with ES6.0.0-rc1 to upgrade from our ancient 2.4.6 install. I'm trying to take our index with hundreds of types and split it into multiple indexes (since it's 1 type per index now). I'm coming across speed issues with Kibana and the queries it runs. Was wondering if anyone had any insight on how to improve performance?

Existing System

  • ES 2.4.6
  • 1 index, 7 shards
  • 2 nodes
  • 1 index with approx 400 types
  • 630M documents
  • approx 10s to view discover tab (7 days) in Kibana

New System

  • ES 6.0.0-rc1
  • approx 400 indices, 1 shard each
  • 2 nodes
  • 1 index per type
  • 630M documents
  • approx 50s to view discover tab (7 days) in Kibana

Since the documents in each type have different combinations of fields and mappings (~1000 fields) I don't really want to have them in the same index as it ends up being extremely sparse and I end up with mapping conflicts everywhere.

Do I try to shoehorn the data back into a single index with mapping conflict hell, sparse data and and continue that way? Or maybe there's a better way to do this? Hope that makes sense!

For reference, here is the query Kibana runs:


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.