Mapping optimizations for Aggregations

Elastic Stack Elasticsearch
1

Hi,

We are using ES + Kibana for aggregations only.

Is there anything we can do to optimize the mapping and remove stuff we don't need?

Our main concern is memory usage. Maybe specific filteres/terms/norms or things like that that are not really documented anywhere...

Our ES version is 1.5.2

2

Hi! Just disable indexing fields you never needed index: no and enable doc_values: true for fields you are using for aggregations. Of course you can also set not_analized for strings and other fields that should not be analyzed at all. Hope that help.

PS. try to upgrade to latest 1.7.x because there were some optimizations and bugfixes.

3

Upgrade to 2.X instead, you'll get better performance :slightly_smiling:

4

I would really like to do that, but i can't take my whole cluster down.

I need a way to reindex all the data to a new cluster. @warkolm know any?

5

Do you need to reindex though? Have you run the migration plugin to check?

6

I ran the migration plugin and there were no errors.

However, when i tried to reindex some data to a new cluster i faced some issues with a geoip field.
I don't see any other way without reindexing as i can't take the cluster down.

7

But why are you reindexing?
If the migration plugin found nothing then you can do an in-situ upgrade! (Of course I'd still take a backup)

8

Following the table here: https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html i must do a complete cluster restart no?

9

Ah right, sorry I missed that bit. You do yes.

I use Logstash for reindexing - https://gist.github.com/markwalkom/8a7201e3f6ea4354ae06