Logging indices for separate websites

Elastic Stack Elasticsearch
1

Hey,

We're running multiple webshops on the same application. And we're logging some API calls we're making (with separate index for each logging use-case). We have one shared logging elasticsearch cluster.
But for the cluster, is it better to have one index with documents from all the webshops, or to split them up with one index each (still separating the separate logging use-cases)?

At the moment we have them split up:
shopA-apiX-logging-2020.08
shopB-apiX-logging-2020.08
shopA-apiX-logging-2020.09
shopB-apiX-logging-2020.09

But with ~10 different webshops, the amount of indices on our cluster keeps growing. While the indices are not that big.
Biggest index is 2.1gb, but the median is at 98mb.

2

Having lots of small indices and shards is inefficient and can cause problems, so I would probably recommend consolidating indices. Another way might be to have a monthly index per user rather than per application.

3

Look at using ILM to manage the rollover process.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.