More than 1 day's log per index?

Elastic Stack Elasticsearch
1

It looks like Marvel create one index per day. Things like .marvel-es-1-2016.10.04. Can I configure this to be a bigger date range? for example one index per week or per month?

I want to keep a longer history but don't want to have too many indexes like .marvel-es-1-2016.10.04.

Thank you!

2

Yes, within config/elasticsearch.yml, you can set the time format of the index for the exporters:

marvel.agent.exporters:
  id1:
    # ...
    index.name.time_format: YYYY.ww

This will change the index pattern that it uses when it creates indices, which the UI-side will pick up automatically. This is better documented in 5.x, but it's shown in 2.x docs under the large exporter block.

3

Hi Vincent,

Curious to know why you don't want too many indices. Is it because it's hard to maintain?

Thanks,
Bohyun

4

Too many index means too many segments to maintain, which means too many CPU, Ram for elasticsearch to use on those indexes. I would rather put those resource onto my data.

Cause I learned from elasticsearch: we don't want too many shards/segments.

5

Awesome! Thank you!