Weekly indices with name as starting day of the week

Elastic Stack Logstash
1

Hi,

As of now, we are using daily indices and are trying to create weekly indices going forward. I have tried configuring indexprefix-%{+xxxx.ww} but it is giving indices by week number out of year. I have been trying to find any solution available but couldn't find any. Could you please help me in this. Thanks.

index name should be: indexprefix_2017.10.16
Considering 2017.10.16 as first day in the week.

1 Like
2

This isn't supported out of the box, but you could certainly write a short piece of Ruby code in a ruby filter that inspects @timestamp, figures out the date of closest preceding Monday, and stores that in a field that you reference in the elasticsearch output configuration.

3

@magnusbaeck Thanks. I will try to do that but I have one more doubt. We are also using metricbeat for some of our logs. How can I achieve the same when using metricbeat as we don't use logstash for parsing here. Please let me know your thoughts. Thanks.

4

I'm pretty sure you can't do that with Metricbeat.

5

@magnusbaeck hmm..is there any way, we can achieve weekly indices in logstash as well as in metricbeat apart from ndexprefix-%{+xxxx.ww}. Thanks

6

Not that I know of. I don't understand why you care so much about the index names. Why is that relevant?

7

Currently we have daily indices. If we move to weekly indices, it is confusing to analyze the historical data as indices will be like indexprefix_2017.43 because clients need to remember the week number out of year. But If we have weekly indices with name as starting day of the week, then it will be easy to find out, data to which week or date it belongs.

8

But surely humans aren't choosing index names by hand?

9

Yes. Some times we might need that. When we ingest historical logs which are older than couple of weeks, it will be easy to find whether everything looks good or not if we have date's in the index name. But if we have week numbers, then people will be confused and not sure which week their data belongs? Accept my apologies if I didn't understand your question properly.

10

I think you're solving the wrong problem. If index names are confusing to people, don't expose them to the index names. Over and out.

11

@magnusbaeck thanks for the help. We are going with week numbers instead of dates :slight_smile:

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.