Weekly indices with name as starting day of the week


(Manikanth Reddy) #1

Hi,

As of now, we are using daily indices and are trying to create weekly indices going forward. I have tried configuring indexprefix-%{+xxxx.ww} but it is giving indices by week number out of year. I have been trying to find any solution available but couldn't find any. Could you please help me in this. Thanks.

index name should be: indexprefix_2017.10.16
Considering 2017.10.16 as first day in the week.


(Magnus Bäck) #2

This isn't supported out of the box, but you could certainly write a short piece of Ruby code in a ruby filter that inspects @timestamp, figures out the date of closest preceding Monday, and stores that in a field that you reference in the elasticsearch output configuration.


(Manikanth Reddy) #3

@magnusbaeck Thanks. I will try to do that but I have one more doubt. We are also using metricbeat for some of our logs. How can I achieve the same when using metricbeat as we don't use logstash for parsing here. Please let me know your thoughts. Thanks.


(Magnus Bäck) #4

I'm pretty sure you can't do that with Metricbeat.


(Manikanth Reddy) #5

@magnusbaeck hmm..is there any way, we can achieve weekly indices in logstash as well as in metricbeat apart from ndexprefix-%{+xxxx.ww}. Thanks


(Magnus Bäck) #6

Not that I know of. I don't understand why you care so much about the index names. Why is that relevant?


(Manikanth Reddy) #7

Currently we have daily indices. If we move to weekly indices, it is confusing to analyze the historical data as indices will be like indexprefix_2017.43 because clients need to remember the week number out of year. But If we have weekly indices with name as starting day of the week, then it will be easy to find out, data to which week or date it belongs.


(Magnus Bäck) #8

But surely humans aren't choosing index names by hand?


(Manikanth Reddy) #9

Yes. Some times we might need that. When we ingest historical logs which are older than couple of weeks, it will be easy to find whether everything looks good or not if we have date's in the index name. But if we have week numbers, then people will be confused and not sure which week their data belongs? Accept my apologies if I didn't understand your question properly.


(Magnus Bäck) #10

I think you're solving the wrong problem. If index names are confusing to people, don't expose them to the index names. Over and out.


(Manikanth Reddy) #11

@magnusbaeck thanks for the help. We are going with week numbers instead of dates :slight_smile:


(system) #12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.