Hello,
I'd like to index a part of my varnish logs to spend less disk space ; is it possible by filtering on a time field or by sampling (perhaps a random in a ruby script ?) ?
I have two fields with a time : [14/Jan/2016:15:59:34 +0100] or 2016-01-14T14:59:34.839Z
The best solution would be to index one event per second or per minute but I think it isn't possible.
So I've got two ideas to solve this problem :
- use a random in a ruby filter
- use a filter which filter events on a part of a time field (only if it contains 0 second for example)
Which solution would be the best for you or is there a better way to solve this problem ?
Thanks.
Frederic.