I have an ammount of tickets over two years. My goal is to calculate the average, which I do with the .movingaverage(). The problem is that I don't want the weekends included in the average. I managed to hide the dates with 0 tickets with .if(lt, 1, null).fit(none), but I don't think that's the real solution, because it also hides entire weeks like christmas, where no tickets are submitted.
In the search/query bar you can write something like myWeekendField:false and try and see if that works to exclude the weekends?
There isn't a built in easy config to hide weekends. A possible solution could be date range aggregation with ranges set to not include weekends. Or possibly at index time to add a field stating whether it's a weekend or not and filter it out.
Thanks
Rashmi
If I type myWeekendField:false in the searchbar, like you suggested, I just get anything! The funktion doesn't seem to work. How can I manage to get it work? And also how can I do a date range aggregation, in timelion?
am cc ing @timroes here for more clarification.
Thanks
Rashmi
Hi,
you cannot filter out weekends (or other specific weekdays) with regular date histogram aggregations. You will need to write that information into your documents when indexing them. Meaning you should use whatever system inserts those documents, to calculate whether that event occurred on a weekend or not and write an appropriate boolean flag to the document.
In that case you can then create a filter on that field, like isWeekend:false (if you named the fields isWeekend).
You can achieve something similar with scripted fields, and calculate that on the fly (warning: is slower), but that won't help you for filtering, since scripted fields are not query-able. You could only use the field in classical aggregations (not timelion), which also won't cover your needs here. So I would recommend adding that flag to the documents before indexing them - maybe even the weekday directly, so that you can later filter on any weekdays you wish.
Cheers,
Tim
Wouldn't it be much easier if Elasticsearch or Kibana could do that?
Maybe it would, but this is a lot of domain knowledge. So why only weekend? Why not also the weekday, the hour of the day, etc.? If Elasticsearch would need to store all those for any date field, it would waste a lot of storage for a lot of people, who might never want to use that. And if it would calculate it on the fly, we are basically back to scripted fields, which will just be too slow for big datasets.
So it doesn't have possibilities to filter for those, the same as it doesn't give you any arbitrary options on other fields, e.g. you can neither say, filter for DNS names if you just stored IP addresses, just because it's an IP address, doesn't give Elasticsearch any more domain knowledge, so you will need to store in your data, what you want to use.
This is something that the data preprocessing pipelines should do. And it seems, that e.g. Logstash is able to do this: https://stackoverflow.com/a/32223083/1044403
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.