Here is my issue: I have to produce visualizations for some extended reporting. For brevity I will simply tell you I end up with events that look like they do below. The key thing is that I have multiple events per day with the same simple data stamp and growing byte counts. The byte counts are the max at various times during the day, so in the end I need the max of the field, not the sum of the bytes field.
In the end, I get data that looks like:
bizunit: finance simple_date: 2020-02-01 bytes: 1234
bizunit: sales simple_date: 2020-02-01 bytes: 1234
bizunit: finance simple_date: 2020-02-01 bytes: 1934
bizunit: sales simple_date: 2020-02-01 bytes: 19034
bizunit: finance simple_date: 2020-02-01 bytes: 2234
bizunit: sales simple_date: 2020-02-01 bytes: 20345
bizunit: finance simple_date: 2020-02-01 bytes: 4234
bizunit: sales simple_date: 2020-02-01 bytes: 569834
What I need to do is to be able to show things like how many bytes each bizunit consumed per day, month, etc? I have tried doing an aggregation which will show the results, but the catch is that i need to get this to a place that kibana can visualize and their are no kibana options for summarizing a max per day and using that max to sum up over weeks or months. If I use sum, it will only sum all of the numbers in the field, not take the max. If I use max, it will give the largest number a bizunit or a day had for the month, but not each day+each bizunit, if that makes sense.
So, a very long story short, I need to be able to either put in an aggregation query which buckets all of this correctly and have kibana visualizations be able to read it and graph it, OR I need to be able to reindex my data in such a way as I only get one event per day that has a max bytes field, data and bizunit. That way, kibana could just sum it all.
I can add my aggregation that buckets if that helps, but I don't think that is really the issue. I have attempted _reindex, but that does not seem to handle the aggregation fields, it just transfers all the previous events to the new one. Can anyone help?