I'm used to processing a huge amount of raw data (typically hundreds of thousands of rows, whole months of data per day, per hour, per detail). In Excel I perform some aggregations and calculations as shown in picture and in the attached spreadsheet. Since Kibana is configured with some limitations (i.e. no more than 65536) and the admins confirmed that they cannot remove this restriction, I need to aggregate them directly into it. But it is not easy for me to do it like in Excel and this is why I call for your help.
I derive from a large string in the field "detail" some specific patterns that identify the type of detail for me (called "LU" and that can be "PEM" or "FEU"). Inside this detail type ("PEM" or "FEU") there are a few rows (instances) that I need to average among them but excluding the lowest three. In Excel I do it with text functions (thanks to MID function I extract the interested pattern that I call 'LU') and with the LARGE and the AVERAGE functions I finally calculate the outcome each hour, per "userbot", per detail-type ("LU")
In your opinion are these steps (or part of them) possible to execute directly in Kibana?