Inline script to loop through unknown field names

Elastic Stack Elasticsearch
1

Hi,
I have in my document this construct
"data": {
"group_randomId1": {
"date1": "ISO8601",
"date2": "ISO8601"
},
"group_randomId2": {
"date1": "ISO8601",
"date2": "ISO8601"
},
"group_randomIdN": {
"date1": "ISO8601",
"date2": "ISO8601"
}
}

What I want to do is use a percentile aggregation to compute the percentiles on date2-date1.
I'm blocked at this point because I actually would likle to have something like:
{
"size": 0,
"aggs": {
"toMove": {
"percentiles": {
"script": {
"lang": "painless",
"source": "doc['data.*.date2'].value.getMillis() - doc['data.*.date1'].value.getMillis()"
}
}
}
}
}

The '*' notation is not supported so how can I write my script to loop through the different groups?
The data groups are added by upsert operations (and a pipeline processor).

Thanks for your help!

2

bumping this message.

Thinking more about it, even if I use an array representation
"data": [
{
"date1": "ISO8601",
"date2": "ISO8601"
},
{
"date1": "ISO8601",
"date2": "ISO8601"
},
{
"date1": "ISO8601",
"date2": "ISO8601"
}
]
that would simplify the iteration part but

  1. I'm not sure I can use an ingest script pipeline to update an existing data field incrementally?
  2. I will probably have to use an upsert + a script from my client app to update this document data field
  3. not sure I can keep the correlation between a date1 and date2 field in my search query percentile query?

I really needed to use an ingest pipeline to update this data field because one of the date I need to systematically inject is the indexation timestamp : {{_ingest.timestamp}}