# Access integer array via doc without sorting?

**URL:** <https://discuss.elastic.co/t/access-integer-array-via-doc-without-sorting/219499>\
**Category:** Elasticsearch\
**Created:** [February 16, 2020, 3:40am UTC](https://discuss.elastic.co/t/access-integer-array-via-doc-without-sorting/219499 "2020-02-16T03:40:31Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![PhracturedBlue](https://avatars.discourse-cdn.com/v4/letter/p/9de0a6/32.png) [@PhracturedBlue](https://discuss.elastic.co/u/PhracturedBlue)\
**Post date:** [February 16, 2020, 3:40am UTC](https://discuss.elastic.co/t/access-integer-array-via-doc-without-sorting/219499/1 "2020-02-16T03:40:32Z")

</div>

I was trying to do some scripting on the new 'histogram' datatype that comes with 7.6, but histograms don't support scripting.  
I thought I might be able to use arrays instead, but have not had any luck because arrays are sorted when accessing the \_doc object. I know I could use \_source, but that will be painful in the future when I have a large number of documents.

what I have:

```auto
{
    "_source" : {
      "data": {
         "values" : [5.0, 5.5, 6.0, 7.0,10.0, 20.0, 30.0],
         "counts" : [0, 60, 0, 0, 0, 0, 0]
      }
    },
    "fields" : {
      "scripted" : [0, 0, 0, 0, 0, 0, 60]
    }
  },

```

with a query like:

```auto
    "script_fields": {
      "scripted": {
        "script": {
          "source": "doc['data.counts']" 
        }
      }

```

I thought I could possibly get around this by altering the mapping to have 'index: false, doc\_values: true', but it had no effect on the query.

Is there any work-around, or should I just store my data as an object instead of an array?

---

<div class="post-metadata">

**Author:** ![Ignacio\_Vera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ignacio_vera/32/36674_2.png) [@Ignacio\_Vera](https://discuss.elastic.co/u/Ignacio_Vera)\
**Post date:** [February 17, 2020, 4:27pm UTC](https://discuss.elastic.co/t/access-integer-array-via-doc-without-sorting/219499/2 "2020-02-17T16:27:55Z")

</div>

Hi!,

The histogram field does not support scripting indeed. I am very curious what is your use case that it needs to access data counts through a script, are you able to share this need?

---

<div class="post-metadata">

**Author:** ![PhracturedBlue](https://avatars.discourse-cdn.com/v4/letter/p/9de0a6/32.png) [@PhracturedBlue](https://discuss.elastic.co/u/PhracturedBlue)\
**Post date:** [February 17, 2020, 4:43pm UTC](https://discuss.elastic.co/t/access-integer-array-via-doc-without-sorting/219499/3 "2020-02-17T16:43:56Z")

</div>

I store pre-aggregated data in elasticsearch. I want to generate heatmaps fom this data, but Kibana doesn't support heatmaps from pre-aggregated data, so instead I generate these using Grafana.  
In the past I've done this be defining queries per-swimlane with data like:

```auto
    data.bucket_0_5: 10,
    data.bucket_5_10: 5,
    data.bucket_10_20: 0,
    ...

```

And just doing a query on each field for each row in the heatmap.

I was hoping to do this with a elasticsearch histogram instead because it should be more space-efficient, as well as meaning I don't need to hand-craft each query in grafana, but there isn't really a way to extract a specific index that I can find.  
I tried substituting my key/value pairs with arrays, but that doesn't work due to array sorting.  
Instead I've currently chosen a prometheus-esque storage solution:

```auto
    data.values: [5, 10, 20, ...]
    data.counts: [10, 15, 15, ...]

```

Where the counts are cumulative.  
I can then use scripting to extract data by index:

```auto
    return doc['data.counts'][i] - doc['data.counts'][i-1]

```

I can also generate a linearized 'max' approximation via:

```auto
    def c = doc['data.counts'];
    def v = doc['data.values'];
    for (i=0; i < c.length-1; i++) {
        if (c[i] ==c[i+1]) {
            return v[i]
        }
    }
    return v[-1];

```

This solution is actually less space-efficient than the original key/value solution, but it does make it easier to do various manipulations like the one above

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 16, 2020, 4:43pm UTC](https://discuss.elastic.co/t/access-integer-array-via-doc-without-sorting/219499/4 "2020-03-16T16:43:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.