# Flatten Array in Ingest Pipeline with Painless

**URL:** <https://discuss.elastic.co/t/flatten-array-in-ingest-pipeline-with-painless/277383>\
**Category:** Elasticsearch\
**Tags:** painless, ingest-pipeline\
**Created:** [June 29, 2021, 8:27pm UTC](https://discuss.elastic.co/t/flatten-array-in-ingest-pipeline-with-painless/277383 "2021-06-29T20:27:13Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![nicjp](https://avatars.discourse-cdn.com/v4/letter/n/a587f6/32.png) [@nicjp](https://discuss.elastic.co/u/nicjp)\
**Post date:** [June 29, 2021, 8:27pm UTC](https://discuss.elastic.co/t/flatten-array-in-ingest-pipeline-with-painless/277383/1 "2021-06-29T20:27:13Z")

</div>

Hi all,

I'm working on getting some logs onboarded and I'm having an issue with a field that contains Key/Values inside an array. (For reference please see below)

```auto
{
  "redis_cache_read_bytes": 741,
  "format": "json",
  "params": [
    {
      "value": "backend",
      "key": "namespace_id"
    },
    {
      "value": "web-app",
      "key": "project_id"
    },
    {
      "value": "617",
      "key": "id"
    }
  ],
  ...
}

```

What I would like to be able to do is something like this:

```auto
{
  "redis_cache_read_bytes": 741,
  "format": "json",
  "item": {
    "backend": "namespace_id",
    "project_id": "web-app",
    "id": "617"
  }
}

```

I've considered a few ways of doing this (foreach processor into script processor, for loop via script processor etc. ) but I can't seem to make it do anything other than create a document with the last k/v value in the array (example below)

```auto
for (item in ctx.params) { ctx.item = item; }

```

```auto
{
  "item": {
    "value": "617",
    "key": "id"
  },
  "redis_cache_read_bytes": 741,
  "format": "json",
  "params": [
    {
      "value": "backend",
      "key": "namespace_id"
    },
    {
      "value": "web-app",
      "key": "project_id"
    },
    {
      "value": "617",
      "key": "id"
    }
  ],
...
}

```

---

<div class="post-metadata">

**Author:** ![stu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stu/32/75063_2.png) [@stu](https://discuss.elastic.co/u/stu)\
**Post date:** [June 30, 2021, 6:06pm UTC](https://discuss.elastic.co/t/flatten-array-in-ingest-pipeline-with-painless/277383/2 "2021-06-30T18:06:06Z")

</div>

Hi @nicjp, the code above is going through all params and assigning the current item, `item`, in the param list to `ctx.item`. This will overwrite whatever was in `ctx.item`, which is why you end up with the last k/v value.

Instead of that, you should create a new Map in `ctx.item` and update it with the values from the current item in the param list.

Try this:

```auto
ctx.item = new HashMap();
for (kv in ctx.params) {
  ctx.item[kv.get("key")] = kv.get("value");
}

```

---

<div class="post-metadata">

**Author:** ![nicjp](https://avatars.discourse-cdn.com/v4/letter/n/a587f6/32.png) [@nicjp](https://discuss.elastic.co/u/nicjp)\
**Post date:** [July 1, 2021, 10:15am UTC](https://discuss.elastic.co/t/flatten-array-in-ingest-pipeline-with-painless/277383/3 "2021-07-01T10:15:58Z")

</div>

that is perfect - thank you @stu

I reached out to a former colleague (shout out to @Marius_Iversen) and he provided me with the following (very similar!) result.

```auto
 {
        "script": {
          "lang": "painless",
          "if": "ctx.params != null && ctx.params instanceof List",
          "source": """
          ctx.item = new HashMap();
          for (i in ctx.params) {
            ctx.item[i['key']] = i['value'];
          }
          """
        }
      }

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 29, 2021, 10:16am UTC](https://discuss.elastic.co/t/flatten-array-in-ingest-pipeline-with-painless/277383/4 "2021-07-29T10:16:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.