# Ingest: transforming multiple values in an array

**URL:** https://discuss.elastic.co/t/ingest-transforming-multiple-values-in-an-array/346147
**Category:** Elasticsearch
**Tags:** ingest-pipeline
**Created:** [October 31, 2023, 3:44pm UTC](https://discuss.elastic.co/t/ingest-transforming-multiple-values-in-an-array/346147 "2023-10-31T15:44:08Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![nemhods](https://avatars.discourse-cdn.com/v4/letter/n/48db29/32.png) [@nemhods](https://discuss.elastic.co/u/nemhods)
#### Post date: [October 31, 2023, 3:44pm UTC](https://discuss.elastic.co/t/ingest-transforming-multiple-values-in-an-array/346147/1 "2023-10-31T15:44:08Z")

</div>

Hey,

I'm looking for a way to transform

```auto
{
  "related": {
    "user": [
      "user1@domain",
      "user2@anotherdomain"
    ]
  }
}

```

into

```auto
{
  "related": {
    "user": [
      "user1@domain",
      "user1",
      "user2@anotherdomain",
      "user2"
    ]
  }
}

```

... using ingest processors. Basically, I have an array of User Principal Names and want to extend the field with just the sAMAccountNames also.

I've tried

- Just Grok: does not work on Array fields
- Just Split: does not work on Array fields
- Foreach + Grok: Target expression `%{DATA:_ingest.temp_user_name}@%{GREEDYDATA:_ingest.temp_user_domain}` always overrides the `_ingest.temp_user_name` and does not append - so at the end of the foreach, only one value is in \_ingest.temp\_user\_name.
- Foreach + Split without target\_field (in-place): yields `[["user1","domain"],["user2","anotherdomain"]]` - but also overrides the original data. plus I wouldn't know how to collect this into just `["user1", "user2"]`
- Foreach + Split with target\_field: Will also override the target\_field and only keep the value of the last iteration.

Anyone got more Ideas on how to solve this? do i have to go with script processors? ☹

---

<div class="post-metadata">

### Author: ![ashishtiwari1993](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ashishtiwari1993/32/135241_2.png) [@ashishtiwari1993](https://discuss.elastic.co/u/ashishtiwari1993)
#### Post date: [November 1, 2023, 9:22am UTC](https://discuss.elastic.co/t/ingest-transforming-multiple-values-in-an-array/346147/2 "2023-11-01T09:22:20Z")

</div>

HI @nemhods ,

Welcome to the Elastic community. I think yes you have to go with script processor, Since this requirement looks more custom.

Below is something which worked for me.

# Create a Pipeline

```sh
PUT _ingest/pipeline/test-p1
{
  "description": "Convert email addresses to both full and username format",
  "processors": [
    {
      "script": {
        "source": """
          ctx.related.tmp_user = new ArrayList();
          for (int i = 0; i < ctx.related.user.size(); i++) {
            def email = ctx.related.user[i];
            def username = email.splitOnToken('@')[0];
            ctx.related.tmp_user.add(username);
            ctx.related.tmp_user.add(email);
          }
          ctx.related.user = ctx.related.tmp_user;
          ctx.related.remove('tmp_user');
        """,
        "lang": "painless"
      }
    }
  ]
}

```

# Index sample data

```sh
POST test-index1/_doc?pipeline=test-p1
{
  "related":{
    "user":[
      "test@domain.com",
      "test1@domain1.com",
      "test2@domain2.com"
    ]
  }
}

```

# Output

```sh
GET test-index1/_search

```

Docs

```auto
{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": 1,
    "hits": [
      {
        "_index": "test-index1",
        "_id": "KuIriosBa7YTodDiw2wW",
        "_score": 1,
        "_source": {
          "related": {
            "user": [
              "test",
              "test@domain.com",
              "test1",
              "test1@domain1.com",
              "test2",
              "test2@domain2.com"
            ]
          }
        }
      }
    ]
  }
}

```

---

<div class="post-metadata">

### Author: ![nemhods](https://avatars.discourse-cdn.com/v4/letter/n/48db29/32.png) [@nemhods](https://discuss.elastic.co/u/nemhods)
#### Post date: [November 2, 2023, 8:35am UTC](https://discuss.elastic.co/t/ingest-transforming-multiple-values-in-an-array/346147/3 "2023-11-02T08:35:42Z")

</div>

Hey,

awesome, thanks for already providing the script solution.  
I've added a check to make sure the operation only runs on array members that contain an "@" sign.

```auto
ctx.related.tmp_user = new ArrayList();
for (int i = 0; i < ctx.related.user.size(); i++) {
    def email = ctx.related.user[i]; // email/upn
    ctx.related.tmp_user.add(email);
    if (email.contains("@")) {
        def username = email.splitOnToken('@')[0];
        ctx.related.tmp_user.add(username);
    }
}
ctx.related.user = ctx.related.tmp_user;
ctx.related.remove('tmp_user');

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 30, 2023, 8:36am UTC](https://discuss.elastic.co/t/ingest-transforming-multiple-values-in-an-array/346147/4 "2023-11-30T08:36:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.