# Painless: Loop all fields inside a document to extract phonenumbers

**URL:** <https://discuss.elastic.co/t/painless-loop-all-fields-inside-a-document-to-extract-phonenumbers/175558>\
**Category:** Elasticsearch\
**Tags:** painless\
**Created:** [April 5, 2019, 9:40am UTC](https://discuss.elastic.co/t/painless-loop-all-fields-inside-a-document-to-extract-phonenumbers/175558 "2019-04-05T09:40:01Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ErikvdVen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/erikvdven/32/24602_2.png) [@ErikvdVen](https://discuss.elastic.co/u/ErikvdVen)\
**Post date:** [April 5, 2019, 9:40am UTC](https://discuss.elastic.co/t/painless-loop-all-fields-inside-a-document-to-extract-phonenumbers/175558/1 "2019-04-05T09:40:01Z")

</div>

I'm indexing my documents using a third-party piece of software, so I am kinda stuck to the mapping made by that piece of software.

Now I want to make some small adaptions, so I can search any phonenumbers inside the documents. I thought an ingest pipeline, which searches for phonenumbers in each field and put them in an array, would be an ideal solution for now. So each document would get a separate field with an array of all phonenumbers, available inside that document.

The painless script works if I hardcode each field, but documents are kinda variable and there might be a LOT of different fields. So I thought I could just loop each field and extract, using regex, all phonenumbers. But I am not able to loop through all fields:

```auto
PUT _ingest/pipeline/merge_phonenumbers
{
  "description": "Combine multiple phonenumbers fields in one array for aggregation",
  "processors": [
    {
      "script": {
        "source": """
          void loopAllFields(def x){
            for (int i = 0; i < x._fields.length; i++) {
              def phonenumber = /([0-9]+)/.matcher(x[i].data);
              if(phonenumber != null)
              {
                x.combined_phonenumbers.add(phonenumber);
              }
            }
          }
          
          if (ctx.combined_phonenumbers == null) {
              ctx.combined_phonenumbers = new ArrayList();
          } 
          
          loopAllFields(ctx);
        """
      }
    }
  ]
}

```

Any thoughts if it's possible to loop over all fields? I have to mention some fields might be nested objects, which is the reason I would like to create a recursive function.

**EDIT:**  
I've made some progress:

```auto
PUT _ingest/pipeline/merge_phonenumbers
{
  "description": "Combine multiple phonenumbers fields in one array for aggregation",
  "processors": [
    {
      "script": {
        "source": """
          void loopAllFields(def x){
            if(x instanceof Map){
              for (entry in x.entrySet()) {
                if (entry.getKey() == "_source") { 
                  continue;
                }
                if(entry instanceof Map || 
                   entry instanceof ArrayList ||
                   entry instanceof HashMap)
                {
                  loopAllFields(entry);
                  continue;
                }
                def phonenumber = /[0-9]+/.matcher(entry.getValue());
                x.combined_phonenumbers.add(phonenumber);
              }
            }
          }
          
          if (ctx.combined_phonenumbers == null) {
              ctx.combined_phonenumbers = new ArrayList();
          } 
          
          loopAllFields(ctx);
        """
      }
    }
  ]
}

```

This function does loop but gives me an exception:

```auto
ScriptException[runtime error]; nested: ClassCastException[Cannot cast java.util.ArrayList to java.lang.CharSequence];

```

---

<div class="post-metadata">

**Author:** ![ErikvdVen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/erikvdven/32/24602_2.png) [@ErikvdVen](https://discuss.elastic.co/u/ErikvdVen)\
**Post date:** [April 8, 2019, 2:28pm UTC](https://discuss.elastic.co/t/painless-loop-all-fields-inside-a-document-to-extract-phonenumbers/175558/2 "2019-04-08T14:28:11Z")

</div>

Got a working solution, forgot I had to typecast the value to a string first before extracting the numbers:

```auto
PUT _ingest/pipeline/merge_fields
{
  "description": "Look multiple fields for numbers and put them in one array for aggregation",
  "processors": [
    {
      "script": {
        "source": """
          def loopAllFields(def x, def list){
            if(x instanceof Map){
              for (entry in x.entrySet()) {
                if (entry.getKey() == "_source") { 
                  continue;
                }
                
                if(entry instanceof Map || 
                   entry instanceof ArrayList ||
                   entry instanceof HashMap)
                {
                  loopAllFields(entry, list);
                }

                def s = entry.getValue().toString();
                def m = /[0-9]{10,20}/.matcher(s);
                
                while(m.find()) 
                {
                  def keyword = m.group(0).trim();
                  list.add(keyword);
                }
              }
            }
            return list;
          }

          def l = new ArrayList();
          l = loopAllFields(ctx, l);
          if (ctx.combined == null) {
              ctx.combined = l;
          } 
        """
      }
    }
  ]
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 6, 2019, 2:28pm UTC](https://discuss.elastic.co/t/painless-loop-all-fields-inside-a-document-to-extract-phonenumbers/175558/3 "2019-05-06T14:28:12Z")

</div>

Closed on May 6, 2019, 2:28pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.