# Create new field on object using painless

**URL:** <https://discuss.elastic.co/t/create-new-field-on-object-using-painless/245211>\
**Category:** Elasticsearch\
**Created:** [August 17, 2020, 9:53am UTC](https://discuss.elastic.co/t/create-new-field-on-object-using-painless/245211 "2020-08-17T09:53:00Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![david.preston](https://avatars.discourse-cdn.com/v4/letter/d/51bf81/32.png) [@david.preston](https://discuss.elastic.co/u/david.preston)\
**Post date:** [August 17, 2020, 9:53am UTC](https://discuss.elastic.co/t/create-new-field-on-object-using-painless/245211/1 "2020-08-17T09:53:00Z")

</div>

Hi I'm trying to set an object field from another using an update query but am getting an error.

Currently by doc has an event object in the root of the doc with a field: id  
i.e.

```
{
  event { id: 1 }
}

```

all the documents have a duration\_secs field:

i.e.

```
{
  event { id: 1 },
  duration_secs: 10
}

```

But to make existing documents more ECS compliant I want to add the duration to the event object as duration..  
i.e.

```
{
  event { id: 1, duration: 10 },
  duration_secs: 10
}

```

I have used an \_update\_by\_query before, but am getting an error.  
Here is my request:

```
POST myindex-*/_update_by_query
{
  "query": {
   "bool": {
      "should": [
        {
          "exists": {
            "field" : "duration_secs"
          }
        },
       {
        "exists": {
          "field": "event"
        }
       }
      ]
   }
  },
  "script" : {
      "inline": "ctx._source.event.duration = ctx._source.duration_secs; ",
      "lang": "painless"
  }
}

```

And the error is:

```
{
  "error" : {
    "root_cause" : [
      {
        "type" : "script_exception",
        "reason" : "runtime error",
        "script_stack" : [
          "ctx._source.event.duration = ctx._source.duration_secs; ",
          " ^---- HERE"
        ],
        "script" : "ctx._source.event.duration = ctx._source.duration_secs; ",
        "lang" : "painless",
        "position" : {
          "offset" : 17,
          "start" : 0,
          "end" : 56
        }
      }
    ],
    "type" : "script_exception",
    "reason" : "runtime error",
    "script_stack" : [
      "ctx._source.event.duration = ctx._source.duration_secs; ",
      " ^---- HERE"
    ],
    "script" : "ctx._source.event.duration = ctx._source.duration_secs; ",
    "lang" : "painless",
    "position" : {
      "offset" : 17,
      "start" : 0,
      "end" : 56
    },
    "caused_by" : {
      "type" : "null_pointer_exception",
      "reason" : "Cannot invoke \"Object.getClass()\" because \"callArgs[0]\" is null"
    }
  },
  "status" : 400
}

```

I cant find any resources on the error:  
_**Cannot invoke "Object.getClass()" because "callArgs[0]" is null**_  
But suspect it is a scoping issue of some sort?

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [August 17, 2020, 11:44am UTC](https://discuss.elastic.co/t/create-new-field-on-object-using-painless/245211/2 "2020-08-17T11:44:40Z")

</div>

Check to see if duration\_secs has an entry (a number) in every record or there might be some that are NULL. That is what I am seeing with the `null_pointer_exception` error.

---

<div class="post-metadata">

**Author:** ![david.preston](https://avatars.discourse-cdn.com/v4/letter/d/51bf81/32.png) [@david.preston](https://discuss.elastic.co/u/david.preston)\
**Post date:** [August 17, 2020, 12:59pm UTC](https://discuss.elastic.co/t/create-new-field-on-object-using-painless/245211/3 "2020-08-17T12:59:01Z")

</div>

My query is removing any results that dont have a duration\_secs defined. The reason for this is: not all message have the duration\_secs field, it is added later.

I've double checked the values in the duration\_secs field, and where it exists the value is numeric.

I think its because the detination field doesnt not exist: ctx.\_source.event.duration

```
"ctx._source.event.duration = ctx._source.duration_secs; ",
" ^---- HERE"

```

But I would have thought it would create this - when i have done the same with text fields it has.

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [August 17, 2020, 1:40pm UTC](https://discuss.elastic.co/t/create-new-field-on-object-using-painless/245211/4 "2020-08-17T13:40:18Z")

</div>

> [@david.preston](#):
>
> ```auto
> "inline": "ctx._source.event.duration = ctx._source.duration_secs; ",
> 
> ```

Try

```auto
      "inline": "ctx._source.event_duration = ctx._source.duration_secs; ",

```

If that works then it just doesn't let you use the dot notation like you are trying.

---

<div class="post-metadata">

**Author:** ![david.preston](https://avatars.discourse-cdn.com/v4/letter/d/51bf81/32.png) [@david.preston](https://discuss.elastic.co/u/david.preston)\
**Post date:** [August 17, 2020, 2:08pm UTC](https://discuss.elastic.co/t/create-new-field-on-object-using-painless/245211/5 "2020-08-17T14:08:47Z")

</div>

> [@aaron-nimocks](#):
>
> `"inline": "ctx._source.event_duration = ctx._source.duration_secs; ",`

This would create another top level field, I want to update the event object with a duration field - to be ECS compliant.

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [August 17, 2020, 2:13pm UTC](https://discuss.elastic.co/t/create-new-field-on-object-using-painless/245211/6 "2020-08-17T14:13:26Z")

</div>

Understand. That is just a test to see if that is where the problem is.

Can you post your mapping? Interested in the event field.

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [August 17, 2020, 2:20pm UTC](https://discuss.elastic.co/t/create-new-field-on-object-using-painless/245211/7 "2020-08-17T14:20:05Z")

</div>

```auto
PUT test/_doc/1
{
  "event": { "id": 1 },
  "duration_secs": 10
}

POST test/_update_by_query
{
  "query": {
   "bool": {
      "should": [
        {
          "exists": {
            "field" : "duration_secs"
          }
        },
       {
        "exists": {
          "field": "event"
        }
       }
      ]
   }
  },
  "script" : {
      "inline": "ctx._source.event.duration = ctx._source.duration_secs; ",
      "lang": "painless"
  }
}

GET test/_search

```

I ran the above and got the below results.

```auto
    "hits" : [
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "event" : {
            "duration" : 10,
            "id" : 1
          },
          "duration_secs" : 10
        }
      }
    ]

```

---

<div class="post-metadata">

**Author:** ![david.preston](https://avatars.discourse-cdn.com/v4/letter/d/51bf81/32.png) [@david.preston](https://discuss.elastic.co/u/david.preston)\
**Post date:** [August 17, 2020, 2:32pm UTC](https://discuss.elastic.co/t/create-new-field-on-object-using-painless/245211/8 "2020-08-17T14:32:53Z")

</div>

The mapping for event on the index is:

```
"event" : {
     "properties" : {
         "id" : {
              "type" : "text"
          }            
     }
}

```

Do i need to update the mapping on existing indices, for this field to work?

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [August 17, 2020, 2:34pm UTC](https://discuss.elastic.co/t/create-new-field-on-object-using-painless/245211/9 "2020-08-17T14:34:47Z")

</div>

```auto
{
  "test" : {
    "mappings" : {
      "properties" : {
        "duration_secs" : {
          "type" : "long"
        },
        "event" : {
          "properties" : {
            "id" : {
              "type" : "long"
            }
          }
        }
      }
    }
  }
}

```

That is my mapping when I run the above. Which looks like what you are using.

Can you run my test to see if that works for you?

---

<div class="post-metadata">

**Author:** ![david.preston](https://avatars.discourse-cdn.com/v4/letter/d/51bf81/32.png) [@david.preston](https://discuss.elastic.co/u/david.preston)\
**Post date:** [August 17, 2020, 2:38pm UTC](https://discuss.elastic.co/t/create-new-field-on-object-using-painless/245211/10 "2020-08-17T14:38:13Z")

</div>

Yes your test worked fine for me...

Do you think it might be worth me trying to do this using a reindex with a pipeline?

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [August 17, 2020, 2:41pm UTC](https://discuss.elastic.co/t/create-new-field-on-object-using-painless/245211/11 "2020-08-17T14:41:27Z")

</div>

Possible. The mapping looks like it should work so not sure what's going on unless the data set you are working with has some odd balls in there but you said you checked that already.

---

<div class="post-metadata">

**Author:** ![david.preston](https://avatars.discourse-cdn.com/v4/letter/d/51bf81/32.png) [@david.preston](https://discuss.elastic.co/u/david.preston)\
**Post date:** [August 17, 2020, 2:50pm UTC](https://discuss.elastic.co/t/create-new-field-on-object-using-painless/245211/12 "2020-08-17T14:50:44Z")

</div>

Yes i was able to eyeball them with:

```
GET myindex-*/_search
{
  "_source": ["duration_secs"],
"query": {
   "bool": {
      "should": [
        {
          "exists": {
            "field" : "duration_secs"
          }
        },
       {
        "exists": {
          "field": "event"
        }
       }
      ]
   }
  }
}

```

there is a few months of data but there aren't all that many docs, and far fewer have duration\_secs

Thanks for your help - I've got some other work to look at on same dataset, so will try a reindex instead.

thanks again.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 14, 2020, 2:51pm UTC](https://discuss.elastic.co/t/create-new-field-on-object-using-painless/245211/13 "2020-09-14T14:51:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.