# A way to ensure additional field will be with correct type inside index

**URL:** <https://discuss.elastic.co/t/a-way-to-ensure-additional-field-will-be-with-correct-type-inside-index/310770>\
**Category:** Elasticsearch\
**Tags:** painless, ingest-pipeline\
**Created:** [July 27, 2022, 4:07pm UTC](https://discuss.elastic.co/t/a-way-to-ensure-additional-field-will-be-with-correct-type-inside-index/310770 "2022-07-27T16:07:09Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [July 27, 2022, 4:07pm UTC](https://discuss.elastic.co/t/a-way-to-ensure-additional-field-will-be-with-correct-type-inside-index/310770/1 "2022-07-27T16:07:09Z")

</div>

Hello,

I was using the scripted field to get a date field from system.uptime.duration.ms field value. I used below painless script.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/2/42fbc4edbd17f3f70b0e6d904bf5bbd1351882c6.png)

```auto
String val = "system.uptime.duration.ms";

if (doc[val] != null){ 
    if (doc[val].size() != 0 ){
    
        long now = doc["@timestamp"].value.toInstant().toEpochMilli();
    
        long elapsedTime = now - doc[val].value;
    
        return elapsedTime;
    }
}

```

I wanted to migrate from this solution but not be forced to remember to add some additional fields whenever I do beats update.

I tried two painless scripts for the default pipeline.

1.) This one sets value from the object of ZonedDateTime but elasticsearch dynamically recognizes it as a keyword which messes up my dashboard.

```auto
if (ctx?.system?.uptime?.duration?.ms != null) {
        ZonedDateTime zdt = ZonedDateTime.parse(ctx['@timestamp']);
        long now = zdt.toInstant().toEpochMilli();
        long elapsedTime = now - ctx.system.uptime.duration.ms;
        Instant instant = Instant.ofEpochMilli(elapsedTime);
        ZonedDateTime zdt = ZonedDateTime.ofInstant(instant, ZoneId.of('Z'));
        ctx['system.uptime.since'] = zdt;
}

```

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/1/d168730ce4383d7d923b760de2da787d06faab69.png)

2.) This one sets it up as long but I cannot set up the date format inside the index pattern. So just big number is displayed inside of the date.

```auto
if (ctx?.system?.uptime?.duration?.ms != null) {
        ZonedDateTime zdt = ZonedDateTime.parse(ctx['@timestamp']);
        long now = zdt.toInstant().toEpochMilli();
        long elapsedTime = now - ctx.system.uptime.duration.ms;
        ctx['system.uptime.since'] = elapsedTime ;
}

```

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/c/6c056b9bd6ce4bae7c6585f8672db5702de51db0.png)

Is there a way besides adding manually mapping to the index template that says that this field should be recognized as date not long or keyword?

ELK/Kibana 7.17-5

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 27, 2022, 11:15pm UTC](https://discuss.elastic.co/t/a-way-to-ensure-additional-field-will-be-with-correct-type-inside-index/310770/2 "2022-07-27T23:15:11Z")

</div>

> [@Adriann](#):
>
> Is there a way besides adding manually mapping to the index template that says that this field should be recognized as date not long or keyword?

Honestly, your best bet is to do this in a template so that it's definitively matched.

Alternatively did you look at runtime fields?

---

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [July 28, 2022, 7:56am UTC](https://discuss.elastic.co/t/a-way-to-ensure-additional-field-will-be-with-correct-type-inside-index/310770/3 "2022-07-28T07:56:39Z")

</div>

I have just added this field as a runtime field to the index template of metricbeat indices.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/e/9e8003406bb8ab5eeb20833693ffb075d53234f0.png)

I don't know how to force the index pattern to recognize this field. I guess I need to wait for a new index to generate from the updated index template or is there another way?

I think that this will work when the index pattern finally refreshes but please tell me what would be the best approach to make this field (automatically if possible) be added to every new metricbeat index template when I run `metricbeat -e setup template` command after updating beat to a new version?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 28, 2022, 10:19am UTC](https://discuss.elastic.co/t/a-way-to-ensure-additional-field-will-be-with-correct-type-inside-index/310770/4 "2022-07-28T10:19:16Z")

</div>

> [@Adriann](#):
>
> I don't know how to force the index pattern to recognize this field. I guess I need to wait for a new index to generate from the updated index template or is there another way?

Yes. But you can also add it directly to the mapping of the current index and it will start creating the field.

---

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [July 28, 2022, 11:11am UTC](https://discuss.elastic.co/t/a-way-to-ensure-additional-field-will-be-with-correct-type-inside-index/310770/5 "2022-07-28T11:11:20Z")

</div>

I just changed the index name for one of the beats. It works.

![image](https://us1.discourse-cdn.com/elastic/original/3X/b/e/be825225969a531a29bb7fcb0ca479d7e3cc410f.png)

1.) What is the best way(as automatic as possible) to add this field to every new template that will be uploaded after a beat update and command metricbeat -e setup template.

2.) What is the best way to introduce this new field to the currently working weekly index? Some kind of reindex? The approach of reindexing the current index(i.e metricbeat-7.17.1-system-2022.30) to some other name(i.e metricbeat-7.17.1-system-2022.30\_reindex) and just deleting(metricbeat-7.17.1-system-2022.30) it so the new would be generated would be appropriate? How to prevent any data loss?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 25, 2022, 11:11am UTC](https://discuss.elastic.co/t/a-way-to-ensure-additional-field-will-be-with-correct-type-inside-index/310770/6 "2022-08-25T11:11:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.