# Adding added field to index

**URL:** <https://discuss.elastic.co/t/adding-added-field-to-index/335440>\
**Category:** Elasticsearch\
**Tags:** docker\
**Created:** [June 7, 2023, 1:05pm UTC](https://discuss.elastic.co/t/adding-added-field-to-index/335440 "2023-06-07T13:05:54Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![rexxdad](https://avatars.discourse-cdn.com/v4/letter/r/91b2a8/32.png) [@rexxdad](https://discuss.elastic.co/u/rexxdad)\
**Post date:** [June 7, 2023, 1:05pm UTC](https://discuss.elastic.co/t/adding-added-field-to-index/335440/1 "2023-06-07T13:05:54Z")

</div>

i use the docker elk stack on macos with the logstash http pipeline to accept our apps http posts as input

it created an index (I didn't before)

reviewing the content, there are two fields that combined could make a geopoint, so I added that field thru the web UI. and I can use the visualizer to see the map of the points..

now I want my external app to use the qeopoint filter query on that field

```auto
 result = await client.search({
    index: 'logs-*', 
    "query": {
      "bool": {
        "must": {
          "match_all": {}
        },
          "filter" : {
              "geo_bounding_box" : {
                  "navigate_location" : { // <- the manually created field
                      "top_left" : {
                          "lat" : 34.73,
                          "lon" : -98.00
                      },
                      "bottom_right" : {
                          "lat" : 30.717,
                          "lon" : -97.030
                      }
                  }
              }
          }
      }
  }

```

when I query on other fields, I get the data record

```auto
2023 @ 11: 18:28.383event.original{
  "devices": [],
  "end": 1685981908383,
  "retrycount": 0,
  "type": 1,
  "status": "canceled",
  "appVersion": "1.0",
  "location": {
    "coords": {
      "altitude": 235.09720251895487,
      "longitude": -97.628558119899495, // part of constructed point
      "altitudeAccuracy": 1.9524757439985525,
      "speed": 0.021378767720873718,
      "latitude": 30.463607267669115, / part of constructed point
      "heading": -1,
      "accuracy": 7.068291962625123
    },
    "timestamp": 1685981907000.032
  },
  "logtype": "navigate",
  "phoneinfo": {
    "manufacturer": "Apple",
    "platform": "iOS",
    "version": "16.5",
    "model": "iPhone13,2"
  },
  "start": 1685981884953
}

```

but not the constructed field ..  
when I try to add it to the index thru the developers console I get an error, already exists  
when I dump out the mappings I see

```auto
        "navigate_location": {
          "type": "geo_point"
        },

```

but I cant get it by name

```auto
get logs-generic-default/_mapping/navigate_location

```

yields

```auto
{
  "error": "no handler found for uri [/logs-generic-default/_mapping/navigate_location?pretty=true] and method [GET]"
}

```

altho it just showed be the datatype mapping in that index...

how do I resolve this..

---

<div class="post-metadata">

**Author:** ![rexxdad](https://avatars.discourse-cdn.com/v4/letter/r/91b2a8/32.png) [@rexxdad](https://discuss.elastic.co/u/rexxdad)\
**Post date:** [June 7, 2023, 1:11pm UTC](https://discuss.elastic.co/t/adding-added-field-to-index/335440/2 "2023-06-07T13:11:10Z")

</div>

if I do this with a more specific index name

```auto
result = await client.search({
      index: 'logs-generic-default', 
    "query": {
      "bool": {
        "must": {
          // "match_all": { }
          "term": { "logtype": "navigate" }
        },
          "filter" : {
              "geo_bounding_box" : {
                  "navigate_location" : {
                      "top_left" : {
                      "lat": 30.6627347,
                      "lon": -97.922001
                      },
                      "bottom_right" : {
                          "lat" : 30.415,
                        "lon": -97.0185702
                      }
                  }
              }
          }
      }

```

i get a successful search, but no hits

```auto
{
  took: 7,
  timed_out: false,
  _shards: { total: 1, successful: 1, skipped: 0, failed: 0 },
  hits: { total: { value: 0, relation: 'eq' }, max_score: null, hits: [] }

```

at least its not an error on the field...

BUT the data says there should be 100's of hits.. with this quadrant

---

<div class="post-metadata">

**Author:** ![rexxdad](https://avatars.discourse-cdn.com/v4/letter/r/91b2a8/32.png) [@rexxdad](https://discuss.elastic.co/u/rexxdad)\
**Post date:** [June 7, 2023, 3:01pm UTC](https://discuss.elastic.co/t/adding-added-field-to-index/335440/3 "2023-06-07T15:01:29Z")

</div>

a little more data

the field has a script (because the actual data isn't an object with lat/lon)

```auto
if(doc['location.coords.longitude'].size()!= 0){
    emit ( doc['location.coords.latitude'].value ,doc['location.coords.longitude'].value)
}

```

on the previous field, i forced the datatype via a put to the mappings, and destroyed the script.. and now no data..

BUT no error..

I just created another field with the script (via the UI).. and have data, but the new field is NOT in the mappings, so the search fails

```auto
"failed to find geo field [fribble]"

```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 8, 2023, 10:58pm UTC](https://discuss.elastic.co/t/adding-added-field-to-index/335440/4 "2023-06-08T22:58:49Z")

</div>

> [@rexxdad](#):
>
> ```auto
> "error": "no handler found for uri [/logs-generic-default/_mapping/navigate_location?pretty=true] and method [GET]"
> 
> ```

You can't get a specific field for mappings, only the entire thing.

It'd help if you shared the mapping and a sample document you are using here, as you're referencing things we can't guess at.

---

<div class="post-metadata">

**Author:** ![rexxdad](https://avatars.discourse-cdn.com/v4/letter/r/91b2a8/32.png) [@rexxdad](https://discuss.elastic.co/u/rexxdad)\
**Post date:** [June 9, 2023, 1:45pm UTC](https://discuss.elastic.co/t/adding-added-field-to-index/335440/5 "2023-06-09T13:45:04Z")

</div>

thanks.. using the UI, discover tab, here is a document record in the selected timeframe window, with the original content field as well..

```auto
{
  "@timestamp": [
    "2023-06-05T16:18:28.765Z"
  ],
  "@version": [
    "1"
  ],
  "appVersion": [
    "1.0"
  ],
  "data_stream.dataset": [
    "generic"
  ],
  "data_stream.namespace": [
    "default"
  ],
  "data_stream.type": [
    "logs"
  ],
  "database_post_delay": [
    2850303289
  ],
  "devices.address": [
    "97CAB758-A70B-9867-D2D5-4C678EF5BD1D"
  ],
  "devices.sequence": [
    5
  ],
  "devices.subtype": [
    "10"
  ],
  "devices.td": [
    2367
  ],
  "devices.type": [
    1
  ],
  "end": [
    1683131605476
  ],
  "end_date": [
    "2023-05-03T16:33:25.476Z"
  ],
  "event.original": [
    "{\"start\":1683131605305,\"logtype\":\"scanresult\",\"end\":1683131605476,\"devices\":[{\"address\":\"97CAB758-A70B-9867-D2D5-4C678EF5BD1D\",\"sequence\":5,\"subtype\":\"10\",\"type\":1,\"td\":2367}],\"phoneinfo\":{\"manufacturer\":\"Apple\",\"version\":\"16.4.1\",\"model\":\"iPhone13,2\",\"platform\":\"iOS\"},\"location\":{\"coords\":{\"altitude\":234.23440742492676,\"heading\":-1,\"latitude\":30.463546480177587,\"longitude\":-97.628650949809725,\"altitudeAccuracy\":10.748244285583496,\"speed\":-1,\"accuracy\":35},\"timestamp\":1683131599474.551},\"type\":1,\"status\":\"success\",\"appVersion\":\"1.0\",\"retrycount\":0}"
  ],
  "fribble": [
    {
      "coordinates": [
        -97.62865447998047,
        30.463546752929688
      ],
      "type": "Point"
    }
  ],
  "host.ip": [
    "172.22.0.1"
  ],
  "http.method": [
    "POST"
  ],
  "http.request.body.bytes": [
    "640"
  ],
  "http.request.mime_type": [
    "application/json"
  ],
  "http.version": [
    "HTTP/1.1"
  ],
  "location.coords.accuracy": [
    35
  ],
  "location.coords.altitude": [
    234.2344
  ],
  "location.coords.altitudeAccuracy": [
    10.748244
  ],
  "location.coords.heading": [
    -1
  ],
  "location.coords.latitude": [
    30.463547
  ],
  "location.coords.longitude": [
    -97.628654
  ],
  "location.coords.speed": [
    -1
  ],
  "location.timestamp": [
    1683131600000
  ],
  "logtype": [
    "scanresult"
  ],
  "navigate_elapsed": [
    171
  ],
  "phoneinfo.manufacturer": [
    "Apple"
  ],
  "phoneinfo.model": [
    "iPhone13,2"
  ],
  "phoneinfo.platform": [
    "iOS"
  ],
  "phoneinfo.version": [
    "16.4.1"
  ],
  "retrycount": [
    0
  ],
  "start": [
    1683131605305
  ],
  "start_date": [
    "2023-05-03T16:33:25.305Z"
  ],
  "status": [
    "success"
  ],
  "type": [
    1
  ],
  "url.path": [
    "/"
  ],
  "url.port": [
    3000
  ],
  "user_agent.original": [
    "App/1 CFNetwork/1408.0.4 Darwin/22.5.0"
  ],
  "_id": "__BajIgBAgeSes04TwTH",
  "_index": ".ds-logs-generic-default-2023.04.30-000001",
  "_score": null
}

```

note the variable fribble..

i created the variable in the discover tab using this

 ![Screenshot 2023-06-09 at 8.22.17 AM](https://us1.discourse-cdn.com/elastic/original/3X/1/7/17a400a9987695b8c86b86e3e8088a72e371b8b0.png)

and using discover I can see the data  
---- sorry, new user cannot insert more than one media item .... see next post

so, now I want to use that field in the geo\_bounding\_box filter from an external app

```auto
"filter" : {
              "geo_bounding_box" : {
                  "fribble" : { // <- the manually created field
                      "top_left" : {

```

the document record indicates it is in index .ds.logs-generic-default-  
when I dump the mappings for the old navigate\_location in that index I see

```auto
        "navigate_location": {
          "type": "geo_point"
        },

```

but I don't know if I'm supposed to see the script in this output

(sorry, trying lots of different things before coming here )  
(force insert mapping, .... however fribble is NOT shown in the mappings output of this index

the execution of the query returns (for either field)

```auto
error={"name":"ResponseError","meta":{"body":{"error":{"root_cause":[{"type":"x_content_parse_exception","reason":"[1:43] [bool] unknown field [navigate_location]"}],"type":"x_content_parse_exception","reason":"[1:43] [bool] unknown field [navigate_location]"},"status":400},"statusCode":400,"headers":{"x-elastic-product":"Elasticsearch"
...

```

---

<div class="post-metadata">

**Author:** ![rexxdad](https://avatars.discourse-cdn.com/v4/letter/r/91b2a8/32.png) [@rexxdad](https://discuss.elastic.co/u/rexxdad)\
**Post date:** [June 9, 2023, 1:46pm UTC](https://discuss.elastic.co/t/adding-added-field-to-index/335440/6 "2023-06-09T13:46:15Z")

</div>

here is the other media item using the discover to view the variable data, so the script is successful  
 ![Screenshot 2023-06-09 at 8.24.21 AM](https://us1.discourse-cdn.com/elastic/original/3X/e/5/e57af414038a9a2ceacb55dda3966b0e0dc61850.png)

---

<div class="post-metadata">

**Author:** ![rexxdad](https://avatars.discourse-cdn.com/v4/letter/r/91b2a8/32.png) [@rexxdad](https://discuss.elastic.co/u/rexxdad)\
**Post date:** [June 9, 2023, 1:51pm UTC](https://discuss.elastic.co/t/adding-added-field-to-index/335440/7 "2023-06-09T13:51:05Z")

</div>

here is the node js test app

```auto
'use strict'

const { Client } = require('@elastic/elasticsearch')
const client = new Client({
  node: "http://localhost:9200",
  auth: {
    username: use as appropriate
    password: 
  },
})

async function run () {

  // Let's search!
let result
  try{
    result = await client.search({
    index: 'logs-generic-default', 
    "query": {
      "bool": {
        "must": {
           "match_all": { }
        },
        "filter": {
          "geo_bounding_box": {
            "navigate_location": { // or "fribble" fribble gets variable not found, navigate returns no hits
              "top": 30.6627347,
              "bottom": 30.415,
              "left": -97.922001,
              "right": -97.0185702
            }
          }
        }  
      }
  }

  })
 } catch(error){
    console.log("error="+JSON.stringify(error));
 }
 finally {
  console.log(result)
    const data = result.rows.map(row => {
    const obj = {}
    for (let i = 0; i < row.length; i++) {
       obj[result.columns[i].name] = row[i]
    }
    return obj
  })
 }
}

run().catch(console.log)

```

---

<div class="post-metadata">

**Author:** ![rexxdad](https://avatars.discourse-cdn.com/v4/letter/r/91b2a8/32.png) [@rexxdad](https://discuss.elastic.co/u/rexxdad)\
**Post date:** [June 10, 2023, 7:41pm UTC](https://discuss.elastic.co/t/adding-added-field-to-index/335440/8 "2023-06-10T19:41:26Z")

</div>

after looking at some more examples while trying to solve another problem, I came to this, which does exactly what I want, and I don't have to fiddle with the index..

```auto
'use strict'

const { Client } = require('@elastic/elasticsearch')
const client = new Client({
  node: "http://localhost:9200",
    auth: {
        username: as appropriate 
        password: 
    }
})

async function run () {

  // Let's search!
let result
  try{
    result = await client.search({
        index: 'logs-generic-default', 
        "runtime_mappings": {
          "navigate_location": {
            "type": "geo_point",
            "script": {
              "source":
                "if(doc['location.coords.longitude'].size()!= 0){emit(doc['location.coords.latitude'].value, doc['location.coords.longitude'].value)}"
            }
          }
        },
        "query": {
          "bool": {
            "must": {
              "term": { "logtype": "navigate" }
            },
            "filter": {
              "geo_bounding_box": {
                "navigate_location": {  
                  "top": 30.6627347,
                  "bottom": 30.415,
                  "left": -97.922001,
                  "right": -97.0185702
                }
              }
            }
          }
        },
        "fields": ["navigate_location"]
      }
  )
 } catch(error){
    console.log("error="+JSON.stringify(error));
 }
 finally {
      console.log(result)
      let i = 1;
      const data = result.hits.hits.forEach(row => {
        const obj = {}
        console.log("result document "+i++)
        Object.keys(row).forEach(field => {
        
            console.log("field=" + field + " value=" + JSON.stringify(row[field]))
        })
        console.log("\n")
    })
}

run().catch(console.log)

```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 14, 2023, 12:42am UTC](https://discuss.elastic.co/t/adding-added-field-to-index/335440/9 "2023-06-14T00:42:09Z")

</div>

Oh, is the `fribble` a runtime field?

---

<div class="post-metadata">

**Author:** ![rexxdad](https://avatars.discourse-cdn.com/v4/letter/r/91b2a8/32.png) [@rexxdad](https://discuss.elastic.co/u/rexxdad)\
**Post date:** [June 14, 2023, 1:12am UTC](https://discuss.elastic.co/t/adding-added-field-to-index/335440/10 "2023-06-14T01:12:33Z")

</div>

yes, just trying to make another field...certain to not be in the index

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 14, 2023, 1:34am UTC](https://discuss.elastic.co/t/adding-added-field-to-index/335440/11 "2023-06-14T01:34:19Z")

</div>

Runtime fields defined in Kibana like this do not show in the mappings as it's applied at the data view level. After you define the runtime field you should be able to see a little info icon and text popup when you open the data view in Stack Management, like this one;  
 ![SCR-20230614-khku](https://us1.discourse-cdn.com/elastic/original/3X/b/2/b223459caf18bb213ccfbd8286f14429d90c2179.png)

You can also check this with `GET kbn:/api/data_views/data_view/<view-id>`.

If you want this to be available for _all_ requests to the index in Elasticsearch then you need to update the mapping for the index, which can be done in Dev Tools.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 14, 2023, 1:55am UTC](https://discuss.elastic.co/t/adding-added-field-to-index/335440/12 "2023-06-14T01:55:06Z")

</div>

I've made a PR to make this clearer in the docs [Update manage-data-views.asciidoc by markwalkom · Pull Request #159636 · elastic/kibana · GitHub](https://github.com/elastic/kibana/pull/159636)

---

<div class="post-metadata">

**Author:** ![rexxdad](https://avatars.discourse-cdn.com/v4/letter/r/91b2a8/32.png) [@rexxdad](https://discuss.elastic.co/u/rexxdad)\
**Post date:** [June 14, 2023, 2:33am UTC](https://discuss.elastic.co/t/adding-added-field-to-index/335440/13 "2023-06-14T02:33:17Z")

</div>

thank you for that

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 12, 2023, 2:34am UTC](https://discuss.elastic.co/t/adding-added-field-to-index/335440/14 "2023-07-12T02:34:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.