Question about adding geo_point and updating mappings in general

I've been trying for a few days now to add custom mapping (geo_point) to my elasticsearch server. This server has daily created indices "filebeat-2019.08.22". However, all documentation tells me I need to update the template, but I cannot find what the name of this template is.

I have tried to create the mapping through creating a template named "filebeat", and one named "filebeat-*" and one named "filebeat*" but none seem to work or show the information in the next created index.

One thing I tried yesterday is this:

curl -X PUT "localhost:9200/_template/filebeat?pretty" -H 'Content-Type: application/json' -d'
{
   "index_patterns" : ["filebeat-*"],
   "order" : 1,
   "settings" : {
       "index.refresh_interval" : "5s",
       "number_of_replicas" : "0"
   },
  "properties": {
    "geoip": {
      "dynamic": true,
      "properties": {
        "location": {
          "type": "geo_point"
        }
      }
    }
  }
}

This did not produce anything with regard to the geo_point, but it seems it did do something with the replicas (which I also needed to set to 0), so it worked partially.

Requesting the template shows this:

{"filebeat":{"order":1,"index_patterns":["filebeat-*"],"settings":{"index":{"number_of_replicas":"0","refresh_interval":"5s"}},"mappings":{},"aliases":{}}}

So it seems it did not take the properties.

Basically the question here is. How do I get a property to change in the next created index? Also if you dont know the index name (indices are created like filebeat-xxxx.xx.xx but from which template?), can you (like I did or tried to do) simply create a new template with reference to the index pattern and it will take?

it seems to me, that the properties field needs to be put within a mapping field, similar to what you do when creating an index.

Please take a look at the response returned from the above call, I am pretty sure you get an error back and not a message that this template has been stored successfully (at least under Elasticsearch 7.x).

There was no error (Elasticsearch 6.0.0.0), and the first part seems to have been executed. Maybe I do need to encapsulate the properties in a mappings field. The process on how to do this is not clear.

Basically I have a system where a new index is created every day (filebeat-xxxx.xx.xx), but when I scroll through the loaded templates, there is no (or seems to be no) template on which this is created, so there is no template I can edit.

What is the procedure for this? Do I create a new template (like I did above) with only the fields I need changed (geoip location/geo_point) and keep the rest as it was, or do I need to export all the mappings from the index, put that (including the change I need) in a new json block and import that as a new template?

Also, when I export/dump the current index mappings, the geo/location fields fall below a group/_type with a specific name ("indexname"->mappings->"_type name"->properties->geoip->properties). Do I need to add that group/type in my new template as well, or does elasticsearch map the new geoip settings to everything "below" the level i change it on?

if several templates match, they will be merged together, so if there is the filebeat template as well, both will be used.

Ok, i've just loaded the following template, where "xxxxxxxxxxxx" is the _type:

curl -X PUT "localhost:9200/_template/filebeat?pretty" -H 'Content-Type: application/json' -d'
{
   "index_patterns" : ["filebeat-*"],
   "order" : 1,
   "settings" : {
       "index.refresh_interval" : "5s",
       "number_of_replicas" : "0"
   },
   "mappings" : {
     "xxxxxxxxxxxx" : {
       "properties": {
         "geoip": {
           "dynamic": true,
           "properties": {
             "location": {
               "type": "geo_point"
             }
           }
         }
       }
    }
  }
}
'

No errors, and a "curl 'localhost:9200/_template/filebeat' " shows the geoip section as well now. Unfortunately I have to wait for the new index to see if it actually worked.

Update 23/08: The new index uses the geo_point type.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.