Template "inheritance"

[RhysEvans]

Hi

I want to setup a default template for some specific fields, that I want added to every other template.

So template 1 has fields x,y,z and template 2 has fields a,b,c

The index should know the "types" (don't know if that is the correct word) for a,b,c,x,y,z

Basically for now I am using the following template

https://raw.githubusercontent.com/wazuh/wazuh/master/extensions/elasticsearch/wazuh-elastic6-template-alerts.json

and would like to add the following

standard.action
standard.dst_ip
etc
etc

I am trying to do this via the following, which doesn't let the index create due to "Rejecting mapping update to [wazuh-logs-3.x-2019.01.31] as the final mapping would have more than 1 type"

put _template/standard-fields
{
  "index_patterns": ["*"],
  "order": -1,
    "mappings": {
        "standard": {
          "properties": {
             "src_ip": {
               "type": "keyword",
               "doc_values": "true"
             },
             "dst_ip": {
               "type": "keyword",
               "doc_values": "true"
             },
             "src_port": {
               "type": "keyword",
               "doc_values": "true"
             },
             "dst_port": {
               "type": "keyword",
               "doc_values": "true"
             },
             "action": {
               "type": "keyword",
               "doc_values": "true"
             }
          }
         }
    }
}

I have then tried the following

{
  "index_patterns": ["*"],
  "order": -1,
  "mappings": {
    "wazuh": {
      "dynamic_templates": [
        {
          "string_as_keyword": {
            "match_mapping_type": "string",
            "mapping": {
              "type": "keyword",
              "doc_values": "true"
            }
          }
        }
      ],
      "properties": {
        "standard": {
          "properties": {
             "src_ip": {
               "type": "keyword",
               "doc_values": "true"
             },
             "dst_ip": {
               "type": "keyword",
               "doc_values": "true"
             },
             "src_port": {
               "type": "keyword",
               "doc_values": "true"
             },
             "dst_port": {
               "type": "keyword",
               "doc_values": "true"
             },
             "action": {
               "type": "keyword",
               "doc_values": "true"
             }
          }
          }
        }
       }
}
}

This allows the index to create, however

1. The "new" fields aren't "mapped" to a "type" (again not sure if my terminology is correct), I presume as it is overwritten by the hyperlinked (above) template.
2. I presume I will have issues with other index creations (of other names) due to the issue "more than 1 type issue"

So any advice on how I would best "apply" those fields (and relevant type) across all indexes ?

Any help is appreciated

PS I am running 6.5.4

Thanks

[RhysEvans]

Hi

Ok, so I have now tried to put the fields int one big template. but I am still having issues

Adding the following to the hyperlinked template above

        "standard": {
          "properties": {
             "src_ip": {
               "type": "keyword",
               "doc_values": "true"
             },
             "dst_ip": {
               "type": "keyword",
               "doc_values": "true"
             },
             "src_port": {
               "type": "keyword",
               "doc_values": "true"
             },
             "dst_port": {
               "type": "keyword",
               "doc_values": "true"
             },
             "action": {
               "type": "keyword",
               "doc_values": "true"
             },
            "src_Geo": {
              "properties": {
                "area_code": {
                  "type": "long"
                },
                "city_name": {
                  "type": "keyword",
                  "doc_values": "true"
                },
                "continent_code": {
                  "type": "text"
                },
                "coordinates": {
                  "type": "double"
                },
                "country_code2": {
                  "type": "text"
                },
                "country_code3": {
                  "type": "text"
                },
                "country_name": {
                  "type": "keyword",
                  "doc_values": "true"
                },
                "dma_code": {
                  "type": "long"
                },
                "ip": {
                  "type": "keyword",
                  "doc_values": "true"
                },
                "latitude": {
                  "type": "double"
                },
                "location": {
                  "type": "geo_point"
                },
                "longitude": {
                  "type": "double"
                },
                "postal_code": {
                  "type": "keyword"
                },
                "real_region_name": {
                  "type": "keyword",
                  "doc_values": "true"
                },
                "region_name": {
                  "type": "keyword",
                  "doc_values": "true"
                },
                "timezone": {
                  "type": "text"
                },
                "as_org": {
                  "type": "text"
                },
                "asn": {
                  "type": "long"
                }
              }
            }
          }
        }

Just after the

             "action": {
               "type": "keyword",
               "doc_values": "true"
             },

section.

if i do a "GET wazuh-logs-3.x-2019.02.01?pretty" I get (some code removed)

          "standard" : {
            "properties" : {
              "action" : {
                "type" : "keyword"
              },
              "dst_Geo" : {
                "properties" : {
                  "as_org" : {
                    "type" : "keyword"
                  },
                  "asn" : {
                    "type" : "long"
                  },
                  "city_name" : {
                    "type" : "keyword"
                  },
                  "country_name" : {
                    "type" : "keyword"
                  },
                  "location" : {
                    "properties" : {
                      "lat" : {
                        "type" : "float"
                      },
                      "lon" : {
                        "type" : "float"
                      }
                    }
                  },
                  "region_name" : {
                    "type" : "keyword"
                  }
                }
              },
              "dst_ip" : {
                "type" : "keyword"
              },
              "dst_port" : {
                "type" : "keyword"
              },
              "src_Geo" : {
                "properties" : {
                  "area_code" : {
                    "type" : "long"
                  },
                  "as_org" : {
                    "type" : "text"
                  },
                  "asn" : {
                    "type" : "long"
                  },
                  "city_name" : {
                    "type" : "keyword"
                  },
                  "continent_code" : {
                    "type" : "text"
                  },
                  "coordinates" : {
                    "type" : "double"
                  },
                  "country_code2" : {
                    "type" : "text"
                  },
                  "country_code3" : {
                    "type" : "text"
                  },
                  "country_name" : {
                    "type" : "keyword"
                  },
                  "dma_code" : {
                    "type" : "long"
                  },
                  "ip" : {
                    "type" : "keyword"
                  },
                  "latitude" : {
                    "type" : "double"
                  },
                  "location" : {
                    "type" : "geo_point"
                  },
                  "longitude" : {
                    "type" : "double"
                  },
                  "postal_code" : {
                    "type" : "keyword"
                  },
                  "real_region_name" : {
                    "type" : "keyword"
                  },
                  "region_name" : {
                    "type" : "keyword"
                  },
                  "timezone" : {
                    "type" : "text"
                  }
                }
              },
              "src_ip" : {
                "type" : "keyword"
              },
              "src_port" : {
                "type" : "keyword"
              },
              "threat" : {
                "properties" : {
                  "src_ip" : {
                    "type" : "keyword"
                  }
                }
              }
            }
          },

Note the "new" "location" looks like the following in Kibana

|# standard.dst_Geo.location.lat |51.5|
|# standard.dst_Geo.location.lon |-3.2|

where I would expect it to look like

GeoLocation_dst.location 	
	{
  "lat": 51.4964,
  "lon": -0.1224
}

It also doesn't seem anything under "standard" is being mapped properly

I can get the fields to work up a level, ie outside of the "standard" block.

Any help is appreciated

Thanks