Index issues

Elastic Stack Beats
1

Hello

I have setup a template
PUT /_template/active-directory

I have setup an index
PUT active-directory

When i confirm my mappings, they are as expected. They just include the mappings i have specifically set.
GET active-directory/_mapping

When i ingest data with Filebeat, the template seems to be overwritten and all the default Filebeat ECS fields are added to my mappings.

Here is filebeat.yml

setup.ilm.enabled: auto
setup.ilm.policy_name: "active-directory"
setup.ilm.rollover_alias: "active-directory"
setup.ilm.pattern: "{now/d}-000001"
setup.ilm.overwrite: false
setup.template.name: "active-directory"
setup.template.pattern: active-directory-*
setup.template.overwrite: false

here is the template

PUT /_template/active-directory
{
	"order" : 1,
    "index_patterns" : [
      "active-directory"
    ],
  "settings": {
		"index": {
			"lifecycle": {
				"name": "active-directory",
				"rollover_alias": "active-directory"
			}
    },
    "mapping" : {
          "total_fields" : {
            "limit" : "10000"
          }
        }
	},
	"mappings": {
		"properties": {
			"@timestamp": {
				"type": "date"
			},
			"ad": {
				"properties": {
					"distinguishedname": {
						"ignore_above": 1024,
						"type": "keyword"
					},
					"enabled": {
						"type": "boolean"
					},
					"objectclass": {
						"ignore_above": 1024,
						"type": "keyword"
					},
					"objectguid": {
						"ignore_above": 1024,
						"type": "keyword"
					},
					"samaccountname": {
						"ignore_above": 1024,
						"type": "keyword"
					},
					"samaccounttype": {
						"type": "long"
					},
					"primarygroupid": {
						"type": "long"
					},
					"primarygroup": {
						"ignore_above": 1024,
						"type": "keyword"
					},
					"protectedfromaccidentaldeletion": {
						"type": "boolean"
					},
					"passwordexpired": {
						"type": "boolean"
					},
					"passwordneverexpires": {
						"type": "boolean"
					},
					"passwordnotrequired": {
						"type": "boolean"
					},
					"sid": {
						"ignore_above": 1024,
						"type": "keyword"
					},
					"name": {
						"ignore_above": 1024,
						"type": "keyword"
					},
					"userprincipalname": {
						"ignore_above": 1024,
						"type": "keyword"
					},
					"memberof": {
						"ignore_above": 1024,
						"type": "keyword"
					},
					"iscriticalsystemobject": {
						"type": "boolean"
					},
					"ipv4address": {
						"type": "ip"
					},
					"instancetype": {
						"type": "long"
					},
					"canonicalname": {
						"ignore_above": 1024,
						"type": "keyword"
					},
					"admincount": {
						"type": "long"
					},
					"allowreversiblepasswordencryption": {
						"type": "boolean"
					},
					"operatingsystem": {
						"ignore_above": 1024,
						"type": "keyword"
					},
					"operatingsystemhotfix": {
						"ignore_above": 1024,
						"type": "keyword"
					},
					"operatingsystemservicepack": {
						"ignore_above": 1024,
						"type": "keyword"
					},
					"operatingsystemversion": {
						"ignore_above": 1024,
						"type": "keyword"
					},
					"whencreated": {
					  "format": "yyyy-MM-dd HH:mm:ss",
						"type": "date"
					},
					"whenchanged": {
					  "format": "yyyy-MM-dd HH:mm:ss",
						"type": "date"
					},
					"pwdlastset": {
					  "format": "yyyy-MM-dd HH:mm:ss",
						"type": "date"
					}
				}
			}
		}
	}
}
2

So your goal is to have only the fields you include in your custom template, and not include other filebeat fields in the index template?

I'm not completely clear on the order things are happening, but one thing I notice is that your template has index_patterns set to "active-directory" when you probably meant "active-directory-*"

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.