Confused about the "properties" and "fields" in my index template

[red888 (red der)]

I used the default logstash template as a starting point and its working but Im confused about 2 things:

{
  "template": "logstash-myindex*",
  "settings": {
    "number_of_shards": 4,
    "number_of_replicas": 1
  },
  "order": 11,
  "mappings": {
    "_default_": {
      "dynamic": false,
      "_all": {
        "enabled": false
      },
      "properties": {
        "@timestamp": {
          "type": "date",
          "include_in_all": false
        },
        "@version": {
          "type": "keyword",
          "include_in_all": false
        },
        "myfield": {
          "type": "text",
          "norms": false,
          "fields": {
            "keyword": {
              "type": "keyword"
            }
          }
        },
        ... etc

I have all of the mappings for my fields nested under "properties" is this necessary? I'm not sure when I would use properties and when I would not.

Also, the "fields" setting under "myfield" is making "myfield" a "keyword" type in addition to a "text" type right? Is writing it this way just less verbose then defining it twice with each type?

[warkolm (Mark Walkom)]

@red888

I have all of the mappings for my fields nested under "properties" is this necessary? I'm not sure when I would use properties and when I would not.

Yes you should, it refers to the properties of the type (the _default_ part).

@red888

Also, the "fields" setting under "myfield" is making "myfield" a "keyword" type in addition to a "text" type right? Is writing it this way just less verbose then defining it twice with each type?

If you want it across multiple types and you don't want to define it separately in each type, yes.

Though be aware we are changing type in 6.0 - Indices, types, and parent / child: current status and upcoming changes in Elasticsearch | Elastic Blog

[red888 (red der)]

Awesome thanks!

Just so I understand how this works, when I look at my events I see a "myfield" and a "myfield.keyword".

If I want to do a full text search on the field I use "myfield" which is analyzed (tokenized).
If there is a specific value I am looking for ( or to filter on ) I can use myfield.keyword.

I realized I have a bunch of my fields defined as "type": "text" AND "type": "keyword", but many of them are just ID fields that I would never need to do a full text search on. I could save space and indexing time by changing them to just keyword right?

If I did change it I would just change it to this right:

"myfield": {
  "type": "keyword",
  "norms": false,
  }

Also, I can still do wildcard searches on keyword fields right?

Sorry for all the questions thanks again.

[warkolm (Mark Walkom)]

@red888

If I want to do a full text search on the field I use "myfield" which is analyzed (tokenized).
If there is a specific value I am looking for ( or to filter on ) I can use myfield.keyword.

Yes, as long as the specific value is a 100% match (case and all).

@red888

I realized I have a bunch of my fields defined as "type": "text" AND "type": "keyword", but many of them are just ID fields that I would never need to do a full text search on. I could save space and indexing time by changing them to just keyword right?

Potentially, yes.

@red888

Also, I can still do wildcard searches on keyword fields right?

Yep - Wildcard query | Elasticsearch Guide [8.11] | Elastic