Changing the analyzer type fields are no longer aggregatable

Elastic Stack Elasticsearch
1

Hi All,
I changed the index mapping for my index, and now I can't perform visualisations on the fields!

Example index template:

PUT /_template/my_logstash
{
"order": 1000,
"index_patterns": [
  "logstash-*",
  "docker-*",
  "syslog-*",
  "ironport-*",
  "radius-*",
  "firewall-*"
],
"settings": {
  "analysis": {
    "analyzer": {
	  "keyword_lowercase": {
	    "tokenizer": "keyword",
	      "filter": ["lowercase"]
	},
	  "whitespace_lowercase": {
	    "tokenizer": "whitespace",
	      "filter": ["lowercase"]
	  }
    }
  }
},
"mappings": {
  "doc": {
    "dynamic": "true",
    "properties": {
      "bytes": {
        "type": "integer"
      },
      "bytes_in": {
        "type": "integer"
      },
      "dest_ip": {
        "type": "ip"
      },
      "src_ip": {
        "type": "ip"
      },
      "host": {
        "type": "text",
		"analyzer": "whitespace_lowercase"
      },
      "logsource": {
        "type": "text",
		"analyzer": "whitespace_lowercase"
      },
      "program": {
        "type": "text",
		"analyzer": "whitespace_lowercase"
      },
<TRUNCATED>

But on newly created indexes (and recent data into existing indexes) I can no longer use the host 'field' to split data etc.
Looking at a newly created index (with data), the 'host' field exists in the documents, and in Kibana, but in Kibana's index-patterns 'host' is searchable ONLY.
There is no 'host.keyword' like their used to be....

What can I do to fix this?

2

No, because you have explicitly set the mapping of the host field so it takes precedence over anything more generic.

You can change what you have there and add a .keyword field using fields | Elasticsearch Guide [6.4] | Elastic

3

Thanks Warkolm,
I think what this forum lacks is people actually saying how they do things, so that when beginners google questions, they can actually find the answers they need.

So what I ended up doing was going to kibana > Dev Tools > Console
Then inputting:

PUT /_template/my_logstash
{
"order": 1000,
"index_patterns": [
  "logstash-*",
  "docker-*",
  "syslog-*",
  "ironport-*",
  "radius-*",
  "test-*",
 "firewall-*"
],
"settings": {
  "analysis": {
    "analyzer": {
      "keyword_lowercase": {
        "tokenizer": "keyword",
          "filter": ["lowercase"]
    },
      "whitespace_lowercase": {
        "tokenizer": "whitespace",
          "filter": ["lowercase"]
      }
    }
  }
},
"mappings": {
  "doc": {
    "dynamic": "true",
    "properties": {
      "bytes": {
        "type": "integer"
      },
      "bytes_in": {
        "type": "integer"
      },
      "bytes_out": {
        "type": "integer"
      },
      "dest_port": {
        "type": "integer"
      },
      "src_port": {
        "type": "integer"
      },
      "dest_translated_ip": {
        "type": "ip"
      },
      "src_translated_ip": {
        "type": "ip"
      },
      "dest_ip": {
        "type": "ip"
      },
      "src_ip": {
        "type": "ip"
      },
      "host": {
        "type": "text",
        "analyzer": "whitespace_lowercase",
        "fields": {
          "keyword": {
            "type": "keyword"
          }
        }
      },
      "logsource": {
        "type": "text",
        "analyzer": "whitespace_lowercase",
        "fields": {
          "keyword": {
            "type": "keyword"
          }
        }
      },
      "program": {
        "type": "text",
        "analyzer": "whitespace_lowercase",
        "fields": {
          "keyword": {
            "type": "keyword"
          }
        }
      },
      "source": {
        "type": "text",
        "analyzer": "keyword_lowercase",
        "fields": {
          "keyword": {
            "type": "keyword"
          }
        }
      },
      "sourcetype": {
        "type": "text",
        "analyzer": "whitespace_lowercase",
        "fields": {
          "keyword": {
            "type": "keyword"
          }
        }
      },
      "username": {
        "type": "text",
        "analyzer": "whitespace_lowercase",
        "fields": {
          "keyword": {
            "type": "keyword"
          }
        }
      },
      "geoip": {
        "properties": {
          "ip": {
            "type": "ip"
          },
          "location": {
            "type": "geo_point"
          },
          "latitude": {
            "type": "half_float"
          },
          "longitude": {
            "type": "half_float"
          }
        }
      }
    }
  }
}
}
4

If you provide too info much it can be overwhelming and people just give up, if it's too little the it can be underwhelming and the same thing can happen.

It's terribly hard to guess what level of knowledge someone has, but if you are after something specific then being explicit really helps us help you :slight_smile:

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.