Kibana Index Patterns - Refresh Types

[chris_ts24]

Hi everyone,

I'm using Kibana 7.13.1. I've established a Logstash instance on some web hosting servers that I connect to an elasticsearch data stream to store IIS (Internet Information Services) logs in a daily rolling index.

The setup works, but I've run into a problem that I noticed I cannot aggregate all fields in my dashboards, which I would like to visualize patterns in my logs with. I realized that it is because text fields cannot be aggregated.

That's alright, because I don't mind using a keyword type. So, I changed the type of the fields from text to keyword in my IIS component template. The index template simply references the component template and nothing else, so I'll just include my mapping for the component.

I had assumed that changing the template field type would automatically carry over to the index pattern that uses the template. This is however not the case.

I've looked into refreshing this index pattern to bring these underlying type changes into the index pattern so that I can aggregate over the keyword fields, but with no success. Can anyone offer advice / insight into what is going on here?

I hope this isn't the case, but do I have to pause document collection via Logstash on all my servers, delete the index pattern, and restart my services, in order to refresh this? Worse would be if I have to delete all my previous indices created by this data stream in order to get these field types to refresh, but that seems drastic. I feel like this can be done in some way.

If anyone has experience in this area, I would really appreciate it!

[chris_ts24]

Hi all,

I've found that the new indexes created by the ILM are producing indices with correct field types. Since my logs roll every day anyway and expire after a week, I'm going to just let them refresh themselves.

I think the only other option that would remain would be to reindex the old indices, but as I said, it's not extremely important that these older logs hold aggregatable fields. For anyone that has this problem that really needs those old logs, I would suggest using a reindex. I tested it, and the fields had aggregatable types that I set in the component templates, so it seems the reindex uses the refreshed templates.

Hope this manages to help someone!