New mapped field not getting added as searchable even after doing "refresh field list"

[lmrc1r0]

Hello,
I recently upgraded from 5.X to 6.7.2.
Most of my new logging indexes are fresh on the system, not imported.
I also imported some old indexes from old ES cluster using reindex.
So all indexes should be in native 6.7.2 format etc.

I recently added a new field to my logging setup in 6.7.2.
I needed to add "fields.type" because, with 6+ ES and filebeats 6.?+ removal of types
it was breaking logstash filter rules for matching.

Filebeats.yml now looks like this

filebeat.inputs:
 
 - type: log
   paths:
     - /var/log/nginx/access.log
   fields:
    environment: env1
   type: nginx_access

There is also a matching logstash filter entry:

if [type] == 'nginx_access' or [fields][type] == 'nginx_access' {

That is working as I see the entries show up in Kibana as

Please note that a preexisting fields.environment is and has been working since I setup the cluster.

I am using a template that has these mapping.
I updated the template to have the new mapping.
(leaving out some details for brevity from a GET _template)

"index_patterns" : [
  "logs-*"
],

---

},
"mappings" : {
  "doc" : {
    "dynamic" : false,
    "properties" : {

---

      "fields.type" : {
        "normalizer" : "lowercase",
        "type" : "keyword"
      },
      "http_upstream_connect_time" : {
        "type" : "float"
      },
      "fields.environment" : {
        "normalizer" : "lowercase",
        "type" : "keyword"
      },

I know logstash is processing and setting the "field.type" and I can see it in Kibana.

I have attempted multiple times to use the "Refresh field list" option in the Kibana UI.

I do have multiple index patterns in Kibana to allow for segmented "views".
I did send some new data with the new fields before setting the template.
However I am getting new data and at least once new index has been created (daily index for logging) with the new fields.

Does anyone have any idea what I might need to do to get Kibana to update and use the new field.type as searchable ?

It does not show up in the list of fields in the index despite multiple field refresh attempts.

I have not seen any issues in Kibana or ES logs. I do see new .kibana indexes being created when I attempt the refresh.

The changes made were in reference to this

Now that support for type has been removed in filebeat 6, how to add filters in logstash config?

From what I read, filebeat does not honor document_type anymore (from version 6 onwards) and all _type gets hardcoded to doc. I read some people getting around this by adding a field called document_type. Something like this in filebeat yml filebeat.prospectors: - type: log # Change to true to enable this prospector configuration. enabled: true # Paths that should be crawled and fetched. Glob based paths. paths: - XXX\LogFiles\W3SVC1\*.log fields: …

Is it possible that type is a reserved name that breaks when used anywhere?

Thanks very much in advance for any help or ideas!