Kibana 5.6.1 remote transport exception

[benpolzin]

Hi @tsmalley, thanks for offering to give this a look. This is from my post in Upgrade Issue with Elastic Stack 5.6.0.

Here's the process I went through.

1. Started with Elastic Stack 5.2 (prior to that we started on one of the last 2.x releases and jumped through 5.0). We have a pretty small environment with 3 ES data nodes and a Kibana server that also has ES acting as a 'coordinating only node'.
2. Upgraded to 5.6.0 and ran into the issue described in the above link exactly.
3. Ran the 8134-patch.sh on Kibana
4. At this point everything was happy on 5.6.0. ES was "green". Kibana was "green" and I could see data, visualizations, etc.
5. Upgraded to 5.6.1
6. ES upgraded fine, its health is green and it's taking in new logs from Logstash
7. Kibana upgraded okay, starts, but when going to the web page the status is Red.
   
   

I think I have this narrowed down, but I'm pretty new to Elastic so I'm not sure what the 'right' or correct way to resolve this is.

Here is what I think is the relevant log line from Kibana's stdout log:

{"type":"log","@timestamp":"2017-09-20T10:57:28Z","tags":["info","elasticsearch"],"pid":17028,"typeName":"url","typeMapping":{"properties":{"accessCount":{"type":"long"},"accessDate":{"type":"date"},"createDate":{"type":"date"},"url":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":2048}}}}},"message":"Adding mappings to kibana index for SavedObject type \"url\""}

And here is the more helpful log line from Elastic's log

[2017-09-21T09:17:51,091][DEBUG][o.e.a.a.i.m.p.TransportPutMappingAction] [elkelastic3] failed to put mappings on indices [[[.kibana-6/LaISONzyRfGlu2gzSLxvhA]]], type [url]
java.lang.IllegalArgumentException: [url] is defined as a field in mapping [url] but this name is already used for an object in other types
        at org.elasticsearch.index.mapper.MapperService.checkFieldUniqueness(MapperService.java:563) ~[elasticsearch-5.6.1.jar:5.6.1]
...

I'm fairly certain what I'm running into is related to the removal of types.

In my .kibana index mapping is this:

          "url" : {
            "properties" : {
              "accessCount" : {
                "type" : "long"
              },
              "accessDate" : {
                "type" : "date"
              },
              "createDate" : {
                "type" : "date"
              },
              "url" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 2048
                  }
                }
              }
            }
          },

And url shows up twice there, which I believe causes the conflict. But I don't see the type setting set to single. Full contents of .kibana index available here.

.kibana is already an alias thanks to the 8134-patch.sh script. So my plan was to find the default mapping/settings for a Kibana 5.6.1 install, create a new index with those settings named .kibana-61 and then update the alias. If I make things worse then I just flip the alias back to the current .kibana-6.

Am I going down the right path here?

[tylersmalley]

Please see Kibana unaccessible after using Migration UI for details and a workaround

[benpolzin]

Thanks! There were a couple errors thrown, but it resolved the issue and Kibana is functional again.