Visualisation don't work even after recreating index templates

Elastic Stack Kibana
1

Ok, so i got a big problem. I wanted to change some settings in my index template.
I have a JSON template definition file which i used to import long time ago, before there was any data and dashboards. At that time this index import definition worked.
It was imported using a command like:

curl -X POST -u USERNAME:PASSWORD http://KIBANASERVER:5601/api/saved_objects/index-pattern/elastiflow-* -H "Content-Type: application/json" -H "kbn-xsrf: true" -d @/PATH/TO/elastiflow.index_pattern.json

When i tried to do the same today: i get an error: {"statusCode":400,"error":"Bad Request","message":"child "attributes" fails because ["attributes" is required]. "template" is not allowed. "settings" is not allowed. "mappings" is not allowed. "aliases" is not allowed","validation":{"source":"payload","keys":["attributes","template","settings","mappings","aliases"]}}

It seems you can't change or upload a new index pattern, as long as one exists.

So i removed my index pattern completely in Kibana, hoping that i could then import it again -> no difference.
Then i stopped logstash, removed all daily elastiflow-* indexes in Elasticsearch using Cerebro, so that all references to data were gone.
Tried to import again -> no difference (i have no idea how this is possible, what is different when i started with a new installation when it worked ???)

Creating a new index pattern without data seems also to be impossible, so i started logstash again, which luckily imported again data to Elastic. Using this new data, i could recreate the index pattern again with the same name (elastiflow-), and now my "Discover" etc seems to work again. However, NONE of my Visualisations and Dashboards work anymore !! All fail with the error "can't find index pattern elastiflow-", however, it is shown when i go to Kibana->Index Patterns.

I have no idea how to get my Dashboards working again....
When i look at a saved object of a simple counter visualisation, i see only a references to the name of the index, so i don't understand why he doesn't find this index:

{
"index": "elastiflow-",
"query": {
"query": {
"query_string": {
"query": "",
"analyze_wildcard": true
}
},
"language": "lucene"
},
"filter": []
}
{
"title": "ElastiFlow: City Count",
"type": "metric",
"params": {
"addLegend": false,
"addTooltip": true,
"fontSize": "32",
"gauge": {
"autoExtend": false,
"backStyle": "Full",
"colorSchema": "Green to Red",
"colorsRange": [
{
"from": 0,
"to": 100
}
],
"gaugeColorMode": "None",
"gaugeStyle": "Full",
"gaugeType": "Metric",
"invertColors": false,
"labels": {
"color": "black",
"show": true
},
"orientation": "vertical",
"percentageMode": false,
"scale": {
"color": "#333",
"labels": false,
"show": false,
"width": 2
},
"style": {
"bgColor": false,
"bgFill": "#000",
"fontSize": "36",
"labelColor": false,
"subText": ""
},
"type": "simple",
"useRange": false,
"verticalSplit": false
},
"handleNoResults": true,
"type": "gauge"
},
"aggs": [
{
"id": "1",
"enabled": true,
"type": "cardinality",
"schema": "metric",
"params": {
"field": "flow.geoip.city_name",
"customLabel": "Cities"
}
}
],
"listeners": {}
}

2

The problem is that the id of the index pattern changed when you recreated it. In the past, the id for all saved objects was just their name, so your old elastiflow-* pattern had the same ID. Likewise, all the saved objects that referenced it (visualizations and saved searches) used that value for their index.

Some time ago, Kibana switched to using UUID strings for the value to allow for saved objects with the same name (mostly to enable multiple index patterns with the same pattern).

Your new index pattern in elastiflow-*, but the id in Elasticsearch is going to be some UUID value, which you'll have to look up. Once you know that value, you have 2 options:

  • Change the id of the index pattern to elastiflow-*
  • Change the kibanaSavedObjectMeta.searchSourceJSON.index value in all of your saved visualization objects to match the UUID of the index pattern
3

ok, so Cerebro shows a UUID string for an index, but i hope it doesn't change per day...

{
"elastiflow-2018.03.30": {
"settings": {
"index": {
"codec": "best_compression",
"refresh_interval": "5s",
"number_of_shards": "2",
"provided_name": "elastiflow-2018.03.30",
"creation_date": "1522442205292",
"number_of_replicas": "0",
"uuid": "ZGSWGSI7RmOAC6spcbC5Vg",
"version": {
"created": "6020299"
}
}
}
}
}

4

Ok, so i removed my index pattern again, and recreated it again, now specifying elastiflow-* as ID under Advanced Options.
And.... it worked. Thank you very much !!!

The index pattern still creates 1 replica by default, giving 2 unassigned shards each day...i can change the number of replica's on existing indexes, but this whole mess started because i wanted to change it on the pattern, so that it would also count for future generated indexes.......

5

Are you talking about controlling the shards in ES for a given pattern? You can use index templates to do that. Kibana doesn't control any of that though, the index pattern in Kibana is just a way to control how data is pulled from Elasticsearch.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.