Kibana index pattern not saved

I have an ELK stack running on the Kubernetes cluster with security enabled. Everything is running fine and I am able to push data to an index. After logging in to Kibana as an admin user, and I to "Discover" it asks me to create an index pattern. So I have some metricbeat data, and I create a pattern and saved it. But when I go back to discover, it is prompting me to create an index pattern again!

I don't find any errors in Kibana/Elastic pods

Really appreciate any pointers

Elastisearch version: 7.10.1

Hi @anoopkv ,

Do you see your index pattern in Saved Objects management? If not try to create it again, but this time open Dev Tools so that you can see what request is sent to save it and what the response is and share request/response here.

Best,
Oleg

Hi @azasypkin

When i go to saved objects i get error "Unable find saved objects"

https://localhost:5601/api/kibana/management/saved_objects/_find?perPage=50&page=1&fields=id&type=config&type=url&type=index-pattern&type=query&type=dashboard&type=visualization&type=map&type=lens&type=search&type=infrastructure-ui-source&type=metrics-explorer-view&type=inventory-view&type=canvas-element&type=canvas-workpad&sortField=type

{"statusCode":400,"error":"Bad Request","message":"all shards failed: search_phase_execution_exception"}

  1. Request URL: https://localhost:5601/api/saved_objects/index-pattern
  2. Request Method: POST
  3. Status Code: 200
  4. Referrer Policy: strict-origin-when-cross-origin

It looks like you have something weird with your setup/Elasticsearch, sounds similar to this issue Created an Index Pattern in Kibana but it's not saved - but no error either · Issue #87607 · elastic/kibana · GitHub.

Have you tried to re-create cluster maybe? Sorry don't have anything concrete in mind yet.

I have now deleted and re-created my cluster. Deleted the associated pv in my k8s cluster
Still no luck :frowning:

Hmm, maybe try to enable Kibana verbose logging (logging.verbose: true) and restart Kibana so that logs contain everything right from the start and see if you can spot anything in there. If you don't see anything, remove any sensitive info from there and share the the file with logs here (via link to gist.github.com or anything like that since you cannot attach it directly here).

recreated kibana and now it is working. I can even see all the index patterns i have been creating earlier
so it seems the issue was only with kibana after all.
Thanks @azasypkin for your pointers, it was helpful :slight_smile:

Awesome, glad it's working now!