ES master:
curl localhost:9200/
{
"name" : "AZPRDBRK007",
"cluster_name" : "monitoring-cluster",
"cluster_uuid" : "7n4PuY7FQgWSIS0-2LxCRg",
"version" : {
"number" : "7.9.0",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "a479a2a7fce0389512d6a9361301708b92dff667",
"build_date" : "2020-08-11T21:36:48.204330Z",
"build_snapshot" : false,
"lucene_version" : "8.6.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
Kibana verion: 7.9.3
I am getting Internal server 500 error - [illegal_argument_exception] application privileges must refer to at least one resource"} on accessing Kibana after I enter the elastic user name and password. Same goes for every user.
I have verified that the .kibana index is getting created when I delete it manually and restart the Kibana
Adding some stack trace from Kibana's logs
Aug 26 05:17:42 AZPRDBRK008 systemd[1]: Started Kibana.
Aug 26 05:17:50 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:17:50Z","tags":["warning","plugins-discovery"],"pid":25404,"message":"Expect plugin \"id\" in camelCase, but found: beats_management"}
Aug 26 05:17:50 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:17:50Z","tags":["warning","plugins-discovery"],"pid":25404,"message":"Expect plugin \"id\" in camelCase, but found: triggers_actions_ui"}
Aug 26 05:18:02 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:02Z","tags":["info","plugins-service"],"pid":25404,"message":"Plugin \"visTypeXy\" is disabled."}
Aug 26 05:18:02 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:02Z","tags":["info","plugins-service"],"pid":25404,"message":"Plugin \"auditTrail\" is disabled."}
Aug 26 05:18:02 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:02Z","tags":["warning","config","deprecation"],"pid":25404,"message":"Config key [monitoring.cluster_alerts.email_notifications.email_address] will be required for email notifications to work in 8.0.\""}
Aug 26 05:18:03 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:03Z","tags":["warning","plugins","security","config"],"pid":25404,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"}
Aug 26 05:18:03 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:03Z","tags":["warning","plugins","reporting","config"],"pid":25404,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
Aug 26 05:18:03 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:03Z","tags":["warning","plugins","encryptedSavedObjects","config"],"pid":25404,"message":"Generating a random key for xpack.encryptedSavedObjects.encryptionKey. To be able to decrypt encrypted saved objects attributes after restart, please set xpack.encryptedSavedObjects.encryptionKey in kibana.yml"}
Aug 26 05:18:03 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:03Z","tags":["warning","plugins","ingestManager"],"pid":25404,"message":"Fleet APIs are disabled due to the Encrypted Saved Objects plugin using an ephemeral encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in kibana.yml."}
Aug 26 05:18:03 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:03Z","tags":["warning","plugins","actions","actions"],"pid":25404,"message":"APIs are disabled due to the Encrypted Saved Objects plugin using an ephemeral encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in kibana.yml."}
Aug 26 05:18:03 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:03Z","tags":["warning","plugins","alerts","plugins","alerting"],"pid":25404,"message":"APIs are disabled due to the Encrypted Saved Objects plugin using an ephemeral encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in kibana.yml."}
Aug 26 05:18:03 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:03Z","tags":["warning","plugins","monitoring","monitoring"],"pid":25404,"message":"X-Pack Monitoring Cluster Alerts will not be available: undefined"}
Aug 26 05:18:03 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:03Z","tags":["info","plugins","reporting","config"],"pid":25404,"message":"Chromium sandbox provides an additional layer of protection, and is supported for Linux Ubuntu Linux 18.04 OS. Automatically enabling Chromium sandbox."}
Aug 26 05:18:03 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:03Z","tags":["info","savedobjects-service"],"pid":25404,"message":"Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations..."}
Aug 26 05:18:03 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:03Z","tags":["info","savedobjects-service"],"pid":25404,"message":"Starting saved objects migrations"}
Aug 26 05:18:04 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:04Z","tags":["info","plugins-system"],"pid":25404,"message":"Starting [92] plugins: [taskManager,licensing,observability,globalSearch,globalSearchProviders,code,usageCollection,ossTelemetry,telemetryCollectionManager,telemetry,telemetryCollectionXpack,kibanaUsageCollection,newsfeed,mapsLegacy,kibanaLegacy,translations,timelion,share,legacyExport,esUiShared,bfetch,expressions,data,home,cloud,console,consoleExtensions,apmOss,searchprofiler,painlessLab,grokdebugger,management,upgradeAssistant,licenseManagement,indexPatternManagement,advancedSettings,fileUpload,dataEnhanced,savedObjects,visualizations,visTypeVega,visualize,visTypeTimelion,features,security,snapshotRestore,reporting,enterpriseSearch,encryptedSavedObjects,ingestManager,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,beats_management,transform,ingestPipelines,maps,graph,canvas,visTypeTable,visTypeMarkdown,tileMap,regionMap,inputControlVis,dashboard,dashboardMode,charts,lens,visTypeVislib,visTypeTimeseries,rollup,visTypeTagcloud,visTypeMetric,watcher,discover,discoverEnhanced,savedObjectsManagement,spaces,lists,eventLog,actions,case,alerts,alertingBuiltins,ml,securitySolution,infra,monitoring,logstash,apm,uptime]"}
Aug 26 05:18:04 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:04Z","tags":["info","plugins","taskManager","taskManager"],"pid":25404,"message":"TaskManager is identified by the Kibana UUID: 6175f21d-d4d8-44da-b8ff-8c0bf72675f1"}
Aug 26 05:18:04 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:04Z","tags":["info","plugins","crossClusterReplication"],"pid":25404,"message":"Your basic license does not support crossClusterReplication. Please upgrade your license."}
Aug 26 05:18:04 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:04Z","tags":["info","plugins","watcher"],"pid":25404,"message":"Your basic license does not support watcher. Please upgrade your license."}
Aug 26 05:18:04 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:04Z","tags":["info","plugins","monitoring","monitoring","kibana-monitoring"],"pid":25404,"message":"Starting monitoring stats collection"}
Aug 26 05:18:05 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:05Z","tags":["status","plugin:kibana@7.9.0","info"],"pid":25404,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
Aug 26 05:18:05 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:05Z","tags":["status","plugin:elasticsearch@7.9.0","info"],"pid":25404,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
Aug 26 05:18:05 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:05Z","tags":["warning"],"pid":25404,"kibanaVersion":"7.9.0","nodes":[{"ip":"10.129.23.13","version":"7.9.3","http":{"publish_address":"10.129.23.13:9200"},"name":"v7.9.3 @ 10.129.23.13:9200 (10.129.23.13)"},{"ip":"10.129.23.12","version":"7.9.3","http":{"publish_address":"10.129.23.12:9200"},"name":"v7.9.3 @ 10.129.23.12:9200 (10.129.23.12)"},{"ip":"10.129.23.6","version":"7.9.3","http":{"publish_address":"10.129.23.6:9200"},"name":"v7.9.3 @ 10.129.23.6:9200 (10.129.23.6)"},{"ip":"10.129.23.14","version":"7.9.3","http":{"publish_address":"10.129.23.14:9200"},"name":"v7.9.3 @ 10.129.23.14:9200 (10.129.23.14)"}],"message":"You're running Kibana 7.9.0 with some different versions of Elasticsearch. Update Kibana or Elasticsearch to the same version to prevent compatibility issues: v7.9.3 @ 10.129.23.13:9200 (10.129.23.13), v7.9.3 @ 10.129.23.12:9200 (10.129.23.12), v7.9.3 @ 10.129.23.6:9200 (10.129.23.6), v7.9.3 @ 10.129.23.14:9200 (10.129.23.14)"}
Aug 26 05:18:05 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:05Z","tags":["status","plugin:elasticsearch@7.9.0","info"],"pid":25404,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
Aug 26 05:18:05 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:05Z","tags":["status","plugin:xpack_main@7.9.0","info"],"pid":25404,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
Aug 26 05:18:05 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:05Z","tags":["status","plugin:monitoring@7.9.0","info"],"pid":25404,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
Aug 26 05:18:05 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:05Z","tags":["status","plugin:spaces@7.9.0","info"],"pid":25404,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
Aug 26 05:18:05 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:05Z","tags":["status","plugin:security@7.9.0","info"],"pid":25404,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
Aug 26 05:18:05 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:05Z","tags":["status","plugin:beats_management@7.9.0","info"],"pid":25404,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
Aug 26 05:18:06 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:06Z","tags":["status","plugin:apm_oss@7.9.0","info"],"pid":25404,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
Aug 26 05:18:06 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:06Z","tags":["status","plugin:console_legacy@7.9.0","info"],"pid":25404,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
Aug 26 05:18:06 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:06Z","tags":["listening","info"],"pid":25404,"message":"Server running at https://0.0.0.0:5601"}
Aug 26 05:18:07 AZPRDBRK008 kibana[25404]: {"type":"log","@timestamp":"2021-08-26T05:18:07Z","tags":["info","http","server","Kibana"],"pid":25404,"message":"http server running at https://0.0.0.0:5601"}
Aug 26 05:18:25 AZPRDBRK008 kibana[25404]: {"type":"error","@timestamp":"2021-08-26T05:18:25Z","tags":["connection","client","error"],"pid":25404,"level":"error","error":{"message":"47621343212288:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:322:\n","name":"Error","stack":"Error: 47621343212288:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:322:\n"},"message":"47621343212288:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:322:\n"}
Aug 26 05:19:37 AZPRDBRK008 kibana[25404]: {"type":"error","@timestamp":"2021-08-26T05:19:37Z","tags":["connection","client","error"],"pid":25404,"level":"error","error":{"message":"47621343212288:error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown:../deps/openssl/openssl/ssl/record/rec_layer_s3.c:1550:SSL alert number 46\n","name":"Error","stack":"Error: 47621343212288:error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown:../deps/openssl/openssl/ssl/record/rec_layer_s3.c:1550:SSL alert number 46\n"},"message":"47621343212288:error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown:../deps/openssl/openssl/ssl/record/rec_layer_s3.c:1550:SSL alert number 46\n"}