Kibana not finding filebeat-* index. Direct search on Elasticsearch works


(Srirang G D) #1

I am trying to get a simple log analysis system setup to push the logs of my Rails application to Elasticsearch. I have setup Elasticsearch, Filebeat and Kibana as per the docs on a Ubuntu 16.04 server (using the apt repository). I am using JRE-8 from OpenJDK. Not using Logstash at the moment.

  • I have installed X-Pack for Elasticsearch and Kibana.
  • User passwords were setup for the 3 reserved users and the credentials updated in kibana.yml

My pipeline is like this :

[Rails log files] ==> [Filebeat] ==> [Elasticsearch] <== [Kibana]

Filebeat is successfully reading the log file and pushing it to Elasticsearch. It has successfully created the the index in Elasticsearch and also loaded the templates in Kibana.

  • On launching Kibana (http://my-machine-ip:5601/), it loads up fine and shows me the login page
  • I login with the credentials of the "kibana" user setup earlier after X-Pack installation
  • Monitoring shows all the indexes (indices?) available, including the filebeat index along with the count of documents and amount of data index. All the usual stuff.

So Kibana is able to access Elasticsearch.

If I query Elasticsearch directly, I see the documents pushed by Filebeat under the filebeat-* index.

  • http://my-machine-ip:9200/filebeat-*/_search?q=940 successfully returns all records with 940 in it.

However when I try to view the same data under Kibana it is not able to find any filebeat-* index. Here is the error it throws up :

Error: No matching indices found: No indices match pattern "filebeat-*"
KbnError@http://10.10.93.51:5601/bundles/commons.bundle.js?v=16627:1:20695
IndexPatternMissingIndices@http://10.10.93.51:5601/bundles/commons.bundle.js?v=16627:1:26214
request/<@http://10.10.93.51:5601/bundles/commons.bundle.js?v=16627:21:576005
processQueue@http://10.10.93.51:5601/bundles/vendors.bundle.js?v=16627:116:132456
scheduleProcessQueue/<@http://10.10.93.51:5601/bundles/vendors.bundle.js?v=16627:116:133349
$digest@http://10.10.93.51:5601/bundles/vendors.bundle.js?v=16627:116:144239
$apply@http://10.10.93.51:5601/bundles/vendors.bundle.js?v=16627:116:147007
done@http://10.10.93.51:5601/bundles/vendors.bundle.js?v=16627:116:100015
completeRequest@http://10.10.93.51:5601/bundles/vendors.bundle.js?v=16627:116:104697
createHttpBackend/</xhr.onload@http://10.10.93.51:5601/bundles/vendors.bundle.js?v=16627:116:105435

What am I missing here? Why is Kibana unable to find the filebeat index?

Any more information needed?


(Christian Dahlqvist) #2

Are you using a user that has access to the filebeat-* indices (typically a user with the kibana_user role as well as a role that grants access to logstash-*)?


(Srirang G D) #3

How do I check what all roles the current user has?

I have not created any users explicitly. I have only set the password for the three reserved users during x-pack installation. The three users are

  • elastic - This is the user used while querying elasticsearch directly and also in filebeat.yml
  • kibana - This is the user specified in kibana.yml and also used to login in Kibana dashboard.
  • logstash_system

I haven't altered or assigned roles to these users. They are what they are by default.

The kibana user doesn't seem to have the necessary privileges to be able to manage users and roles.

How do I go about manging users and roles in this scenario? Do I login to Kibana dashboard using the elastic user?


(Christian Dahlqvist) #4

The standard kibana user is the one the Kibana application uses to get access to Elasticsearch, and does therefore not have access to any data. To use Kibana, you need to either log in as elastic, which is a superuser, or create a new user as I described above.

You can manage users and roles by logging into Kibana using the elastic user.


(Srirang G D) #5

Ok, something weird is happening.

  1. I was logged into Kibana dashboard using the kibana user (which was having all the access issues described in my previous posts)
  2. I logged out and was taken to the login screen
  3. I logged in using the elastic user (using which I was able to query Elasticsearch directly).
  4. Login was successful, but when the dashboard loaded it still shows kibana as the logged in user and all the access issues are still there.

All this time kibana.yml has the credentials of the kibana user.

  1. I change it to the credentials of elastic user (i.e in kibana.yml file) and restart kibana service.
  2. I logout from the Kibana dashboard and login with the credentials of elastic user.
  3. Login successful, but the logged in user shown is still kibana and all the access issues still persist.

What is happening here?

I am doing all of this in a Firefox private browsing window (if that has any bearing here).

I have a few questions about the authentication / authorization in this ES + Kibana setup based on what you have described. Here is what I had initially thought :

  1. The user mentioned in the kibana.yml is what the Kibana system uses to read data from Elasticsearch and by default this user has access to read all data in Elasticsearch.

  2. The authentication and authorization of the user whose credentials are provided in the Kibana login screen is handled by Kibana itself and Kibana acts as the gatekeeper deciding who has access to what data. However Kibana will always use the user mentioned in kibana.yml to actually read the data from Elasticsearch.

From what you have described, this doesn't seem to be the case. It appears that Kibana just uses the same credentials that are provided during the login to further connect to Elasticsearch and thereby offloading all authentication and authorization responsibility to Elasticsearch itself.

If this is indeed the case, then for what purposes does Kibana use the user credentials specified in kibana.yml? Is that only for certain Kibana specific housekeeping works?

I am going through the Elasticsearch and Kibana documentation, but if there is a specific page that you think I should be reading to get the answers to the above questions, please do point me to that.

Thank you.


(Srirang G D) #6

Hi Christian,

I think you can ignore the "weird" report mentioned in my previous post. It was probably something to do with the way Kibana was (is) maintaning the session. I closed that browser window and opened a new one and I am able to logout and login as a different user.

I logged in using the elastic user and created another user with the kibana_user role. Additionally I created another role with explicit permissions to filebeat-* indices and assigned that role to this new user. Now when I login with this new user, I am able to see the data. I skipped logstash-* indices since I am not using Logstash currently.

Thank you very much Christian.

Further, it would be very helpful if you can shed some light on my question about the authentication / authorization model present in ES + Kibana.


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.