We have placed the Elastic Search behind secure session based proxy to avoid unauthorized access.
Is there any other way apart from using shield to make sure Kibana can access elastic search with custom authentication?
What I am confused is with configuring elasticsearch.url parameter in kibana.yml
If we configure the initial elasticsearch.url with some default authentication, would be able to point it to a dynamic url later?
Can you clarify what you mean by "point to a dynamic url later"?
I'll be honest, there isn't a robust and easy way to do this without shield, but it should be possible.
One idea: since requests from Kibana to Elasticsearch are proxied from the client itself, you could put ES behind a reverse proxy that requires standard basic auth and then the first time that Kibana tries to query ES, it would prompt the user for a username and password.
But even that only gets you the client-side access, so things like creating the kibana index at startup would still need to be authenticated. I'm not sure what the best way to handle that is that is also future-proof. You could lock down access between Kibana and ES at the firewall level so that no explicit authentication is required between the two, but that's really only ideal when Kibana can access Elasticsearch from behind the firewall (on a local network) rather than on the open internet.
I hope this helps...