Index ".kibana" is not getting created when using Elasticsearch and Kibana on docker-compose

I am using Docker Compose on my Mac machine to run the Elasticsearch and Kibana.

After bringing docker compose up, I am able to access both elasticsearch and kibana using their URLs, but Kibana doesn't has default ".kibana" index.

when I run command in Dev tools to see list of indexes, I am seeing only below listed indexes. It doesn't have .kibana.

GET _cat/indices

Output for above command:
green open .monitoring-es-6-2018.03.15 Jd4rZnZIQz-_adEvaQpQcg 1 1 480 14 1.1mb 664kb
green open .monitoring-es-6-2018.03.16 rBGtnqCQQl-rMG6gPnY6KQ 1 1 1613 30 3.5mb 1.7mb
green open .monitoring-kibana-6-2018.03.16 WOcerQJaT16gd6IG1rtv3A 1 1 142 0 324.7kb 183.3kb

NOTE:
When I pulled and ran Elaticsearch & Kibana individually using Docker (docker pull and docker run commands), that time everything works fine. ".kibana" index was also got created and I was able to create index pattern etc.

I am facing issue when I am trying to configure my environment using "docker compose".
I tried using different URL for elasticserch in kibana like "http://localhost:9200" but that too behave same as code posted below.

Working Environment : I am using Mac machine

Updated:
I noticed below warnings in logs:

ibana | {"type":"log","@timestamp":"2018-03-16T03:35:23Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}

kibana | {"type":"log","@timestamp":"2018-03-16T03:35:25Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}

kibana | {"type":"log","@timestamp":"2018-03-16T03:35:25Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}

kibana | {"type":"log","@timestamp":"2018-03-16T03:35:28Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}

========= docker-compose.yml ===================
Below is my code to run in docker compose.

=====

version: '2.2'

services:

elasticsearch:

image: docker.elastic.co/elasticsearch/elasticsearch:6.2.2

container_name: elasticsearch

environment:

  - cluster.name=docker-cluster

  - bootstrap.memory_lock=true

  - "ES_JAVA_OPTS=-Xms512m -Xmx512m"

ulimits:

  memlock:

    soft: -1

    hard: -1

volumes:

  - esdata1:/usr/share/elasticsearch/data

ports:

  - 9200:9200

networks:

  - esnet

elasticsearch2:

image: docker.elastic.co/elasticsearch/elasticsearch:6.2.2

container_name: elasticsearch2

environment:

  - cluster.name=docker-cluster

  - bootstrap.memory_lock=true

  - "ES_JAVA_OPTS=-Xms512m -Xmx512m"

  - "discovery.zen.ping.unicast.hosts=elasticsearch"

ulimits:

  memlock:

    soft: -1

    hard: -1

volumes:

  - esdata2:/usr/share/elasticsearch/data

networks:

  - esnet

kibana:

image: docker.elastic.co/kibana/kibana:6.2.2

container_name: kibana

environment:

  - "ELASTICSEARCH_URL=http://elasticsearch:9200"

ports:

  - 5601:5601

networks:

  - esnet

volumes:

esdata1:

driver: local

esdata2:

driver: local

networks:

esnet:

Are you sure elasticsearch instance is running at "ELASTICSEARCH_URL=http://elasticsearch:9200" ? Double check that that address is correct.

Another thing to note:

We now only create the .kibana index once data has been persisted. For example, saving a search will create the index if it doesn't exist.

Thanks Archana for your reply.

Somehow, now I am not at all able to connect to Kibana. Only once I was able to see home screen but with some invalid credentials disabled related screen, otherwise not able to connect to Kibana.

Regarding Elasticsearch URL, Elasticsearch wasn't getting access on URL http://elasticsearch:9200 hence I changed the docker-compose.yml with "ELASTICSEARCH_URL=http://localhost:9200"
also tried, "ELASTICSEARCH_URL=http://127.0.0.1:9200"
NOTE - On these both URLs, I am able to access Elasticsearch (testing using both curl command as well as from browser).
But now I am not able to access Kibana at all with any elasticsearch URL configuration (old/new).

Please suggest! Many thanks for your help!

I am getting error message on Kibana - appearing a login screen saying -- "Login is currently disabled. Administrators should consult the Kibana logs for more details", with disabled text fields "username" and "password" to login.

Though I didn't make any other change in docker-compose.yml. I am not setting/using any password related configurations anywhere. All is default

Attached logs contain "transport related" exception I am getting during the startup of Elastic search
However, Elasticsearch is accessible at http://localhost:9200
But, during startup, noticed below exception -

Due to limitation, couldn;t upload whole log in single post. Sorry for that.

elasticsearch | [2018-03-19T02:24:01,227][WARN ][o.e.d.z.ElectMasterService] [5sEPH2C] value for setting "discovery.zen.minimum_master_nodes" is too low. This can result in data loss! Please set it to at least a quorum of master-eligible nodes (current value: [1], total number of master-eligible nodes used for publishing in this round: [2])

elasticsearch2 | [2018-03-19T02:24:01,373][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [pFjzYR5] publish_address {172.19.0.2:9200}, bound_addresses {0.0.0.0:9200}

elasticsearch2 | [2018-03-19T02:24:01,375][INFO ][o.e.n.Node ] [pFjzYR5] started

elasticsearch2 | [2018-03-19T02:24:02,611][INFO ][o.e.x.s.a.TokenService ] [pFjzYR5] refresh keys

elasticsearch2 | [2018-03-19T02:24:03,026][INFO ][o.e.x.s.a.TokenService ] [pFjzYR5] refreshed keys

elasticsearch2 | [2018-03-19T02:24:03,062][INFO ][o.e.l.LicenseService ] [pFjzYR5] license [c68e8d41-960e-4e1a-af1e-f01c5033432a] mode [basic] - valid

elasticsearch2 | [2018-03-19T02:24:03,246][INFO ][o.e.d.z.ZenDiscovery ] [pFjzYR5] master_left [{5sEPH2C}{5sEPH2CPQIeGJNrxpnjCmA}{D3PnNgbRT7eMRM0C3R7oWA}{172.19.0.3}{172.19.0.3:9300}{ml.machine_memory=2095816704, ml.max_open_jobs=20, ml.enabled=true}], reason [failed to ping, tried [3] times, each with maximum [30s] timeout]

elasticsearch2 | [2018-03-19T02:24:03,247][WARN ][o.e.d.z.ZenDiscovery ] [pFjzYR5] master left (reason = failed to ping, tried [3] times, each with maximum [30s] timeout), current nodes: nodes:

elasticsearch2 | {5sEPH2C}{5sEPH2CPQIeGJNrxpnjCmA}{D3PnNgbRT7eMRM0C3R7oWA}{172.19.0.3}{172.19.0.3:9300}{ml.machine_memory=2095816704, ml.max_open_jobs=20, ml.enabled=true}, master

elasticsearch2 | {pFjzYR5}{pFjzYR5OTlqYiOESQZDvig}{OAMi8f_5T5GVswy-XgN7UA}{172.19.0.2}{172.19.0.2:9300}{ml.machine_memory=2095816704, ml.max_open_jobs=20, ml.enabled=true}, local

elasticsearch2 |

elasticsearch2 | [2018-03-19T02:24:03,251][INFO ][o.e.x.w.WatcherService ] [pFjzYR5] stopping watch service, reason [no master node]

elasticsearch2 | [2018-03-19T02:24:03,282][WARN ][o.e.d.z.UnicastZenPing ] [pFjzYR5] failed to send ping to [{5sEPH2C}{5sEPH2CPQIeGJNrxpnjCmA}{D3PnNgbRT7eMRM0C3R7oWA}{172.19.0.3}{172.19.0.3:9300}{ml.machine_memory=2095816704, ml.max_open_jobs=20, ml.enabled=true}]

elasticsearch2 | org.elasticsearch.transport.RemoteTransportException: [5sEPH2C][172.19.0.3:9300][internal:discovery/zen/unicast]

elasticsearch2 | Caused by: org.elasticsearch.ElasticsearchSecurityException: missing authentication token for action [internal:discovery/zen/unicast]

elasticsearch2 | at org.elasticsearch.xpack.core.security.support.Exceptions.authenticationError(Exceptions.java:36) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.core.security.authc.DefaultAuthenticationFailureHandler.missingToken(DefaultAuthenticationFailureHandler.java:74) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.security.authc.AuthenticationService$AuditableTransportRequest.anonymousAccessDenied(AuthenticationService.java:535) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$handleNullToken$17(AuthenticationService.java:347) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.handleNullToken(AuthenticationService.java:352) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.consumeToken(AuthenticationService.java:267) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$extractToken$7(AuthenticationService.java:238) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.extractToken(AuthenticationService.java:256) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$0(AuthenticationService.java:190) ~[?:?]

elasticsearch2 | at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:60) ~[elasticsearch-6.2.2.jar:6.2.2]

elasticsearch2 | at org.elasticsearch.xpack.security.authc.TokenService.getAndValidateToken(TokenService.java:313) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:186) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$4(AuthenticationService.java:217) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:228) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:182) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:143) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:113) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.inbound(ServerTransportFilter.java:142) ~[?:?]

elasticsearch2 | at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:314) ~[?:?]

elasticsearch2 | at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66) ~[elasticsearch-6.2.2.jar:6.2.2]

elasticsearch2 | at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1555) ~[elasticsearch-6.2.2.jar:6.2.2]

elasticsearch2 | at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.2.2.jar:6.2.2]

elasticsearch2 | at org.elasticsearch.common.util.concurrent.EsExecutors$1.execute(EsExecutors.java:135) ~[elasticsearch-6.2.2.jar:6.2.2]

elasticsearch2 | at org.elasticsearch.transport.TcpTransport.handleRequest(TcpTransport.java:1512) [elasticsearch-6.2.2.jar:6.2.2]

elasticsearch2 | at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1382) [elasticsearch-6.2.2.jar:6.2.2]

elasticsearch2 | at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:64) [transport-netty4-6.2.2.jar:6.2.2]

elasticsearch2 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) [netty-codec-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) [netty-codec-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241) [netty-handler-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:545) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:499) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) [netty-transport-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.16.Final.jar:4.1.16.Final]

elasticsearch2 | at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]

elasticsearch | [2018-03-19T02:24:03,911][INFO ][o.e.l.LicenseService ] [5sEPH2C] license [c68e8d41-960e-4e1a-af1e-f01c5033432a] mode [basic] - valid

elasticsearch | [2018-03-19T02:24:03,930][INFO ][o.e.g.GatewayService ] [5sEPH2C] recovered [2] indices into cluster_state

================****************** Below Logs related to KIBANA ***************************

kibana | {"type":"log","@timestamp":"2018-03-19T02:29:27Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}

kibana | {"type":"log","@timestamp":"2018-03-19T02:29:29Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://localhost:9200/"}

kibana | {"type":"log","@timestamp":"2018-03-19T02:29:29Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}

kibana | {"type":"log","@timestamp":"2018-03-19T02:29:29Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://localhost:9200/"}

kibana | {"type":"log","@timestamp":"2018-03-19T02:29:29Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}

kibana | {"type":"log","@timestamp":"2018-03-19T02:29:29Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://localhost:9200/"}

kibana | {"type":"log","@timestamp":"2018-03-19T02:29:29Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}

kibana | {"type":"log","@timestamp":"2018-03-19T02:29:32Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://localhost:9200/"}

kibana | {"type":"log","@timestamp":"2018-03-19T02:29:32Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}

kibana | {"type":"log","@timestamp":"2018-03-19T02:29:34Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://localhost:9200/"}

kibana | {"type":"log","@timestamp":"2018-03-19T02:29:34Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.