Docker elasticsearch home directory and all files related to elasticsearch

Elastic Stack Elasticsearch
,
1

Hi i have installed elasticsearch using docker by using the link - https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls-docker.html and generated certs accordingly.

Q1 : I want to where will be the elasticsearch home directory ? I checked under usr/share/ there is no elasticsearch folder created.
Q2: So i placed certs in other folder and getting the below exception

es02    | "Caused by: java.security.AccessControlException: access denied (\"java.io.FilePermission\" \"/apps/certs/ca/ca.crt\" \"read\")",
es02    | "at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]",
es02    | "at java.security.AccessController.checkPermission(AccessController.java:1042) ~[?:?]",
es02    | "at java.lang.SecurityManager.checkPermission(SecurityManager.java:408) ~[?:?]",
es02    | "at java.lang.SecurityManager.checkRead(SecurityManager.java:747) ~[?:?]",
es02    | "at sun.nio.fs.UnixChannelFactory.open(UnixChannelFactory.java:255) ~[?:?]",
es02    | "at sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:143) ~[?:?]",
es02    | "at sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:156) ~[?:?]",
es02    | "at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:217) ~[?:?]",
es02    | "at java.nio.file.Files.newByteChannel(Files.java:373) ~[?:?]",
es02    | "at java.nio.file.Files.newByteChannel(Files.java:424) ~[?:?]",
es02    | "at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420) ~[?:?]",
es02    | "at java.nio.file.Files.newInputStream(Files.java:158) ~[?:?]",
es02    | "at org.elasticsearch.xpack.core.ssl.CertParsingUtils.readCertificates(CertParsingUtils.java:93) ~[?:?]",
es02    | "at org.elasticsearch.xpack.core.ssl.CertParsingUtils.readCertificates(CertParsingUtils.java:86) ~[?:?]",
es02    | "at org.elasticsearch.xpack.core.ssl.PEMTrustConfig.createTrustManager(PEMTrustConfig.java:45) ~[?:?]",
es02    | "at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:382) ~[?:?]",
es02    | "at java.util.HashMap.computeIfAbsent(HashMap.java:1133) ~[?:?]",
es02    | "at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSSLConfigurations$2(SSLService.java:426) ~[?:?]",
es02    | "at java.util.HashMap.forEach(HashMap.java:1333) ~[?:?]",
es02    | "at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:423) ~[?:?]",
es02    | "at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:119) ~[?:?]",
es02    | "at org.elasticsearch.xpack.core.XPackPlugin.<init>(XPackPlugin.java:143) ~[?:?]",
es02    | "at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]",
es02    | "at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]",
es02    | "at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]",
es02    | "at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]",
es02    | "at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]",
es02    | "at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:605) ~[elasticsearch-7.2.0.jar:7.2.0]",
es02    | "at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556) ~[elasticsearch-7.2.0.jar:7.2.0]",
es02    | "at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[elasticsearch-7.2.0.jar:7.2.0]",
es02    | "at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:163) ~[elasticsearch-7.2.0.jar:7.2.0]",
es02    | "at org.elasticsearch.node.Node.<init>(Node.java:307) ~[elasticsearch-7.2.0.jar:7.2.0]",
es02    | "at org.elasticsearch.node.Node.<init>(Node.java:251) ~[elasticsearch-7.2.0.jar:7.2.0]",
es02    | "at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) ~[elasticsearch-7.2.0.jar:7.2.0]",
es02    | "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.2.0.jar:7.2.0]",
es02    | "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.2.0.jar:7.2.0]",
es02    | "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.2.0.jar:7.2.0]",

Below is my yml

version: '2.2'
services:
  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.2.0
    container_name: es02
    environment:
      - node.name=es02
      - http.port=9205
      - discovery.seed_hosts=es02
      - cluster.initial_master_nodes=es02
      - cluster.name=docker-cluster-test
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - xpack.license.self_generated.type=trial 
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=/apps/certs/es02/es02.key
      - xpack.security.http.ssl.certificate_authorities=/apps/certs/ca/ca.crt
      - xpack.security.http.ssl.certificate=/apps/certs/es02/es02.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.verification_mode=certificate 
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata01test:/opt/mad/tools/data1
    #ports:
     # - 9202:9202
    networks:
      - esnet

volumes:
  esdata01test:
    driver: local

networks:
  esnet:

Please help me on this. Thanks!

2

Is there any specific reason that you can't follow the instructions exactly as described in the docs ? I don't see the volumes mounted as described in the docs

3

Hi @ikakavas,

In the docs they have mentioned volumes as /usr/share/elasticsearch path and i dont see elasticsearch under /usr/share.

So i asked in case of docker when we pull image from docker.elastic.co/elasticsearch/elasticsearch:7.2.0 i want to know where the files like config,bin and files related to elasticsearch,plugins will be stored in linux.

When we download elasticsearch for linux will get all the files under elasticsearch-7.2.0 similarly in case of docker where it will be stored.

Please help me on this. Thanks!

4

Read in https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html to see how named volumes are used for the configuration directories in our docker images and see https://docs.docker.com/compose/compose-file/#volume-configuration-reference and https://docs.docker.com/storage/volumes/ on what volumes are and how they are used/configured in docker and docker-compose files.

5

Thanks for the info @ikakavas.
I will check and come back here if i have any issues.

6

Hi @ikakavas,

I have followed the blog - https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls-docker.html and everything is working fine and elastic accessible through curl but not from browser. I have following queries :

Q1; How to access elastic from browser ?
Q2: Im not able to access elastic from docker ip or host ip. Below is my yml for hosting elastic on docker or linux host ip

version: '2.2'
services:
  es04:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.2.0
    container_name: es04
    environment:
      - node.name=es04
      #- discovery.seed_hosts=es02
      - transport.tcp.port=9309
      - transport.publish_port=9309
      - cluster.initial_master_nodes=es04
      - network.host=<docker or host ip>
      - network.publish_host=<docker or host ip>
      - cluster.name=docker-test
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata04:/usr/share/elasticsearch/data
    ports:
      - 9201:9201
    networks:
      - esnet
volumes:
  esdata04:
    driver: local
 
networks:
  esnet:

And errors logs looks like below:

es04    | "stacktrace": ["org.elasticsearch.bootstrap.StartupException: BindTransportException[Failed to bind to [9309]]; nested: BindException[Cannot assign requested address];",
es04    | "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.2.0.jar:7.2.0]",
es04    | "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.2.0.jar:7.2.0]",
es04    | "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.2.0.jar:7.2.0]",
es04    | "at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.2.0.jar:7.2.0]",
es04    | "at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.2.0.jar:7.2.0]",
es04    | "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.2.0.jar:7.2.0]",
es04    | "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.2.0.jar:7.2.0]",
es04    | "Caused by: org.elasticsearch.transport.BindTransportException: Failed to bind to [9309]",
es04    | "at org.elasticsearch.transport.TcpTransport.bindToPort(TcpTransport.java:389) ~[elasticsearch-7.2.0.jar:7.2.0]",
es04    | "at org.elasticsearch.transport.TcpTransport.bindServer(TcpTransport.java:355) ~[elasticsearch-7.2.0.jar:7.2.0]",
es04    | "at org.elasticsearch.transport.netty4.Netty4Transport.doStart(Netty4Transport.java:136) ~[?:?]",
es04    | "at org.elasticsearch.xpack.core.security.transport.netty4.SecurityNetty4Transport.doStart(SecurityNetty4Transport.java:81) ~[?:?]",
es04    | "at org.elasticsearch.xpack.security.transport.netty4.SecurityNetty4ServerTransport.doStart(SecurityNetty4ServerTransport.java:43) ~[?:?]",
es04    | "at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:59) ~[elasticsearch-7.2.0.jar:7.2.0]",
es04    | "at org.elasticsearch.transport.TransportService.doStart(TransportService.java:230) ~[elasticsearch-7.2.0.jar:7.2.0]",
es04    | "at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:59) ~[elasticsearch-7.2.0.jar:7.2.0]",
es04    | "at org.elasticsearch.node.Node.start(Node.java:662) ~[elasticsearch-7.2.0.jar:7.2.0]",
es04    | "at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:273) ~[elasticsearch-7.2.0.jar:7.2.0]",
es04    | "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:358) ~[elasticsearch-7.2.0.jar:7.2.0]",
es04    | "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.2.0.jar:7.2.0]",
es04    | "... 6 more",
es04    | "Caused by: java.net.BindException: Cannot assign requested address",
es04    | "at sun.nio.ch.Net.bind0(Native Method) ~[?:?]",
es04    | "at sun.nio.ch.Net.bind(Net.java:455) ~[?:?]",
es04    | "at sun.nio.ch.Net.bind(Net.java:447) ~[?:?]",
es04    | "at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:219) ~[?:?]",
es04    | "at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:132) ~[?:?]",
es04    | "at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:563) ~[?:?]",
es04    | "at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1332) ~[?:?]",
es04    | "at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:503) ~[?:?]",
es04    | "at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:488) ~[?:?]",
es04    | "at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:984) ~[?:?]",
es04    | "at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:259) ~[?:?]",
es04    | "at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:366) ~[?:?]",
es04    | "at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) ~[?:?]",
es04    | "at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:405) ~[?:?]",
es04    | "at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) ~[?:?]",
es04    | "at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906) ~[?:?]",
es04    | "at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]",
es04    | "at java.lang.Thread.run(Thread.java:835) [?:?]"] }

Please help on the above queries. Thanks!

7

Hi @SukeshGupta

Please excuse my straightforward answer, but exposing a port from a container to the network is basic knowledge when using docker. You do this correctly (9201:9201), but 9201 is not a port from Elasticsearch. I do not see that you configure Elasticsearch to use this port.

The default ports for Elasticsearch are 9200 (HTTP) and 9300 (Transport). You do configure the transport port as 9309, but leave the HTTP-port untouched. You would need to reconfigure the HTTP-port to use 9201 or change the exposed port.

Hope this helps.

MiTschMR

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.