Lost data on docker

Elastic Stack Elasticsearch
1

hello

this my docker-compose :

  search:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.8.1
    environment:
      - container_name:es01
      - bootstrap.memory_lock:true
      - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
      - "discovery.type=single-node"
      - "MAX_LOCKED_MEMORY=unlimited"
      - "LimitMEMLOCK=infinity"
      - discovery.seed_hosts:["127.0.0.1:9300"]
      - cluster.initial_master_nodes:["es01","search"]
    deploy:
      mode: replicated
      replicas: 1
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - /opt/elastic:/usr/share/elasticsearch/data
    deploy:
      placement:
        constraints: [node.role == manager]
    ports:
      - 0.0.0.0:9200:9200

sometimes my elastic search data will erase, I do not the reason, can any one help?

2

Welcome to the community!

When you say sometimes, you mean while this container is running, or when you stop and restart or recreate this container, as those are very different things.

Do you get errors or all the indexes are gone or what?

Also, as a cluster, don't you have to expose port 9300 also?

discovery.seed_hosts:["127.0.0.1:9300"]

Kinda odd to see that, as means it uses itself to join a cluster? Or is there another container running on port 9300? If so, you still have a problem as this node also needs 9300 exposed for 'transport'.

3

when I deploy, sometimes all data in elastic will erase. as you see my elastic container is created two weeks ago:

c3eb8ff74f5c        docker.elastic.co/elasticsearch/elasticsearch:7.8.1   "/tini -- /usr/local…"   2 weeks ago         Up 2 weeks          9200/tcp, 9300/tcp        ingress_search.1.ssq1gly3kvtnqx7v0zfp62q8a

But data are gone yesterday, I can not understand why all data are gone

4

That makes no sense if the container has been up - no OTHER container also mounts this data area does it? Elasticsearch cannot share volumes so if you have more than one ES container on this VM/host, it has to have a separate mount (which is why Docker volumes are better).

5

the volume is not shared. then what would be reason?

6

Sorry, I do not know - makes no sense for a running node to lose all its data - but do you get errors, or all indexes missing or what?

How many nodes, as if you are one node and it loses data, you'll have more issues; it'll be a new cluster.

7

I'm looking at the whole environment section. I'm not 100% it does what you think it does. May I ask upon what is it based?

This bit

       - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
      - "discovery.type=single-node"
      - "MAX_LOCKED_MEMORY=unlimited"
      - "LimitMEMLOCK=infinity"

will create environment variables:

  • ES_JAVA_OPTS
  • discovery.type
  • MAX_LOCKED_MEMORY
  • LimitMEMLOCK

The rest looks more like JSON structures (which is part of YAML). I don't know how docker compose behaves in that case.

Perhaps you need to separate your environment and your configuration?

8

I believe it's a mix and yes, confusing, as it's all picked up by the start script in the container - some things are written to the config file, some to the JVM command line and/or env vars - this is how it's put in composer to feed all that.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.