# First time trying to run elk with docker-compose

**URL:** <https://discuss.elastic.co/t/first-time-trying-to-run-elk-with-docker-compose/246368>\
**Category:** Kibana\
**Tags:** docker\
**Created:** [August 25, 2020, 11:45pm UTC](https://discuss.elastic.co/t/first-time-trying-to-run-elk-with-docker-compose/246368 "2020-08-25T23:45:42Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![leostereo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leostereo/32/74891_2.png) [@leostereo](https://discuss.elastic.co/u/leostereo)\
**Post date:** [August 25, 2020, 11:45pm UTC](https://discuss.elastic.co/t/first-time-trying-to-run-elk-with-docker-compose/246368/1 "2020-08-25T23:45:43Z")

</div>

Hi guys, im very interested setting up a centralized log server using docker / docker-compose:  
After following oficial , and no oficial documentation:  
[https://www.elastic.co/guide/en/elastic-stack-get-started/current/get-started-docker.html](https://www.elastic.co/guide/en/elastic-stack-get-started/current/get-started-docker.html)  
[https://github.com/deviantony/docker-elk](https://github.com/deviantony/docker-elk)

Got same result.  
After running:  
`docker-compose up`  
Lines begin to run on the screen , but im no sure when stack is ready to use:  
After one hour I connect to server on a second console and can see:

```
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5a2a7760cd8f docker-elk_kibana "/usr/local/bin/dumb…" 9 minutes ago Up 9 minutes 0.0.0.0:5601->5601/tcp docker-elk_kibana_1
352e379055d9 docker-elk_logstash "/usr/local/bin/dock…" 9 minutes ago Up 9 minutes 0.0.0.0:5000->5000/tcp, 0.0.0.0:9600->9600/tcp, 0.0.0.0:5000->5000/udp, 5044/tcp docker-elk_logstash_1
2c27c53d6893 docker-elk_elasticsearch "/tini -- /usr/local…" 9 minutes ago Up 9 minutes 0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp docker-elk_elasticsearch_1'

```

Everything seems ok from there, but when try connecting from browse , I got:  
`Kibana server is not ready yet`

I have no idea where to beging debugin this ... some questions came to my mind:  
How long does the stack to be ready ? How should I check ?  
Where should I look for errors ?

Also read about assign 4Gb to the stack.  
This is what I did:  
`sysctl -w vm.max_map_count=262144`  
Im trying to read about this, so any advice would be wellcome.

btw:  
I did not change any config file at all.  
Im running statck on fresh centos 7 vm with 8Gb memory.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [August 26, 2020, 5:08am UTC](https://discuss.elastic.co/t/first-time-trying-to-run-elk-with-docker-compose/246368/2 "2020-08-26T05:08:16Z")

</div>

Hi @leostereo

It should only take a couple minutes to come up

Can you post your docker compose file?

Did you look at the log and see and errors? If you run docker compose from the command line the logs get directed to stdout / console. Look for errors.

If you are running on a machine with 8GB of RAM you should be OK assuming you there are not other processes that have already consumed the memory or you did not increase the Elasticsearch heap setting up too high.

Did you try the following commands to see in elasticsearch came up?

`curl -X GET "localhost:9200`  
`curl -X GET "localhost:9200/_cat/nodes?v&pretty`

You can try this compose file I just ran it ... it works. It is a sample it is not made for production but its a simple test.

You run it with the following command.

`TAG=7.9.0 docker-compose`

```
---
  version: '3.1'
  services:
    elasticsearch:
      image: docker.elastic.co/elasticsearch/elasticsearch:${TAG}
      environment: ['ES_JAVA_OPTS=-Xms2g -Xmx2g','bootstrap.memory_lock=true','discovery.type=single-node', 'http.host=0.0.0.0', 'transport.host=127.0.0.1']
      ports: ['127.0.0.1:9200:9200']
      networks: ['stack']
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536
          hard: 65536
  
    kibana:
      image: docker.elastic.co/kibana/kibana:${TAG}
      ports: ['127.0.0.1:5601:5601']
      networks: ['stack']
      depends_on: ['elasticsearch']
  
  networks: {stack: {}}
```

---

<div class="post-metadata">

**Author:** ![leostereo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leostereo/32/74891_2.png) [@leostereo](https://discuss.elastic.co/u/leostereo)\
**Post date:** [August 26, 2020, 11:51pm UTC](https://discuss.elastic.co/t/first-time-trying-to-run-elk-with-docker-compose/246368/3 "2020-08-26T23:51:19Z")

</div>

Dear @stephenb  
Thanks for your words.  
The yml file you share works ok on my local pc but not on my remote vm.

The file im using is: (I did not change nothing here, just copy/paste it)

```
version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.9.0
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es02,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - elastic

  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.9.0
    container_name: es02
    environment:
      - node.name=es02
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data02:/usr/share/elasticsearch/data
    ports:
      - 9201:9201
    networks:
      - elastic

  es03:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.9.0
    container_name: es03
    environment:
      - node.name=es03
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es02
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data03:/usr/share/elasticsearch/data
    ports:
      - 9202:9202
    networks:
      - elastic

  kib01:
    image: docker.elastic.co/kibana/kibana:7.9.0
    container_name: kib01
    ports:
      - 5601:5601
    environment:
      ELASTICSEARCH_URL: http://es01:9200
      ELASTICSEARCH_HOSTS: http://es01:9200
    networks:
      - elastic

volumes:
  data01:
    driver: local
  data02:
    driver: local
  data03:
    driver: local

networks:
  elastic:
    driver: bridge

```

Also , the output you mention shows an error:

```
[root@deploy ~]# curl -X GET "localhost:9200"
{
  "name" : "es01",
  "cluster_name" : "es-docker-cluster",
  "cluster_uuid" : "_na_",
  "version" : {
    "number" : "7.9.0",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "a479a2a7fce0389512d6a9361301708b92dff667",
    "build_date" : "2020-08-11T21:36:48.204330Z",
    "build_snapshot" : false,
    "lucene_version" : "8.6.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
[root@deploy ~]# curl -X GET "localhost:9200/_cat/nodes?v&pretty"
{
  "error" : {
    "root_cause" : [
      {
        "type" : "master_not_discovered_exception",
        "reason" : null
      }
    ],
    "type" : "master_not_discovered_exception",
    "reason" : null
  },
  "status" : 503
}

```

About log error ... this is what I can show (trunked):

```
[root@deploy ~]# docker logs 187 | grep -E "error|invalid"
[BABEL] Note: The code generator has deoptimised the styling of /usr/share/kibana/x-pack/plugins/canvas/server/templates/pitch_presentation.js as it exceeds the max of 500KB.
{"type":"log","@timestamp":"2020-08-26T23:29:32Z","tags":["warning","plugins","security","config"],"pid":6,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"}
{"type":"log","@timestamp":"2020-08-26T23:29:32Z","tags":["warning","plugins","reporting","config"],"pid":6,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
{"type":"log","@timestamp":"2020-08-26T23:29:34Z","tags":["error","elasticsearch","data"],"pid":6,"message":"Request error, retrying\nGET http://es01:9200/_nodes?filter_path=nodes.*.version%2Cnodes.*.http.publish_address%2Cnodes.*.ip => connect EHOSTUNREACH 172.26.0.4:9200"}
{"type":"log","@timestamp":"2020-08-26T23:29:34Z","tags":["error","elasticsearch","monitoring"],"pid":6,"message":"Request error, retrying\nGET http://es01:9200/_xpack => connect EHOSTUNREACH 172.26.0.4:9200"}
{"type":"log","@timestamp":"2020-08-26T23:29:34Z","tags":["error","elasticsearch","monitoring"],"pid":6,"message":"Request error, retrying\nGET http://es01:9200/_xpack => connect EHOSTUNREACH 172.26.0.4:9200"}
{"type":"log","@timestamp":"2020-08-26T23:29:35Z","tags":["warning","plugins","licensing"],"pid":6,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"}
{"type":"log","@timestamp":"2020-08-26T23:29:37Z","tags":["error","savedobjects-service"],"pid":6,"message":"Unable to retrieve version information from Elasticsearch nodes."}
{"type":"log","@timestamp":"2020-08-26T23:29:37Z","tags":["warning","plugins","licensing"],"pid":6,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"}
{"type":"log","@timestamp":"2020-08-26T23:30:04Z","tags":["warning","plugins","licensing"],"pid":6,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"}
{"type":"log","@timestamp":"2020-08-26T23:30:34Z","tags":["warning","plugins","licensing"],"pid":6,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"}
{"type":"log","@timestamp":"2020-08-26T23:31:04Z","tags":["warning","plugins","licensing"],"pid":6,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"}
{"type":"log","@timestamp":"2020-08-26T23:31:34Z","tags":["warning","plugins","licensing"],"pid":6,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"}
{"type":"log","@timestamp":"2020-08-26T23:32:04Z","tags":["warning","plugins","licensing"],"pid":6,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"}

```

mmmm .... thanks again.  
im lost here.  
LEo

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [August 27, 2020, 2:55am UTC](https://discuss.elastic.co/t/first-time-trying-to-run-elk-with-docker-compose/246368/4 "2020-08-27T02:55:54Z")

</div>

Hi @leostereo

Closer look at :  
[https://www.elastic.co/guide/en/elasticsearch/reference/7.9/docker.html#\_set\_vm\_max\_map\_count\_to\_at\_least\_262144](https://www.elastic.co/guide/en/elasticsearch/reference/7.9/docker.html#_set_vm_max_map_count_to_at_least_262144)

I see you ran  
`sysctl -w vm.max_map_count=262144`

I believe what you really need to do is the following

The `vm.max_map_count` setting should be set permanently in `/etc/sysctl.conf` which means you need to edit the `/etc/sysctl.conf` file and append `vm.max_map_count=262144` to the bottom otherwise it is not taking affect.

I would clean up everything after you set that and try again

That is what I did to get your compose file working.

I waited a couple minutes then ran the following and everything looks good!

```
$ curl localhost:9200/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.32.5 34 92 9 0.33 0.63 1.27 dilmrt - es02
192.168.32.2 51 92 9 0.33 0.63 1.27 dilmrt - es03
192.168.32.3 37 92 9 0.33 0.63 1.27 dilmrt * es01
```

---

<div class="post-metadata">

**Author:** ![leostereo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leostereo/32/74891_2.png) [@leostereo](https://discuss.elastic.co/u/leostereo)\
**Post date:** [August 27, 2020, 7:56pm UTC](https://discuss.elastic.co/t/first-time-trying-to-run-elk-with-docker-compose/246368/5 "2020-08-27T19:56:24Z")

</div>

Thanks @stephenb  
You were right, about sysctrl.conf.  
Now error message is different but it seems closer to work.

```
[root@deploy ~]# curl localhost:9200/_cat/nodes?v
{"error":{"root_cause":[{"type":"master_not_discovered_exception","reason":null}],"type":"master_not_discovered_exception","reason":null},"status":503} 

```

Im reading about this error message.  
Seems to be an internal comunication problem between stack elements.  
Any update I will let you know.  
Leandro.

---

<div class="post-metadata">

**Author:** ![leostereo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leostereo/32/74891_2.png) [@leostereo](https://discuss.elastic.co/u/leostereo)\
**Post date:** [August 29, 2020, 5:16pm UTC](https://discuss.elastic.co/t/first-time-trying-to-run-elk-with-docker-compose/246368/6 "2020-08-29T17:16:29Z")

</div>

dear @stephenb, thanks for your words.  
Final idea of elk , is to collect netflow so ... I change to regular installation.  
ELK stack is working now.  
Thanks.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [August 29, 2020, 5:20pm UTC](https://discuss.elastic.co/t/first-time-trying-to-run-elk-with-docker-compose/246368/7 "2020-08-29T17:20:18Z")

</div>

@leostereo  
Good to hear.  
Did you ever get the docker working?  
That said ELK direct on host should be higher performing.  
So you have a 3 node cluster working with direct installation?

Make sure you checkout the Filebeat Netflow Module  
[https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-netflow.html](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-netflow.html)

Good Luck!

---

<div class="post-metadata">

**Author:** ![leostereo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leostereo/32/74891_2.png) [@leostereo](https://discuss.elastic.co/u/leostereo)\
**Post date:** [August 29, 2020, 5:43pm UTC](https://discuss.elastic.co/t/first-time-trying-to-run-elk-with-docker-compose/246368/8 "2020-08-29T17:43:02Z")

</div>

> [@stephenb](#):
>
> So you have a 3 node cluster working with direct installation?

What do you mean with 3 nodes?  
This I what i can see:

```
[root@deploy ~]# curl -X GET "localhost:9200/_cat/nodes?v&pretty"
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
127.0.0.1 54 79 99 7.38 7.06 7.14 dilmrt * deploy

```

How can I check the overall status of the stack?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [August 29, 2020, 5:53pm UTC](https://discuss.elastic.co/t/first-time-trying-to-run-elk-with-docker-compose/246368/9 "2020-08-29T17:53:20Z")

</div>

That is 1 node, that will be fine for POC. Best practice a production cluster will have more than 1 node for performance and HA and scale.

```auto
curl -X GET "localhost:9200/"
curl -X GET "localhost:9200/_cat/health"

```

Health will most likely show Yellow because there is no replication (no copies of the data so you are at risk of losing data if the node dies)

Next you should log into Kibana and poke around.

At this point I would recommend some of the free training and webinars and blogs, you have much to learn.  
[https://www.elastic.co/videos/](https://www.elastic.co/videos/)  
[https://www.elastic.co/training/free](https://www.elastic.co/training/free)  
[https://www.elastic.co/blog/](https://www.elastic.co/blog/)

And if you are going to be devops for a Production Elastic I would get some actual real training  
[https://www.elastic.co/training/](https://www.elastic.co/training/)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 26, 2020, 5:53pm UTC](https://discuss.elastic.co/t/first-time-trying-to-run-elk-with-docker-compose/246368/10 "2020-09-26T17:53:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.