Filebeat docker help with creating the container

[zebulyon]

Hello, complete beginner with the ELK-stack here so this might be really trivial.

I'm using the docker image docker.elastic.co/beats/filebeat:7.6.0 as well as the other corresponding images of the ELK-stack all of version 7.6.0.

These are the three version of commands I'm using to try and create the container:

VER 1

docker run -d \
  --name=filebeat \
  --user=root \
  --volume="$(pwd)/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro" \
  --volume="$(pwd)/filebeat_input.log:/usr/share/filebeat/sample.log:ro" \
  --volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" \
  --volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
  --link 64bc6d9b5663:elasticsearch \
  docker.elastic.co/beats/filebeat:7.6.0 filebeat \
  -E output.elasticsearch.hosts=["elasticsearch:9200"]

VER 1 give me this as the error response when trying to run it.

Exiting: error loading config file: config file ("filebeat.yml") must be owned by the user identifier (uid=0) or root

VER 2
So read up some and added the setup setup -strict.perms=false

  docker run -d \
  --name=filebeat \
  --user=root \
  --volume="$(pwd)/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro" \
  --volume="$(pwd)/filebeat_input.log:/usr/share/filebeat/sample.log:ro" \
  --volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" \
  --volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
  --link [CONTAINER_ID_ELASTICSEARCH]:elasticsearch \
  docker.elastic.co/beats/filebeat:7.6.0 setup -strict.perms=false filebeat \
  -E output.elasticsearch.hosts=["elasticsearch:9200"]

While VER 2 gives me this as my response:

Overwriting ILM policy is disabled. Set `setup.ilm.overwrite:true` for enabling.

Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to http://localhost:5601/api/status fails: fail to execute the HTTP GET request: Get http://localhost:5601/api/status: dial tcp 127.0.0.1:5601: connect: connection refused. Response: .

VER 3
Thought that ok it needs to be linked to Kibana as well so added --link [CONTAINER_ID_KIBANA]:kibana \

docker run -d \
  --name=filebeat \
  --user=root \
  --volume="$(pwd)/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro" \
  --volume="$(pwd)/filebeat_input.log:/usr/share/filebeat/sample.log:ro" \
  --volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" \
  --volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
  --link [CONTAINER_ID_ELASTICSEARCH]:elasticsearch \
  --link [CONTAINER_ID_KIBANA]:kibana \
  docker.elastic.co/beats/filebeat:7.6.0 filebeat \
  -E output.elasticsearch.hosts=["elasticsearch:9200"]

Which gives me the same response as VER 1:

Exiting: error loading config file: config file ("filebeat.yml") must be owned by the user identifier (uid=0) or root

This is my filebeat.yml file:

filebeat.config:
  modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false

filebeat.inputs:
  type: log
  enabled: true
  paths: ./sample.log

processors:
- add_cloud_metadata: ~
- add_docker_metadata: ~

output.elasticsearch:
  hosts: '${localhost:elasticsearch:9200}'

All of these are based on the examples provided in https://www.elastic.co/guide/en/beats/filebeat/current/running-on-docker.html

Thanks in advance to anyone that can assist me,

[Ayush_Mathur]

@zebulyon can you check the ownership of the configmap (filebeat.yml). The file must be owned by the root, something like:

-rw-r--r--. 1 root root

[zebulyon]

Thank you for the quick response.

When using the ls -l filebeat.yml command I get this response

-rw-r--r-- 1 filebeat filebeat 295 Feb 18 16:57 filebeat.yml

Have tried using the chown command found here

https://www.elastic.co/guide/en/beats/libbeat/5.3/config-file-permissions.html

[Ayush_Mathur]

@zebulyon you don't have to change the ownership of filebeat directory, but for the config file, i.e. filebeat.yml.

You should try something like:
chown root:root filebeat.yml
followed by:
chmod og-w filebeat.yml

Also, I'd suggest you to follow the Elastic documentation for your version of stack (you are using 7.x but referring the docs for 5.x): https://www.elastic.co/guide/en/beats/libbeat/7.x/config-file-permissions.html

[zebulyon]

I tried using your commands and some other similar ones but I still get the same result
As long as I try to simply run the command filbeat

I have also tried the command filebeat setup -E -strict.perms=false but I get the exact same result.
Exiting: error loading config file: config file ("filebeat.yml") must be owned by the user identifier (uid=0) or root

[zebulyon]

I have managed to solve the problem.
It seems like when copying the file into the docker container using this command
--volume="$(pwd)/filebeat_input.log:/usr/share/filebeat/sample.log:ro" \
This leads to the file filebeat.yml somehow being given a permission requirement that the container is unable to provide.

I am however confused to why this does not apply to the file sample.log considering it uses the same command :
--volume="$(pwd)/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro" \