Filebeat docker container - Can we modify filebeat.yml file using docker ENTRYPOINT

Elastic Stack Beats
,
1

Hi, we are using docker/ECS filebeat containers, currently as part of docker build we copy filebeat-(aws_account).yml to /usr/share/filebeat/filebeat.yml with account specific values. But while doing this process we have to use multiple ECR's. We want to have single docker image with entrypoint/cmd so that it copy's the account specific file to filebeat.yml as part of release. So we are looking for something like, filebeat.yml to be updated when starting docker container instead of updating it while building docker image.

2

So basically you're asking if there's a way to launch the container but before starting the filebeat service, doing additional dynamic configuration based on an environment variable?

3

Yes, that's true and we tried it below but it is not working...

Dockerfile

FROM docker.elastic.co/beats/filebeat:8.2.2
USER root
ARG AWS_ACCESS_KEY_ID
ARG AWS_SECRET_ACCESS_KEY
ENV AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
ENV AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
RUN chown root:filebeat /usr/share/filebeat -R
COPY ./filebeat1.yml /usr/share/filebeat1.yml
COPY ./filebeat2.yml /usr/share/filebeat2.yml
COPY ./filebeat3.yml /usr/share/filebeat3.yml
COPY ./filebeat4.yml /usr/share/filebeat4.yml
COPY ./modules.d-1/*.* /usr/share/modules.d-1/
COPY ./modules.d-2/*.* /usr/share/modules.d-2/
COPY ./modules.d-3/*.* /usr/share/modules.d-3/
COPY ./modules.d-4/*.* /usr/share/modules.d-4/



RUN chmod 777 /usr/share/filebeat-*.yml
RUN chmod 777 /usr/share/modules.d-*
RUN chmod 777 /usr/share/modules.d-*/*



COPY ./start.sh /



USER filebeat



ENTRYPOINT ["sh", "/start.sh"]

start.sh script…

#!/bin/bash
cp -r /usr/share/filebeat1.yml /usr/share/filebeat/filebeat.yml
4

Can you please post the logs or other data that leads you to believe your method is not working correctly?

Also, can you post your complete Dockerfile and start.sh scripts? There are important bits missing here such as your "CMD" or "ENTRYPOINT" or how you are starting Filebeat from the script.

5

Hi, Below is the docker file with start.sh script. We are running this in ECS so when Azure DevOps Release pipeline tries to update ECS container then new task is created and starts running.. within a minute it fails/stopped. Log says it cannot copy from /usr/share/filebeat1.yml to /usr/share/filebeat/filebeat.yml as it is trying to execute commands within the filebeat.yml file. Commands not found for all the commands in filebeat.yml.

So question here is.. Can we modify filebeat.yml file while starting docker container or do we need to update it while building docker image itself as it is not allowing to update when trying with entrypoint.

Docker file:
FROM docker.elastic.co/beats/filebeat:8.2.2
USER root
ARG AWS_ACCESS_KEY_ID
ARG AWS_SECRET_ACCESS_KEY
ENV AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
ENV AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
RUN chown root:filebeat /usr/share/filebeat -R
COPY ./filebeat1.yml /usr/share/filebeat1.yml
COPY ./filebeat2.yml /usr/share/filebeat2.yml
COPY ./filebeat3.yml /usr/share/filebeat3.yml
COPY ./filebeat4.yml /usr/share/filebeat4.yml
COPY ./modules.d-1/. /usr/share/modules.d-1/
COPY ./modules.d-2/. /usr/share/modules.d-2/
COPY ./modules.d-3/. /usr/share/modules.d-3/
COPY ./modules.d-4/. /usr/share/modules.d-4/

RUN chmod 777 /usr/share/filebeat-.yml
RUN chmod 777 /usr/share/modules.d-
RUN chmod 777 /usr/share/modules.d-/

COPY ./start.sh /

USER filebeat

ENTRYPOINT ["sh", "/start.sh"]

start.sh script:
#!/bin/bash
cp -r /usr/share/filebeat1.yml /usr/share/filebeat/filebeat.yml

6

@ramreddy when you say "cannot copy from /usr/share/filebeat1.yml to /usr/share/filebeat/filebeat.yml" what is the exact error from log?

7

I am getting permission denied error.

8

SOLUTION: Run the chown AFTER the copy commands so the permissions are set correctly when the start.sh script gets run by the filebeat user.

RUN chown filebeat:filebeat /usr/share/filebeat -R

Also, the start.sh script needs to have its final command be to run the filebeat executable, otherwise the container will exit immediately since no further commands to execute.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.