Ideally, I want a logstash container to process all files under a mounted directory to the container.
Before I started trying to create a folder in which will be looked at through the logstash file, I figured to try it as is. When doing the sample given dockerfile:
FROM docker.elastic.co/logstash/logstash:7.5.2
RUN rm -f /usr/share/logstash/pipeline/logstash.conf
ADD pipeline/ /usr/share/logstash/pipeline/
ADD config/ /usr/share/logstash/config/
My logstash.conf file exists in the pipeline/ and saw it is correctly in place. I dont have anything in the config folder, as i wasnt sure if i needed to do that if i wanted to keep a vanilla instance. I was thinking that ideally I can define the inputs in the conf file... and Technically, i would be able to update the IP of the elasticsearch file from within there, but i was hoping to set it to be dynamically set based on a docker param.
I ran:
docker build -t sample-image:v0.1.0 .
docker run -i -t sample-container:v0.1.0 /bin/bash
When i was poking around, in the bash script, I was thinking that logstash would run but i didnt see it in top or ps aux
This is my desired set up:
1- Mount an existing folder on host which has 40 files to look at
2- Update the logstash.conf file to point to the mounted image point
3- start the docker instance with an optional IP (which is the target elasticsearch ip)
4- confirm it is running or not (through logs on the container)
5- shutdown and prep for use later.
Here is the link to what I was looking at: https://www.elastic.co/guide/en/logstash/current/docker-config.html
When running: docker run -t sample-container:v0.1.0 it looks like it is actually running now, but i cant quite determine if the stdout is just running the entire command output, if i can just monitor logs by running this as a headless mode
it looks like I would want to create a volume to monitor the state (I think) to apply the sincedb_path_file to that... but that is just a variable to define in the conf file.
EDIT: I think that since the app seems to be running in the window, the only thing left is to mount a directory so it can look at files, and update the conf file according. Can I also update Docker, such that i can set a property which will update the output elasticsearch ip address? I wasnt sure if i can assign a value which will do some sort of grep and replace in the conf file, or if i need to do something else.
EDIT 2: I dont think ill need the since-from-db since the files, 1 by 1 are being deleted after they are being processed. Essentially, it is going to be a dump location. If i do not have the file though, im curious if it might double-access files which were partially accessed. Maybe I should do this.