Confused with running Logstash in docker

Elastic Stack Logstash
1

Hello everyone,

I'm looking for some instruction with how to run logstash in docker. I can get elasticsearch running okay but then im not sure how I'm supposed to start logstash. I'm also kinda confused about how I link my local system to the container.

if anyone could help me out with the commands that would be very, very helpful to me. I'm running this one a windows machine. I know I need to do something along the lines of "docker run -d --name logstash -p 5400:5400 -v (confused here)"

So how do I get my docker conf file into the container?
Secondly, how does the container see my log files?

My conf file is currently sitting in the directory C:\Users\e-team\Documents\elk_docker
in my conf file the path Im using to read my logs is "C:/Users/e-team/Documents/ELK/logs/**/logs/*.log"

So how do I get the container to see all of this information?

2

Use this command and if this not works , try to get an idea from this what I am trying to do (mount the directories from windows to the docker container)

docker run -d --name logstash -p 5044:5044 --link elasticsearch:elasticsearch -v /c/Users/e-team/Documents/ELK/logs/**/logs/:/tmp/ -v /c/Users/e-team/Documents/elk_docker/:/etc/logstash/conf.d/ logsta
sh -f /etc/logstash/conf.d/logstash.conf

3

okay so docker seems to be accepting this command, how do I know if it is actually doing anything? I tried going out to localhost:9200/_cat/indices to see if it was loading stuff but for some reason its asking for a username and a password to get in? its never done that before

edit: i got the username and password. doesnt look like my index i specified in my .conf file is there

Second edit for more info:

my conf file output section is setup like this, i know this works on my local machine but im not all too sure about it being in docker ocntainers, do i need to modify this?

output {                                                        #The output section tells where the output will go
    elasticsearch{                                              #Output will be going to elasticsearch  
      hosts => ["localhost:9200"]                               #name of the index that the output will be stored
      index => "finalindex"
    }
    stdout { codec => rubydebug }                               #Outputs the contents of the message to the command prompt
}
4

What is your local machine from which you are calling logs and conf file ? Is it Windows or linux ?

5

My host machine that contains the logs and config file and is running the containers is windows

6

Everytime you mount a directory from windows host to docker machine , your file might experience some changes.
Kindly check and if it's yes then use command : dos2unix
to make that file usable for docker ...seems like you are facing the same issue

7

Now I'm getting this error

ERROR logstash.agent - Pipeline aborted due to error {:exception=>#<ArgumentError: File paths must be absolute, relative path specified: C:/Users/e-team/Documents/ELK/logs/**/logs/*.log>

Not sure how im supposed to read all the different files if i cant use a relative path?

8

Well, as far as i know need to give absolute path , path doesn't include the file name as I can see u have given .

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.