Logstash data storage


(Gaurav Agarwal) #1

Hello all
I am running my ELk stack on docker so whenever my logstash loses connection with elasticsearch then the data which is getting parsed where is it stored.
According to my observation it is piling up the logstash containers, so is there a particular directory inside the conatiner which it is supposed to pile up if yes then will mounting the directory as a volume outside the conatiner will it help?

Regards
Gaurav Agarwal


(Magnus Bäck) #2

According to my observation it is piling up the logstash containers, so is there a particular directory inside the conatiner which it is supposed to pile up

The data directory. See https://www.elastic.co/guide/en/logstash/current/dir-layout.html.

if yes then will mounting the directory as a volume outside the conatiner will it help?

Yes.


(Gaurav Agarwal) #3

Is it the data directory which should be mounted outside since I m not very sure where is the storage piling up is happening?


(Magnus Bäck) #4

Is it the data directory which should be mounted outside

I already answered that question.

since I m not very sure where is the storage piling up is happening?

But that's easy to check, isn't it?


(Gaurav Agarwal) #5

It says the data directory is not writable.

[2018-05-28T09:35:49,874][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<ArgumentError: Path "/usr/share/logstash/data" must be a writable directory. It is not writable.>,


(Magnus Bäck) #6

And this happens when mounting a directory from the host...?


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.