ELK creates lots of hidden files

[nousees (Maria)]

Hello everybody,

I've just installed logstash, elasticsearch and kibana. The issue is that I've found lots of hidden files with this format:

.1015766.51411.253032
.200184.53366.432265

I've found it in several directories and the number of them is increasing.

Please, do you know how can I control those files to store in a particular directory?
Who does create them? I'm not sure if logstash or elasticsearch.

Is there some best practices for that?

Thanks so much!

[JKhondhu (Jymit Singh Khondhu)]

Hi,

What version of the Elasticstack if being run here? Please share the directories under which you have found said hidden files? What OS is this currently running on?

[nousees (Maria)]

Hi JKhondhu,

I'm using 5.4.2 version of Elasticsearch under Debian 8.8. The directories under which I've found those hidden files are /etc/logstash/conf.d/ (where I've got the config file for logstash) and /var/log, where I've got the log file I pass to logstash.

Thanks!

[JKhondhu (Jymit Singh Khondhu)]

Hi,

Please share your logstash.yml

Right now I am going with https://www.elastic.co/guide/en/logstash/current/persistent-queues.html#persistent-queues - "you can enable persistent queues to buffer events on disk"

[nousees (Maria)]

This is Logstash.yml content:

path.data: /var/lib/logstash
path.config: /etc/logstash/conf.d
path.logs: /var/log/logstash

[nousees (Maria)]

Hi,

Well, I've just figured out this weird issue. The thing was that in logstash.conf I've got "sincedb_path" param at this way:

sincedb_path => ""

This was the only way I found to force logstash to parse from the begining of the document for each execution because start_position => "beginning" didn't work for me.

So, due to sincedb_patch logstash creates a file from each execution.

Thanks for your help!