Logstash parses log files again: is that recurrent?

Elastic Stack Logstash
1

Hi all,

Our environment is the following:

  • OS: Red Hat Enterprise Linux Server release 7.3
  • Logstash version: 6.3.0
  • ES version: 6.3.2

Our problem: after having stopped the Logstash process for modification purpose in the .conf file, we started the process again but noticed that log files already parsed are treated again; we are using a sincedb file which is not corrupted. The process was working well since a few months until now.

It's the second time we are facing such a problem. Last time, we had to reindex all log files.

Questions:

  • does anybody already encountered such a problem?
  • if yes, what did you do to come back to a normal situation? (to avoid parsing again all log files)

Many thanks in advance.

2

What does your config look like? What type of storage are you reading from?

3

Hello Christian,

We are using the file input plugin. The log files are extracted from a mount point folder path. It is the same with the sincedb file.

file {
	path => "${LS_XXX_PATH}"
	sincedb_path => "${LS_XXX_SINCEDBPATH}"
	start_position => "beginning"
	add_field => {"Origin" => "XXX"}
        discover_interval => 5
	close_older => 60
	codec => multiline {
		pattern => "^(\[%{YEAR}-%{MONTHNUM}-%{MONTHDAY})"
		negate => true
		what => previous
	}
}

And the definition of the environment variables:
LS_XXX_PATH=/data_nfs/xxx/*.log
LS_XXX_SINCEDBPATH=$LS_SINCEDB/XXX.db

4

I know that there have been some issues reading from network volumes in the past, but am not sure what you are experiencing could be attributed to that. There seems to have been some improvements added in Logstash 6.4, but I will have to leave it to someone more knowledgeable about this to comment further.

5

Thank you, Christian.

I can see for example that the same file is parsed and referenced twice in the sincedb with a different minor version:
12223334 0 38
...
12223334 0 39

Do you know what does this minor version (38 and 39) refer to?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.